AN ANALYSIS OF VARIOUS TRAINING EVALUATION

MODELS

By *Varsha Srivastava – Research Scholar– Noida International University, Greater Noida

varshasrivastava19@gmail.com, 91 9650518945
**DR A P Dash- Director- School of Business Management - Noida International University, Greater Noida
dir.sbm@niu.edu.in,9650991299
***Anubha Maurya Walia - Founder Director Prism World (Pvt) Ltd., New Delhi 110076,
anubha@prismphilisophy.com, 91 9818446562

ABSTRACT: A profound training programme is a bridge that helps Organization employees to enhance and
develop their skill sets and perform better in their task.  It actually help in meeting the gap between skills
possess and required to perform the job. Training is an important investment in terms of time, energy and money
by Organization for the upliftment of the employee’s so that they can meet the challenges of global competition.
Therefore it become very essential for an Organization to know whether their investment is being spent
effectively or not, this requires continuous evaluation of training interventions. Evaluation of training
intervention means valuation of the impact of training on participants. This is descriptive paper comes under the
category of general review of various evaluation models. Finally in conclusive remark paper suggests the most
widely use training evaluation model in present scenario.

KEYWORDS: Evaluation, Evaluation Models, Training, Organization, Four Level Evaluation Model.

I. INTRODUCTION:

In order to thrive in today competitive world KSA i.e. knowledge, Skill & Attitude are three most important
aspect to look into. Learning new skills, gaining new knowledge or attitude is an ongoing process, the only
aspect that change in the process of learning is its mode and duration i.e. from school to college and once
becoming part of corporate world training become one of the most prominent mode of learning. Indeed training
enhance and develop the skills sets and knowledge, but requires lot of investment in terms of time, money and
energy. It’s a critical investment in a strategy that leads to internal promotion, succession planning and
employees development. It’s an investment in employees‟ productivity and retention by providing for career
progression and employee’s job satisfaction over the long time (Bowes, 2008). To monitor such investment is
very important from organization point of view.

Increasing expenditure on training and development raise inquisitiveness to know the return on these critical
investments. Human resource department and Trainers are continuously facing the pressure of justifying the
intervention investment through results. So it is becoming important to evaluate the training session properly
and demonstrating as well as communicating the worthwhile contribution of training efforts to the management.

II. Training Evaluation: A Comprehensive Approach

Evaluation is a process of determining the value, worth, and significance of something against set

standards Evaluation is considered as an important element in the designing of training programs because
information gather in the process of evaluation help trainer as well as Organization to modify and develop the
training session so that it becomes more constructive and add more value. “Evaluation is to gather, examine and
interpret the information gained about any aspect of a program of education or training for providing a judgment
on its effectiveness and efficiency and any other outcomes it may have" (ELLINGTON et al. 1993).The process
of examining the effectiveness of any training intervention is called training evaluation. It is one of the most
important aspects of training interventions. Training evaluation check following aspects, first it helps to
determine whether training has had the desired effect, secondly do participants able to implement what they

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

have learned. All training programme should be evaluated not just to justify the investments but evaluation
should be done to bring improvement and modification in the training session so that it becomes more
impactful. Majorly good training interventions initiated with identifying the TNA i.e training need analysis and
ends with evaluation process (Gopal, 2009).Checking effectiveness, assessing whether the defined objective is
achieved or not is what evaluation is. Training effectiveness refers to bring in advantage by a training session to
the Organization and its employees, it can be the enhancement of skills, knowledge, and attitude that will
ultimately help an Organization to become competent in this global competition scenario. Though, it is
considered a tough task to measure the effectiveness of training session because of long duration involved to
figure out its impact on trainees and Organization as well as it is intangible in nature (Prasad, 2005).Evaluation
is the comparison of observe value with the set standard, which will ultimately provide judgment about the
quality of training session (Holli and Colabrese 1998).Training evaluation is a systematic process of collecting
all necessary information about the effect of a session that will help in brining improvement and modification if
requires in the programme so that better outcome can be achieved. Training evaluation gives surety whether
trainees are able to implement their learnings at their work places (Nagar, 2009).Current scenario of justifying
the outcome of training session made it necessary to have a well-planned and reliable evaluation process, which
will ultimately make Organization continue its investment in training programmes. Although evaluation is
considered as difficult to execute because it is tough to set measurable objectives and even tougher to gather
useful information and interpret the outcome, still experts of training evaluation believe that evaluation is one of
the most important process, it is not considered as another element to the training but incorporated within the
training process (Kirkpatrick, 1998).Thus training evaluation cannot be ignored.

III. LITERATURE REVIEW:

Ramachandran (2010) did study on the effectiveness of training programme on different cadre of employees
working in public sector. The study reveals effectiveness differed basis of demographic characters, education
&experience.

Nagar (2009) Training budgets are growing, Organizations are using training to build required skills, study

shows that effectiveness is based on duration, facilities, trainer, training aids, and infrastructure.

Saharan (2011) most Organization use feedback from employees to bring effectiveness in training session.  To

have a competitive edge and better chance of success training is used as a tool. The study shows that training is
imparted to enhance the knowledge and skill sets of the employees.

Smith (1990) viewed that evaluation of management training courses is a subject much discussed
but, apparently carried out. The study finds that there is too much emphasize on providing an objective
evaluation report and too little focus on subjective and peculiar issues which do not necessarily fit the frame.

Blanchard et al. (2000) A survey was conducted in Canada both at management and non-management level.
Survey data reflect that only one-fifth Organization of Canada evaluates training session.

Hashim (2001) made the point that training evaluation hard to define as a concept, and even harder when it
comes to practice. Bringing evaluation in practice has received a lot of criticism. This criticism is largely due
to the unsystematic, informal and ad-hoc evaluation that has been conducted by training institution.

Griffin (2010) finds that there is a disparity between organizations desires to evaluate training and the extent and
effectiveness of actual evaluation. The author has proposed a productivity-based framework to focus data
collection and the utilization of a metric to present results.

Ogundejl (1991) Observed increasing impotence of evaluation and considering it as a powerful tool to enhance

the effectiveness of training. The major role of training evaluation is quality ascription, quality assessment and
quality control.

Iyer, Pardiwalla&Bathia (2009) briefly explore the various training evaluation method to understand

the requirement of training evaluation. Learned concluded that there are different method chalked out to
evaluate a learning session, still the concept is considered as underdeveloped. The concept needs a well-framed

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

evaluation model so that process of justifying investment as well as bringing required modification in the
training session can be done.
IV. Training Evaluation: Purpose, Need and Benefits
Evaluation of a training session help an organization to meet its different goals, evaluation helps in having an
effective training.one of the primary aim of evaluation is to improve training sessions by determining the
required modification and improvement so that outcome can be matched with set standards.

Other purposes of training evaluation are as follows:

 
 To determine whether the objectives set by organization are being met by training session. 
 To justify the cost involved in the training session.
 To determine right audience for future programme.
 To collect the information for bringing effectiveness in future sessions.
 To define the efficiency of the different components of training session (e.g.  Course &contents,
training aids, facilities and environment, programme schedule, presentation style, Trainer etc.) 
 To determine the amount of learning applied at the job by the trainees.
 To determine if TNA i.e. training need analysis met.
 Feedback, Control, Research & Intervention are the four main purpose of evaluation identified by
Bramley and Newby (1984).

Kirkpatrick identifies three reasons to have an evaluation:

 
 To justify the cost of investment by showing the contribution of training session towards the
organization goal.
 To get information for bringing in improvement and modifications in future training sessions.
 To take decision for continue or discontinue particular training session.
 Academics and Training practitioners alike agreed that training evaluation provide number of benefits.
These are some benefits of evaluation:
 Evaluation determines whether learning objectives were met and till what extend and identify scope of
improvement in a particular training session so that better results can be achieved.
 Evaluation demonstrates the benefits gained by organization as well as its employees due to training
session, it help in figuring out the ROI from the training programme.
 Trainers get their feedback that will help them to enhance their skills and knowledge to deliver better.
 Evaluation identify what part of learning are being used on the job? And to what extend?
 Evaluation helps in determining the pace and sequence for training programmes.
 Evaluation even provide feedback to the participants that will help them to analyze enhancement in
their knowledge, skill and attitude.
 Evaluation can even help in reinforcing learning, with the help of a test or performance assessment
trainer or organization identify content retention which will help in designing of future session.

Organizations like IBM, Motorola found out to have a well-defined evaluation mechanism. Maximum
organizations conduct training programs but do not give attention to the evaluation process. Evaluation helps to
ensure that training help in meeting the competency gaps.so evaluation should be considered as an important
aspect of the training process.

V. Models for Training Evaluation

Different models of training evaluation are chalked by different learned, still the concept of evaluation is
considered as underdeveloped. Evaluation is considered as an expense that can be ill afforded.  Still scholars
have tried to come up with effective evaluation mechanism that can actually help in evaluating the training
programme and justifying the investments made in terms of time, energy and money.  Some famous models of
training evaluation are as CIPP Evaluation model by Daniel Stufflebeam and colleagues in the (1960s), CIRO

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

approach by Warr, Bird &Rackson, (1970s), Phillip’s Evaluation approach by Phillips (1996), Kaufman’s five-
level Evaluation,(1992), Kirkpatrick Model (1959),out of which Kirkpatrick did pioneering work in the field and
introduce the well know model “ Four Level Evaluation Model” which is still very popular in training industry.

Here is explanation of few popular models:

A. Kirkpatrick Model:
This training evaluation model is popularly known as “The Kirkpatrick Four-Level Training Evaluation Model”.
Model was develop by Donald Kirkpatrick in 1959, since than gone through several modification and updating.
This models talks about four stages or elements use to measure effectiveness of a training programme these are
reaction; learning; behavior and results.  First stage i.e. Reaction measure or check how participants feel after
attending the training session. At second stage i.e. learning would assess the learning of trainees from the
training session. Third stage talks about evaluating the change in job behavior that reflects after attending
training session. The fourth and final stage i.e. the results would evaluate the outcome of a particular training
session. According to a survey by the American Society for training and development (ASTD), the Kirkpatrick
four-level evaluation approaches is still the most commonly used evaluation framework among Benchmarking
Forum Companies (Bassi& Cheney, 1997). The strong point of the Kirkpatrick evaluation approach is the focus
on behavioral outcomes of the learners involved in the training (Mann & Robertson, 1996).

An in-depth look at the four levels of the Kirkpatrick Model is as follows:

Level 1 Evaluation – Reaction

How do participants feel about the training session? Did the participants like the programme?

Following type of questions get answered at the first stage of Kirkpatrick model i.e. Evaluating Reactions. At
this level trainee’s perception towards the course is measured. In simple terms this level evaluates the reaction
of individuals is identified by asking questions related to the training session. Interviews, questionnaires,
evaluation sheets and participants comment during the session are some likely methodologies use to gather
information about the participants reaction. Evaluation sheet should be design in such a way that it will allow
results to be easily tabulate and can be used to bring required changes if any in future training sessions.

It is recommended that each session needs to be assessed at this level because it helps to identify the area of
improvement if any, and even participants reaction help to decide for continuing or discontinuing a particular
programme. Even though an optimistic reaction does not ensure learning, an unfavorable one definitely makes it
less likely that the user will pay attention to the training.

Level 2 Evaluation – Learning

What have participants learned? What was not learned? What new skills/ knowledge/ attitudes
were gained?

At this level learning of participants is gauge. This is the second level of Kirkpatrick model that is more
challenging and time-consuming as compared to level one. Methodologies involved at this stage are pre and post
testing, observations, interviews, and self-assessment. Pre and post-test help in analyzing the learning due to the
training session. Kirkpatrick emphasizes that test must be accurate and cover all the topics that were part of
course content. Some examples of the tools and methods that can be used at level two are: comparison with
control group, written test, observation etc. this level requires time.

Level 3 Evaluation – Behavior

What change in the behaviour of trainees is noticed? Is learning being applied by the trainees?

This levels talks about the extent to which trainers implement their learning’s. Analysis at this level starts after
3-6 months of training. Lot of factors do work or involved at this level that can affect the outcome. This is the
level where the trainers control over the group is zero. Analysis at this stage is done at their work place, role of
supervisor, colleagues do affect the results. Changes can be figured out by observing application of knowledge,

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

skills learned during training session at job. This level is more challenging compare to first two because
it is usually tuff to anticipate when trainee will apply his learning and till what level. Observation and interview
are two assessment techniques that can be used at this level.

Level 4 Evaluation – Results

What is the final outcome of Training? What are the final results of the training efforts?

This level analyzes impact of the training intervention on the organization. The end result is what actually
matters. Kirkpatrick observed that objective of training session starts in terms of desired results. The complete
success of training session is identified through the end results. This is the toughness level not in terms of
application but defining and justifying the outcome is at time become challenging and complicated.
Observation, yearly/quarterly review, performance analysis, control group etc. are few methodologies that can
be employed at this level.
 Here are the some limitations of Kirkpatrick Model:
 Entire focus of this model is on training session only; this model completely ignores effect of
individual, organization, its culture as well as contextual influences.
 Model work on the fact that each level of this model assumes that next level will provide more
informative data than last level, i.e. according to model level 4 will provide the most useful
information about the training effectiveness.
 Low Incorporation of Level 3 and Level 4 Evaluation. The results of an American Society of Training
and Development (ASTD) survey presented in 2008 indicates that 91% of organizations evaluate at
Level 1, 54% at Level 2, 23% at Level 3 and 8% at Level 4. Reasons for the low level of
implementation of Level 3 and Level 4 may stem from the fact that the measurement of behavior
change is less easy to quantify and interpret than reactions and learning and the results across an entire
organization

Kirkpatrick’s Four Levels of Evaluation at a glance.

Level Measures Key Question Addressed Methodologies or Indicators
1: Reaction Fulfilment What was the participants' reaction program evaluation sheets often
Satisfaction to the program? called Smile Sheets or Happy
Sheets
interviews
questionnaires
participant comments

2: Learning Knowledge What did the participants learn? pre/post testing

observations
interviews
self-assessment

3: Behaviours Transfer of Did the participants' learning impact testing

Learning their behaviour? observation

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

 4: Results Impact Did participants' behaviour changes Indicators can include:
Outcome affect the organization? reduced costs
increased productivity
decreased employee turnover

B. CIRO Model:
Warr,Bird and Rackhman develop CIRO Model in 1970.They mention it first time in their book
“Evaluation of Management Training” .CIRO stands for context, input, reaction and output.  C- Context or
atmosphere within which the training took place  I- Inputs to the training event  R- Reactions to the
training event  O- Outcomes. CIRO model covers all aspects of training cycle, C-Context evaluations help
in getting the idea as what method best fit with the objective of training. Designing and scheduling is
covered in input evaluation. Determining the correctness and accurateness of the inputs is vital for the
success of the training initiative. Reactions Evaluation will check and evaluate the reaction of participants
on the training session so that accordingly changes can be implemented. Outcome Evaluation will keep a
check on the final results achieve at the end of training session i.e. to compare what was planned and how
much is achieved in compare to set objective. CIRO model focus on measuring both the aspects i.e. before
and after the training session, this is the only key difference between CIRO and Kirkpatrick’s models. This
model is considered more suitable to evaluate top and middle level management training programmes rather
than lower level management programme as this model do talk about evaluating behavior.

C. Phillip’s Evaluation:
J. J. Philips in 1996 suggested to add another level i.e. ROI (Return on investment) to Kirkpatrick four level of
evaluation model. This was suggested due to the fact that four level model only focus on evaluating
effectiveness of training session, it doesn’t talk about monetary benefits. As in past few decade justifying the
cost value become challenging for training professionals. According to James and Roffe (2000), ROI Model
translate Kirkpatrick’s fourth level data i.e. results into monetary values. It only give the monetary worth of
benefits received due to particular training session with the help of this formula:

ROI % = (Revenue – Total Costs) / Total Costs * 100

Almost all other training evaluation models focus on measuring satisfaction only Philips ROI model talks about
monetary aspects. Very few Organization conduct evaluations at the ROI level because ROI is considered as a
difficult and expensive process.

D. CIPP Evaluation model:

Context, Input, Process and Product are four parameters on which CIPP Evaluation Models focus on.CIPP
Model of evaluation was develop by Daniel Stufflebeam and colleagues in the 1960s.This framework develop
with aim of linking evaluation with programme decision-making. It is an attempt to make evaluation directly
relevant to decision making.

Context evaluation: This is to determine the extent to which objective of training session match with
organization need. It involves evaluation of training needs analysis and formulating objectives of the session in
the light of these needs.

Input Evaluation: Assessment of action plan this level is to measure the extent to which session strategies,
procedures, and activities support objectives identified. Input evaluation is to examination of the planned
content of the session.

Process Evaluation: It is considered as a critical aspect. Process Evaluation involves evaluation of preparation of
reaction sheets, rating scales and analysis of relevant records (Prasad, 2005). Process evaluation is a continual
assessment of the implementation of the action plan that has been developed by organization. It is an ongoing

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

and systematic monitoring of the programme. A process evaluation provides information that can be used to
guide the implementation of programme strategies, procedures and activities as well as a means to identify
successes and failures. The objectives of process evaluation are  to provide feedback to organization and their
employees about the extent to which the activities are on schedule are being carried out as planned and using
time and resources in an efficient manner;  to provide guidance for modifying or explicating the programmes
action plan as needed, particularly since not all aspects of the plan can be anticipated or planned in advance;  to
assess periodically the extent to which programmes personnel are performing their rules and carting out their
responsibilities;  to provide an extension record of the programmes, how it was implemented and how it
compares to what was intended.

Product evaluation: It involves measuring and interpreting the attainment of training and development
objectives. In other words it can be said that the purpose of product evaluation is to measure, interpret and judge
the extent to which an organization’s improvement efforts have achieved their short term and long term goals. It
also examines both intended and unintended consequences of improvement efforts.

E. Kaufman’s five-level Evaluation

In year 1995 Kaufman, Keller and Watkins drafted Kaufman’s five-level evaluation model. This evaluation
framework connects performance to expectations. Kaufman model is actually reaction to famous Kirkpatrick
model. Four level model evaluate type of impact on trainees while Kaufman’s model evaluate impact on
different audience. The model consist of five different levels as follows:

Level 1: Resources and processes

This level is divided into two levels, 1a and 1b.Level 1a talks about resources. It focus on quality of material
that is required in a training session. Talking about Level 1b, it lens on processes as in how efficient are they? Is
the session able to satisfy the participants?
This level looks into learner satisfaction as well as even focuses on organizational factors that impact the
learning from a particular session.
Level 2: Acquisition

This level is categories as “Micro Level” as focus is on individual and small group payoffs. At this level
competency and mastery of group or individual who attained the training session is evaluated .

Level 3: Application

This level is also categories under “Micro Level” where impact is measured. The focus is to evaluate whether
learning from the training session is been applied and till what level. How much and how well knowledge is
implemented on job is what monitored at this level.

Level 4: Organizational payoffs

This level is considered as “Macro Level” where the overall performance and ROI at organization level is
measured. This level answer the question: what benefits organization received? In simple language what benefit
received by an organization in whole through proposed training session.

Level 5: Societal contributions

This level is final level and considered as “Mega Level “this level is to answer the following questions: What
contribution organization is giving to society? Is organization responsive to society needs? Responsiveness,
possible consequences and payoffs are assessed to determine the success of implementing the proposed training
program.

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

 VI. Conclusions
Training Evaluation is an important aspect, But still an underdeveloped develop concept. Lot of scholars have
contribute to this field by suggesting different models, few popular model have been discussed in this paper
earlier. Still this concept required a sound and efficient model to develop, one of the main reason is that all the
framed model are descriptive and subjective in nature. Factor and indicator for measuring the efficiency is not
well defined and explained. Future studies have shown that among all discussed models Kirkpatrick model is
widely used and most popular.
Future outcome of the studies shows that there are enough model for training evaluation. Now the focus should
be to modify or chalk out a model that clear state the indicators of efficiency, explained properly about each
issue so that evaluation of training session can be correctly carried out with greater effectiveness.

References
1. Kirkpatrick, D. (1996). Revisiting Kirkpatrick’s four-level-model. Training & Development, 1, 54-57.
2. https://www.insources.com.au/in-blog/purpose-and-benefits-of-training-evaluation
3. https://www.sswm.info/content/training-evaluation
4. Kirkpatrick, D. L. (1994). Evaluating training programs: the four levels. San Francisco: Berrett-Koehler.
5. https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/
6. http://www.managementstudyguide.com/training-evaluation.htm
7. https://pdfs.semanticscholar.org/b0d5/90042c11dee1c4dc6d185240307d20a06f38.pdf
8. http://www.blendit-training.co.uk/wp-content/uploads/2011/11/CIRO-Context-Input-Reaction-Outcome.pdf
9. https://www.spearhead-training.co.uk/blog/evaluating-training
10. https://en.wikipedia.org/wiki/CIPP_evaluation_model
11. Kaufman, R., Keller, J., & Watkins, R. (1995). What works and what doesn’t: Evaluation beyond
Kirkpatrick. Performance and Instruction, 35(2): 8-12. 
12. https://www.completelearning.co.nz/wp-content/uploads/2012/03/Measuring-ROI-The-Fifth-Level-of-
Evaluation3.pdf
13. http://etec.ctlt.ubc.ca/510wiki/Evaluation_in_Instructional_Design_Kirkpatrick%27s_4_Level_Model
14. Bates, Reid. (2004) A critical analysis of evaluation practice: The kirkpatrick model and the practice of
beneficence. Evaluation and Program Planning, 27, 341-347.
15. Chapman, Alan. (2007) Kirkpatrick’s Learning and Training Evaluation Model. Retrieved Feb 7, 2009
from http://www.businessballs.com/kirkpatricklearningevaluationmodel.htm
16. Clark, Donald. (2007). Instructional system development – evaluation phase. Retrieved Feb 7, 2009
from http://www.skagitwatershed.org/~donclark/hrd/sat6.html
17. Crone, Glen. (2005). Evaluation of executive training. Treasury Board of Canada Secretariat Retrieved Feb
20, 2009 from http://www.tbs-sct.gc.ca/eval/pubs/eet-efcs/eet-efcs_e.asp
18. Dick, Walter. (2002). Chapter 11 Evaluation in instructional design: The impact of kirkpatrick’s four-level
model. In Robert Reiser& John Dempsey (Eds.), Trends and issues in instructional design and technology
(pp. 145-153). Prentice Hall.
19. Kaufman, R., Keller, J. & Watkins, R. (1995). What works and what doesn’t: Evaluation Beyond
Kirkpatrick. Performance and Instruction, 35(2), 8-12.
20. Kirkpatrick, D. L. (1996). Techniques for Evaluating training programs. In Donald P. Ely, &TjeedPlomp
(Eds). Classic writings on instructional technology (pp.119-141). Libraries Unlimited.
21. Sloman, Martyn. (2008). The value of learning. ASTD 2008 International Conference and Exposition.
Retrieved on Feb 15, 2009 fromhttp://www.astd2008.org/PDF/Speaker%20Handouts/ice08%20handout
%20M120.pdf
22. Wikipedia. Evaluation. Retrieved on Feb 27, 2009 from http://en.wikipedia.org/wiki/Evaluation
23. https://en.wikipedia.org/wiki/Training
24. http://www.hrwale.com/training/training-evaluation/
25. http://www.sswm.info/content/training-evaluation
26. http://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS

AN ANALYSIS OF VARIOUS TRAINING EVALUATION MODELS