ASSESSMENT OF EVALUATION THEORY:

KIRKPATRICK MODEL IN OPPOSITION TO

HAMBLIN MODEL

Ms. Divya Sharma1

1
Department of Management Science, Maharashtra Institute of Technology, Aurangabad (India)

ABSTRACT

Training includes any type of experience designed to facilitate learning which will aid performance in a present
or future job. Evaluation at a learning level provides data on the degree of change to knowledge, skills or
attitude from the program. This paper also talks about the principle of evaluation and its challenges. The paper
also explores the Evaluation strategy model and global evaluation trend. This article reviews the approaches of
Kirkpatrick model and Hamblin model into theoretical aspect. Meticulous attention is paid by comparing the
variation between Kirkpatrick and Hamblin model.

Keywords: Evaluation Strategy, Hamblin and Kirkpatrick model.

I. INTRODUCTION

Training and development programs are considered to be one of the important aspects of organizational
development. There has been a growing need to find ways to determine the efficiency and effectiveness of such
training programs both in terms of the organisation and the training institutions. Such evaluation would not only
form the baseline for further progress, but would also help justify whether the investment in terms of time,
money and energy has demonstrated its value to the organisation.
Many Specialists from other countries, engaged in training and development, have tried to evolve different
methods of assessing the effectiveness of training programs. Except in rare cases there seems to be no accepted
method of establishing the cost benefit ratio because of evident limitations like the high variability and the long
lead time in investment returns from it; turnover among the trained executives; newer and more urgent demands
made on the trained executive which are beyond the scope of the programme; the motives of the sponsor, and
the learner’s attitude towards training which range from relaxation to serious learning.
Kirkpatrick’s 1(1975) evaluation study extends and elaborates the evaluation process from mere reaction
assessment to four stages, viz, reaction; learning; behavior and results of training. In spite of his approach to

194 | P a g e
assessment at various stages, it precludes the trainee variables and organizational variable which could have a
definite impact on the overall assessment of training and development.
2
Hamblin (1974) while linking up training objective with that of evaluation objectives, which may be
directed towards a cost benefit approach, job related approach or the training centered approach, developed a
conceptual scheme of evaluation which cuts through maze of evaluation stages enunciated by Kirkpatrick.
Beside the objective setting and training phase Hamblin has also tried to draw guidelines for developing various
strategies of management at various levels of evaluation.

II. EVALUATION AND VALIDATION

Evaluation means the assessment of value or worth. It would simply mean the act of judging whether or not the
activity to be evaluated is worthwhile in terms of some criterion of value, in the light of the information
available. Evaluation has traditionally been taken to include not only the assessment of value, but also the
collection and analysis of the information on the basis of which the assessment is to be made.
This broader definition incorporates the objective of evaluation which includes investigation before and after
training as well as during the training because one cannot assess training efforts unless something about the
before and after training situation is known.
Validation is defined in terms of Internal and External validation. Internal Validation deals with a series of tests
and assessments designed to ascertain whether a training program has achieved the specified objectives.
External validation is a series of test designed to ascertain whether the objectives of an internally valid program
are realistically based on an accurate initial identification of training needs in relation to the criteria of
effectiveness adopted by the organization.
The term evaluation involves measurement in social and financial terms which may not be tangible for external
training programs. Validation is limited to lay down objectives excluding the anticipated and unanticipated
effects, both desirable and undesirable. For the purpose of research the broader definition of evaluation partly
based on Hamblin’s definition appears appropriate. Evaluation could thus be defined as an attempt to obtain
information or feedback on the effects of a training program, and to assess the value of this training program.

2.1 Principles of Evaluation

To be effective, evaluation of training must be consistent with the purposes, objectives and goals of the training
activity based on the following guidelines:
 Trainer must be clear about the goals and purpose of evaluation
 Essential involvement of the training staff, the trainer and the sponsoring organisation
 Evaluation must be continuous and specific
 Evaluation must provide the means and focus for trainers to be able to appraise themselves
 Evaluation must be based on objective methods and standards
 Effective communication and coordination are essential

195 | P a g e
  Realistic target dates must be set for each phase of the evaluation

2.2 Problems in Evaluation

1. Lack of Goal Congruence

The main challenge is difference in perception about the objectives of the training, they basically lack
in clarity of specific training objectives.
2. Trainers, Organization and Trainees approach to evaluation
Trainers use his past experience and some kind of intuitive feeling about the course rather than
employing a systematic method of feedback to improve subsequent programs. Management does not
establish clear objectives for training but remain vague and ambivalent. The trainee as evaluator may
perceive himself to be in a very superior role of being appraiser.
3. Problems arising from the nature and variability of management tasks
Variety of factors influencing the behavior of managers, they may include individual factors,
personality, motivation, knowledge, skills and attitudes , organizational climate, cooperation of
subordinates and colleagues and the type and nature of industry.
4. Problems arising from the Quantitative approach to Evaluation
Rigid mathematical systems without specific application to reality while the simply refuses to consider
the problems of quantification.

2.3 Evaluation Strategy – Model

Evaluation of training is an important activity because a training program may have to be continuously cut,
lengthened, changed, resequenced, rescheduled, eliminated or even retained as it is, on the basis of the
evaluation findings. Although at first glance evaluation findings determine how well a training program is
achieving its goals and objectives, ultimately a good evaluation system has to be source of information upon
which a variety of decisions can be made and purposes served. Evaluation can be viewed as a method of
measuring change in knowledge, skills, attitudes, job performance, costs and the quality of training facilities.
The process of evaluation evolve certain criteria
 Setting training objectives
 Selecting trainees for the training program
 Linking the training inputs with the goals and objective of the training
 Arriving at the suitable pedagogical tools and techniques
 Measuring the impact of training programs
 Determining the methods of review and feedback

Organization can explore and correlate various biographical and organizational factors related to i.e. training to
comprehend the process of learning. Transfer of training to the job so as to arrive at guidelines for optimizing

196 | P a g e
the training effect. This will help identify the organizational variables which help hinder the learning process so
that by overcoming these hindering factors the organisation can make constructive use of the training effort.

2.4 Global Evaluation Trend

Measurement and evaluation have been changing and evolving in private and public sector, as well as
nongovernmental organizations. This change is occurring across organizations and cultures worldwide. The
following are a few of the most obvious trends.
 Organizations are increasing their investments in measurement and evaluation, with best practice
groups spending 3 to 5 percent of the HRD budget on measurement and evaluation.
 Organisations are moving up the value chain, away from measuring reaction and learning to measuring
application, impact, and ROI
 Needs of the clients and sponsors of training and development projects, programs, initiatives, and
solutions are the primary drivers for the increasing focus on measurements and evaluation.
 Evaluation is an integral part of the design, development, delivery, and implementation of programs.
 A shift from a reactive approach to a proactive approach is occurring as organization address
evaluation early in the cycle.
 Measurement and evaluation processes are systematic and methodical, often a seamless part of the
delivery process.
 Technology is significantly enhancing the measurement and evaluation process, enabling the data
collection, processing, analysis, and integration of large amount of data.
 Evaluation planning is becoming a critical part of the measurement and evaluation cycle.
 The implementation of comprehensive measurement and evaluation processes usually leads to
emphasis on initial need analyses.
 Organization with comprehensive measurement and evaluation systems in place has seen increases in
their program budgets.
 Organization without comprehensive measurement and evaluation systems see the reduction or
elimination of their program budgets.
 The use of ROI is emerging as an essential part of many measurement and evaluation systems. It is a
fast growing metric 70 to 80perecent of organization have it on their wish lists.

Many successful examples of comprehensive measurement and evaluation applications are available in all types
of the organization and cultures.

2.5 Kirkpatrick model

Kirkpatrick developed his four step model in 1959 and provided a simple and pragmatic model for helping
practitioners think about training programs. Kirkpatrick (1977) divided the evaluation model into four parts:
reaction; learning; behavior and results. Reaction would evaluate how participants feel about the programme

197 | P a g e
they attended. The learning would evaluate the extent to which the trainees learned the information and skills,
the behavior would evaluate the extent to which their job behavior had changed as a result of attending the
training. The results would evaluate the extent to which the results have been affected by the training
programme. According to a survey by the American Society for training and development (ASTD), the
Kirkpatrick four level evaluation approaches is still the most commonly used evaluation framework among
Benchmarking Forum Companies (Bassi & Cheney, 1997).
It has been criticized for implying a hierarchy of value related to the different levels, with organizational
performance measures being seen as more important that reactions. More fundamentally, there have been
criticisms of the assumptions that the levels are each associated with the previous and next levels. This implied
casual relationship has not always been established by research. The main strength of the Kirkpatrick evaluation
approach is the focus on behavioural outcomes of the learners involved in the training (Mann & Robertson,
1996).The strengths of this model lie in its simplicity and realistic way of helping practitioners think about
training programs (Alliger and Janak, 1989).
The best known and most widely used framework for classifying evaluation is the Kirkpatrick model. The
model consists of four stages, originally described more recently by Kirkpatrick (1996) as levels. The four levels
are:
Level Dimensions Descriptions
What participants thought of the program, measured by the use
1 Reaction
of reaction questionnaire
Changes in Knowledge ,skill and attitude, , assessed by use of
2 Learning
performance tests
Changes in job behavior and to identify learning are applied.
3 Behavior
Assessment method observation and productivity data.
Contribution of the training program, methods includes ROI,
4 Results
measuring costs.

198 | P a g e
 Source: from Kirkpatrick, 1996

Let us discuss the four level of Kirkpatrick

Level 1 Reaction:
Reaction evaluation is to enhance the quality of training programmes, which in turn leads to improved
performance by measuring the participant’s reactions to training programme. This should be measured
immediately after the programme. Level one evaluation should not just include reactions towards the overall
programme (e.g. did you like the programme.); it should also include measurement of participants‟ reactions or
altitudes towards specific components of the programme such as, the topics, contents, methodology, instructor
etc.
Level 2 Learning:
Evaluation at this level wants to differentiate between what they already knew prior to training and what they
actually learned during the training programme (Jeng & Hsu, nd.). Learning outcome can include changes in
knowledge, skills or attitudes. The Evaluation should focus on measuring what was covered in the training
events i.e. learning objectives.
Level 3 Behavior:
Behavior evaluation is the extent to which the trainees applied the learning and changed their behavior, and this
can be immediately and several months after the training, depending on the situation. This level evaluation
wants to measure the transfer that has occurred in the learner’s job behavior/ job performance due to the training
programme.
Level 4 Results:
The intention at this level is to assess the coat vs. benefits of training programme, i.e. organizational impact in
terms of reduced costs, improved quality of work, higher productivity, reduction in turnover, improved human

199 | P a g e
relation, increased sales, fewer grievances, lower absenteeism. Higher work morale, fewer accidents, greater job
satisfaction etc. Collecting, organizing and analyzing level four information can be difficult, time consuming
and more costly than the other three levels, but the results are often quite worthwhile when viewed in the full
context of its value to the organisation.
Kirkpatrick (1959) originally discussed reactions in terms of how well participants liked a particular program. In
practice, measures at this level have evolved and are most commonly directed at assessing trainees’ affective
responses to the quality (e.g. satisfaction with the instructor) or the relevance (e.g. work-related utility) of
training. Learning measures, level two, are quantifiable indicators of the learning that has taken place during the
course of the training. Level three behavior outcomes address either the extent to which knowledge and skills
gained in training are applied on the job or result in exceptional job-related performance. Finally, level four
outcomes are intended to provide some measure of the impact that training has had on broader organizational
goals and objectives. In recent practice, the typical focus of these measures has been on organizational level
financial measures.

It has provided straightforward system or language for talking about training outcomes and the kinds of
information that can be provided to assess the extent to which training programs have achieved certain
objectives. Kirkpatrick insisted that information about level four outcomes is perhaps the most valuable or
descriptive information about training that can be obtained

Critics on Kirkpatrick Model

Expert Dan McCarthy says that there is low correlation among the four levels (reaction, learning, behavior, and
results). This takes nothing away from the contribution that the Kirkpatrick model has made to the field over the
past 50 years. McCarthy suggests, we only compare what people knew before training to what they know and
do after training, we still will not know what to do to achieve better results.
Bernthal (1995) argues that the model mixes evaluation and effectiveness and that these do not form a
continuum.
Many evaluation studies that have evaluated training on two or more of Kirkpatrick’s levels have reported
different effects of training for different levels (Alliger and Janak, 1989).
Holton (1996) argues that the levels form taxonomy of outcomes rather than a model.
Kraiger and Jung (in Quinones,1997) agree with his view and argue that at the same time Kirkpatrick provides a
model for thinking about how to evaluate, it does little to inform what to evaluate and how to link the results to
strategy.
Other complaints are that the model is too simple and fails to take account of the various intervening affecting
learning transfer.

Percentage that the Kirkpatrick Model is used:

Bassi, et al (1996) reported that 96% of companies surveyed used some form of the Kirkpatrick framework to
evaluate training and development programs.
200 | P a g e
Twitchell, Holton, and Trott (2000) performed a meta-analysis of studies performed in the last 40 years. Their
research indicates the following ranges for the use of Kirkpatrick's four levels:
Level 1: 86-100%
Level 2: 71-90%
Level 3: 43-83%
Level 4: 21-49%
There is no doubt that Kirkpatrick’s model has made valuable contributions to training evaluation thinking and
practice. It has helped focus training evaluation practice on outcomes (Newstrom, 1995), fostered the
recognition that single outcome measures cannot adequately reflect the complexity of organizational training
programs, and underscored the importance of examining multiple measures of training effectiveness. The model
promoted awareness of the importance of thinking about and assessing training in business terms (Wang, 2003).

III. THE FIVE LEVEL APPROACH : HAMBLIN (1974)

Hamblin was one of the first to modify Kirkpatrick’s model. The first three levels in his model correspond
closely to Kirkpatrick’s model. However, the final level is split into two organization and ultimate value. Unlike
Kirkpatrick, Hamblin suggests that the five levels of his model form a hierarchy.
The five level models are therefore:
Level Dimensions Descriptions
What participants thought of the program, measured by the use
1 Reaction
of reaction questionnaire
Changes in Knowledge ,skill and attitude, , assessed by use of
2 Learning
performance tests
Changes in job behavior and to identify learning are applied.
3 Job Behavior
Assessment method observation and productivity data.
Effect on organization, from participants job to performance
4 Organization
changes
The financial effects, both on the organizational and the
5 Ultimate value
economy.

Hamblin framework is intended to supplement other evaluation classification. This framework was first
presented in an earlier article (Hamblin, 1968), but it has been modified and improved a lot. It has proved useful
on a number of training courses, on which it helped trainees to see the connections between widely differing
approaches to training evaluation, and to plan evaluation strategies for their own firms.
The framework is presented in diagrammatic form as below:

201 | P a g e
3.1 Hamblin cycle of evaluation
(‘O’ represents Objective and ‘E’ represents Effects)
In Hamblin model we are assuming that there is a cause and effect chain linking the five levels of training
effects. This can be set out as follows:
TRAINING which leads to REACTIONS which lead to LEARNING which leads to CHANGES IN JOB
BEHAVIOUR which lead to CHANGES IN THE ORGANIZATION which lead to CHANGES IN THE
ACHIEVEMENT OF ULTIMATE GOALS. This chain may snap at any of its links.
Though first three level is already discussed in Kirkpatrick model, let us discus the same through Hamblin
model

Level 1 Reaction:
The effect after training is that the trainees will react to the training and their reactions will be highly complex
and shifting over time. According to his model 2 variable will play an important role i.e. reactions effects and
reactions objective. We must be selective and must have criteria for selection. This means in effect that we must
determine the reactions in which we are interested in investigating that how trainees will react.
Level 2 Learning:
Learning depends on reaction; people who react inappropriately will fail to learn inappropriately. Trainees
existing state of learning (knowledge, skill and attitude) must be compatible and should be receptive to the
training. Trainees must have basic aptitude.
Level 3 Job Behavior:

202 | P a g e
In other circumstances, it may be possible and worthwhile to go beyond the terminal behavior of the trainees,
and to find out whether they have applied their learning on the job. The ultimate objective of the training
program is job behavior and the training is regarded as successful if the desired behavior changes are achieved.
Level 4 Organization:
Hepworth (1972) suggests that all organizations have four primary objectives, which are in order of primacy.
1. Survival of the organization
2. Creation of surplus money
3. Welfare of interested parties
4. Social / Political welfare
Therefore organization at level 5 i.e. Ultimate values must decide to objective of the training to contribute
towards the organization and weighting to give to the different objectives.
Level 5: Ultimate Value
Training should be defined in terms of the trainee’s own personal goals( improved financial reward, job
opportunity and self esteem ) rather than those of the organization. Ultimate value should not be seen purely in
terms of profit or cost effectiveness.
Uses of the Hamblin model:
The model can be seen in two ways.
1. It can be seen as a cycle of events occurring in an organization: training objectives lead to training,
which leads to training effects, which lead to more training objectives.
2. It can be seen as a cycle of evaluation activities the trainer/ evaluator is engaged, in which every
activity apart from the training activity itself is a stage in the process of training.
The purpose of the model is to help the manager or trainer to decide which activity or activities he is
already involved in, and to plan the route which he should follow from there.

IV. CONCLUSION

The purpose of evaluation is to create a feedback loop or a ‘self correcting training system’ (Rackham, Honey,
and Colbert, 1971). A well controlled training program is one which weakness and failures are identified and
corrected by means of negative feedback, and strengths and success are identified and amplified by means; as
there are many Evaluation model. It is observed that basically Kirkpatrick framework is considered as core. If
we see the Hamblin model first three levels is derived from Kirkpatrick framework or expanded model which
encapsulates certain elements both before assessing reactions and after an examination of organizational results.
Though there are many critics about the Kirkpatrick Model, many of the organization uses the model for their
training evaluation.

203 | P a g e
REFERENCES

1. Blanchard PN, Thacker JW,Way SA(2000), Training evaluation : perpestives and evidence from
Canada’, International Journal of Training and Development, Vol. 4(4),pp.41-45
2. J. Philips, Handbook of training evaluation and measurement methods (Butterworth- Heinemann,
Oxford, 1991)
3. R. A. Noe, Employees Training and Development (Irwin, Mc Graw – Hill, 2008).
4. L. M. Prasad, Human Resource Management (New Delhi, Sultan Chand & Sons, 2005)
5. Kirkpatrick DL(1983), ‘ Four steps to measuring training effectiveness’, Personnel Administrator, Vol.
28(11),pp.19-25
6. Kirkpatrick DL(1996), ‘ Great ideas revisited : revisiting Kirkpatrick four level model’, Training and
Development, Vol.50(1), January,pp.54-57
7. Philips, J. J. (1996c). How much is the training worth? Training & Development, 50(4), 20-24.
8. Ulrich, D. (1997). Measuring human resources: An overview of practice and a prescription for results.
Human Resource management, 36(3), 303-320.
9. A Hamblin. Evaluation and control of training. McGraw-Hill, London, 1974.

204 | P a g e