BBA 229

Training and Development

Lecture 5
Training Evaluation
Dr. S. Chan
Department of Business Administration
charmaine@chuhai.edu.hk
http://home.chuhai.hk/~charmaine/
1
 Training Evaluation (1 of 2)
Training effectiveness – the benefits that
the company and the trainees receive from
training

Training outcomes or criteria – that the

trainer and the company use to evaluate
training programs

2
 Training Evaluation (2 of 2)
Training evaluation – the process of
collecting the outcomes needed to
determine if training is effective

Evaluation design – from whom, what,

when, and how information needed for
determining the effectiveness of the
training program will be collected

3
Reasons for Evaluating Training
(1 of 2)
Companies are investing millions of dollars
in training programs to help gain a
competitive advantage

Training investment is increasing because

learning creates knowledge
– This differentiates between those companies
and employees who are successful and those
who are not

4
Reasons for Evaluating Training
(2 of 2)

Because companies have made large

dollar investments in training and
education and view training as a strategy
to be successful, they expect the
outcomes or benefits related to training to
be measurable

5
Training evaluation provides the
data needed to demonstrate that
training does provide benefits to
the company.

6
 Formative Evaluation
The evaluation of training that takes place during
program design and development

Helps to ensure that:

– the training program is well organized and runs
smoothly
– trainees learn and are satisfied with the program
Provides information about how to make the
program better

7
 Pilot Testing
The process of previewing the training program
with potential trainees and managers or with
other customers

It can be used:
– as a “dress rehearsal” to show the program to
managers, trainees, and customers
– for formative evaluation

8
 Summative Evaluation
Evaluation conducted to determine the
extent to which trainees have changed as
a result of participating in the training
program

May also measure the return on

investment (ROI) that the company
receives from the training program

9
Why Should A Training Program
Be Evaluated? (1 of 2)
To identify the program’s strengths and
weaknesses
To assess whether content, organization,
and administration of the program
contribute to learning and the use of
training content on the job
To identify which trainees benefited most
or least from the program

10
Why Should A Training Program
Be Evaluated? (2 of 2)
To gather data to assist in marketing
training programs
To determine the financial benefits and
costs of the programs
To compare the costs and benefits of
training versus non-training investments
To compare the costs and benefits of
different training programs to choose the
best program
11
The Evaluation Process

Conduct a Needs Analysis

Develop Measurable Learning Outcomes

and Analyze Transfer of Training

Develop Outcome Measures

Choose an Evaluation Strategy

Plan and Execute the Evaluation

12
 Outcomes Used in Evaluating
Training Programs: (1 of 4)

Cognitive Skill-Based
Outcomes Outcomes

Return on
Affective Results Investment
Outcomes

13
 Outcomes Used in Evaluating
Training Programs: (2 of 4)
Cognitive outcomes
– determine the degree to which trainees are familiar
with the principles, facts, techniques, procedures, or
processes emphasized in the training program
– measure what knowledge trainees learned in the
program
Skill-based outcomes
– assess the level of technical or motor skills
– include acquisition or learning of skills and use of skills
on the job

14
 Outcomes Used in Evaluating
Training Programs: (3 of 4)
Affective outcomes
–include attitudes and motivation
–reaction outcomes – trainees’ perceptions of
the program including the facilities, trainers,
and content
Results
–determine the training program’s payoff for the
company

15
 Outcomes Used in Evaluating
Training Programs: (4 of 4)
Return on Investment (ROI)
–comparing the training’s monetary benefits
with the cost of the training
direct costs
indirect costs
benefits

16
 Determining Whether
Outcomes Are Good
Good training outcomes need to be:
Relevant
Reliable
Discriminative
Practical

17
 Good Outcomes: Relevance
Criteria relevance – extent to which training programs
are related to learned capabilities emphasized in the
training program
Criterion contamination效標混淆 – extent that training
outcomes measure inappropriate capabilities or are
affected by extraneous conditions=> 當我們收集效標資料
進行效度考驗時，常發現效標資料會受到測驗分數的影響
，這就是效標混淆的問題。例如：一位教師知道學生的性
向測驗分數很低，他對這位學生效標分數的評定，因受那
個測驗分數影響，也給很低；反之，便給很高。這種誤差
來源，稱為效標混淆，類似心理學上所稱的月暈效應（
halo effect）。為了避免此種誤差，最好的方法是使參與
效標評定者，不知道受試者的測驗分數。

18
 Good Outcomes: Relevance
Criterion deficiency – failure to
measure training outcomes that were
emphasized in the training objectives

19
 Good Outcomes (continued)
Reliability – degree to which outcomes can
be measured consistently over time
Discrimination – degree to which trainee’s
performances on the outcome actually reflect
true differences in performance
Practicality – the ease with which the
outcomes measures can be collected

20
 Training Program Objectives and Their
Implications for Evaluation:
Objective

Learning Transfer
Outcomes
Reactions: Did trainees like the program? Skill-Based: Ratings by peers or managers
Did the environment help learning? based on observation of behavior
Was material meaningful?

Cognitive: Pencil-and-paper tests Affective: Trainees’ motivation or job attitudes

Skill-Based: Performance on a work sample Results: Did company benefit through sales,
quality, productivity, reduced
accidents, and complaints?
Performance on work equipment

21
Factors That Influence the Type of
Evaluation Design
Factor How Factor Influences Type of Evaluation Design
Change potential Can program be modified?

Importance Does ineffective training affect customer service, product

development, or relationships between employees?
Scale How many trainees are involved?
Purpose of training Is training conducted for learning, results, or both?
Organization Is demonstrating results part of company norms and
culture expectations?
Expertise Can a complex study be analyzed?
Cost Is evaluation too expensive?
Time frame When do we need the information?

22
 Conditions for choosing a rigorous
evaluation design: (1 of 2)
1. The evaluation results can be used to
change the program
2. The training program is ongoing and has
the potential to affect many employees
(and customers)
3. The training program involves multiple
classes and a large number of trainees
4. Cost justification for training is based on
numerical indicators
23
 Conditions for choosing a rigorous
evaluation design: (2 of 2)
5. You or others have the expertise to
design and evaluate the data collected
from the evaluation study
6. The cost of training creates a need to
show that it works
7. There is sufficient time for conducting an
evaluation
8. There is interest in measuring change
from pre-training levels or in comparing
two or more different programs
24
 Importance of Training Cost
Information
To understand total expenditures for training,
including direct and indirect costs
To compare costs of alternative training
programs
To evaluate the proportion of money spent on
training development, administration, and
evaluation as well as to compare monies spent
on training for different groups of employees
To control costs

25