Organization Development and Change

Chapter Eleven:
Evaluating and Institutionalizing
OD Interventions

Thomas G. Cummings
Christopher G. Worley
 Learning Objectives
for Chapter Eleven
• To understand the issues associated with
evaluating OD interventions
• To understand the process of
institutionalizing OD interventions and the
factors that contribute to it

Cummings & Worley, 7e (c) 2001 S 11-2

outh-Western College Publishing
 Issues in Evaluating OD
Interventions
• Implementation and Evaluation Feedback
• Measurement
– Select the right variables to measure
– Design good measurements
• Operational
• Reliable
• Valid
• Research Design
Cummings & Worley, 7e (c) 2001 S 11-3
outh-Western College Publishing
 Implementation and
Evaluation Feedback
Diagnosis

Design and Implementation of Implementation Evaluation

Implementation Intervention Feedback Feedback
of Interventions Clarify Plan for Measures of Measure
Intention Next Steps the Intervention of
Alternative and Immediate Long-term
Interventions Effects Effects

Cummings & Worley, 7e (c) 2001 S 11-4

outh-Western College Publishing
 Implementation Evaluation
Feedback Feedback
• Feedback aimed at • Feedback aimed at
guiding determining impact of
implementation efforts intervention
• Milestones, • Goals, outcomes,
intermediate targets performance
• Measures of the • Measures of the
intervention’s progress intervention’s effect

Cummings & Worley, 7e (c) 2001 S 11-5

outh-Western College Publishing
 Sources of Reliability
• Rigorous Operational Definition
– How high does a team have to score on a five-
point scale to say that it is effective?
• Multiple Measures
– Multiple items on a survey
– Multiple measures of the same variable (survey,
observation, unobtrusive measure)
• Standardized Instruments
Cummings & Worley, 7e (c) 2001 S 11-6
outh-Western College Publishing
 Types of Validity
• Face Validity: Does the measure “appear” to
reflect the variable of interest?
• Content Validity: Do “experts” agree that the
measure appears valid?
• Criterion or Convergent Validity: Do
measures of “similar” variables correlate?
• Discriminant Validity: Do measures of “non-
similar” variables show no association?
Cummings & Worley, 7e (c) 2001 S 11-7
outh-Western College Publishing
 Elements of Strong Research
Designs in OD Evaluation
• Longitudinal Measurement
– Change is measured over time
• Comparison Units
– Appropriate use of “control” groups
• Statistical Analysis
– Alternative sources of variation have been
controlled

Cummings & Worley, 7e (c) 2001 S 11-8

outh-Western College Publishing
 Evaluating Different
Types of Change
• Alpha Change
– Movement along a stable dimension
• Beta Change
– Recalibration of units of measure in a stable
dimension
• Gamma Change
– Fundamental redefinition of dimension

Cummings & Worley, 7e (c) 2001 S 11-9

outh-Western College Publishing
Institutionalization Framework

Organization

Institutionalization
Institutionalization
Characteristics

Indicators of
Processes
Intervention
Characteristics

Cummings & Worley, 7e (c) 2001 S 11-10

outh-Western College Publishing
 Organization Characteristics
• Congruence
– Extent to which an intervention supports or
aligns with the current environment, strategic
orientation, or other changes taking place
• Stability of Environment and Technology
• Unionization

Cummings & Worley, 7e (c) 2001 S 11-11

outh-Western College Publishing
 Intervention Characteristics
• Goal Specificity
• Programmability
• Level of Change Target
• Internal Support
• Sponsor

Cummings & Worley, 7e (c) 2001 S 11-12

outh-Western College Publishing
 Institutionalization Processes
• Socialization
• Commitment
• Reward Allocation
• Diffusion
• Sensing and Calibration

Cummings & Worley, 7e (c) 2001 S 11-13

outh-Western College Publishing
 Indicators of
Institutionalization
• Knowledge
• Performance
• Preferences
• Normative Consensus

Cummings & Worley, 7e (c) 2001 S 11-14

outh-Western College Publishing