EVALUATING ORGANIZATIONAL

CHANGE
WEEK # 11

Dr. Tazeem Ali Shah

Evaluating and Maintaining Change
 Conti…
• Evaluating and Maintaining the Change
• Once you have transferred the change project to those individuals who will be owning
(utilizing, managing, operating, and/or maintaining) the change moving forward, do the
following:
• Evaluate the entire change initiative:
• Get the group together who worked on the change initiative, along with key stakeholders,
and evaluate the initiative. Ask the following 3 simple questions:

• What went well in launching, managing and transferring this change initiative?
• What could have been done better?
• What else should be shared so that future change projects are successful?
Conti…
• Develop a plan to maintain the change:
• Have a plan in place to evaluate the change over time. This might be on a 6 month,
9 month and 12 month basis initially; and then annually or bi-annually after that.
• As with any process within an organization, regular evaluation is essential to
ensure the process still meets the needs of the organization.
• As organizations change – they grow, new products and services are launched,
new employees come on board, the client base grows, etc. – it is essential to
evaluate processes to ensure that they are still beneficial, effective and help
support the organization.
 What is Evaluation and Feedback
• Evaluation / Assessment—
• The process of determining the merit or value of a condition/trait, process,
and/or outcome of the change process.
• Feedback—
• The process of feeding back diagnostic information to the client to guide
further action.

• Institutionalizing Change—
• The process of making change a permanent part of the organization’s
culture and practices.
Conti…
• CASE EXAMPLE
• Total Quality Transformation—
• System-wide change from traditional manufacturing to continuous process
improvement.
• Evaluation—
• Technical aspects of change were improved; social aspects of change were not
improved.
• Feedback —
• Evaluation data showing improvements and problems were fed back to client to
guide further change.
Conti…
• Aims
• Examine the arguments for evaluating organizational change
• Consider the limitations of evaluation
• Consider different methods for evaluation
• Consider difficulties of evaluation in practice
• Consider costs and benefits in practice
Measuring Organization Change:
• Given all the intervening variables at play, trying to measure change
transformation by isolating specific metrics is usually ineffective.
• Instead, focus on overall results for a more relevant picture of how well your
organization has changed.
• Asking questions around the following four points can help you assess the big
picture more accurately:
• What changes did you make? 
• How aligned were these changes with your organization’s strategic goals?
• Did the design make sense for the organization and for the market?
• Did it fit well with the culture of your organization?
Conti….
• How well did you implement?
• Did the changes called for by the design actually get put into place in your organization?
• Measuring and tracking implementation is essential, because if the organization design
changes aren’t actually being made, it’s impossible to know whether or not the design is
effective.
• What behavior changes resulted from your organization design implementation
efforts? 
• Performance changes will only happen as a result of behavior changes. If behaviors
aren’t changing, you may need to go back and examine both the organization design
blueprint and implementation efforts—including deployment and communication
around the change.
Conti….
• How have organization outcomes changed? 
• The final proof of whether or not your change transformation efforts have been successful
will always be seen in terms of business outcome.
• Measure overall changes in productivity, sales, customer satisfaction, and other areas to
help determine actual market value realized by the change.
• Success is never guaranteed when attempting organization change.
• Unforeseen things can happen to derail even the best planned and executed design.
But knowing what to track and why when measuring organization change will help
you to better understand how the change process is unfolding in your organization
and bring potential problems to light so that they can be addressed and corrected
sooner rather than later.
• In this way, your organization will be better equipped to navigate change successfully
despite intervening factors.
Evaluation process
• KEY EVALUATION QUESTIONS:

• WHAT will be evaluated? What is the object of evaluation?

• WHO are the stakeholders of evaluation?

• HOW—What evaluation design and data will be used?

Methods of Evaluations
• STRATEGY #1 Periodically Monitor Effects of Change on Desired Outcomes:

• Step 1—Design and administer an assessment survey and/or interview questions

to establish a baseline of the problem or need for change.

• Step 2—Re-administer the survey and/or qualitative instrument periodically to

assess progress toward desired outcomes.
Conti….
• STRATEGY #2 Use Implementation Feedback during and after an intervention.

• DURING—Implementation should be guided by feedback about whether

behaviors and procedures are changing as intended.

• AFTER—Post-evaluations alone may only show that outcomes were not achieved,
not why the interventions that were intended to contribute to the outcomes
failed.
Evaluation Models
• 1. Pre-evaluation
• 2. Goal based (Tyler, 1950)
• 3. Realistic evaluation (Pawson & Tilley,1997; Sanderson, 2002)
• 4. Experimental
• 5. Contingent evaluation (Legge, 1984)
• 6. Action learning (Reason & Bradbury, 2001)
Pre-evaluation (Goodman & Dean, 1982)
• The extent to which it is likely that... A has an impact on b
• Scenario planning
• Evidence based practice
• – All current evidence thoroughly reviewed and synthesized
• – Meta-analysis
• – Systematic literature review

• Formative v summative (Scriven, 1967)

Conti….
• Pre-evaluation issues

• Based on theory and past evidence: not clear it will generalize to the specific case

• Formative: influences planning

• Argument: to understand a system you must intervene (Lewin)

Goal Based Evaluation Tyler (1950)
• Objectives used to aid planned change
• Can help clarify models
• Goals from bench marking, theory or pre-evaluation exercises
• Predict changes
• Measure pre and post intervention
• Identify the interventions
• Were objectives achieved?
 Realistic evaluation: Conceptual clarity
(Pawson & Tilley,1997)
• Evidence needs to be based on clear ideas about concepts
• Measures may be derived from theory
• Examine definitions used elsewhere
• Consider specific examples
• Ensure all aspects are covered
Realistic evaluation Towards a theory: What
are you looking for?
• Make assumptions and ideas explicit

• What is your theory of cause and effect?

• –What are you expecting to change (outcome)?

• – How are you hoping to achieve this change (mechanism)?

• –What aspects of the context could be important?

 Realistic evaluation Context-mechanism
outcome
• Context: What environmental aspects may affect the outcome?
• –What else may influence the outcomes?
• –What other effects may there be?

• Mechanism: What will you do to bring about this outcome?

• –How will you intervene (if at all)?
• –What will you observe?
• –How would you expect groups to differ?
• –What mechanisms do you expect to operate?
• Outcome: What effect or outcome do you aim for?
• –What evidence could show it worked?
• –How could you measure it?
Experimental
evaluation:

• Explain, predict and control by identifying causal relationships

• Theory of causality makes predictions about variables, e.g. training increases
productivity
• Two randomly assigned matched groups: experimental and control
• One group experiences intervention, one does not
• Measure outcome variable pre-test and post-test (longitudinal)
• Analyze for statistically significant differences between the two groups
• Outcome linked back to modify theory
 A Contingent approach to evaluation
(Legge, 1984)
• Do you want the proposed change programme to be evaluated? (Stakeholders)

• What functions do you wish its evaluation to serve? (Stakeholders)

• What are the alternative approaches to evaluation? (Researcher)

• Which of the alternatives best matches the requirements? (Discussion)

Action research

• Identify good practice (Reason & Bradbury, 2001) Action research

• – Responds to practical issues in organizations
• – Engages in collaborative relationships
• – Draws on diverse evidence
• – Value orientation - humanist
• – Emergent, developmental
Why not evaluate?
• Expensive in time and resources
• De-motivating for individuals
• Contradiction between “scientific” evaluation models and supportive,
organization learning models
• Individual identification with activity
• Difficulties in objectifying and maintaining commitment
• External evaluation ‘off the shelf’
• inappropriate and unhelpful
Conclusion
• Evaluation is very expensive, demanding and complex

• Evaluation is a political process: need for clarity about why you do it

• Good evaluation always carries the risk of exposing failure

• Therefore evaluation is an emotional process

• Evaluation needs to be acceptable to the organization

Conti…
• Plan and decide which model of evaluation is appropriate

• Identify who will carry out the evaluation and for what purpose

• Do not overload the evaluation process.

• Evaluation can give credibility and enhance learning

• Informal evaluation will take place whether you plan it or not