EVALUATION PHASE

hi the evaluation phase, the creative ideas that were generated in the previous
phase are further analyzed by discussing the apparent advantages and disadvan-
tages of each idea. The team's rating/ranking method for each idea is determined
using a matrix method that forces consideration of factors affecting the acceptabil-
ity of each idea, such as:

• Constructibility
• Maintainability
• Redesign time
• Initial cost savings
• Life cycle cost, savings
• Chance for owner's acceptance
• Recycling
• Production tune
• Delivery time
• New technology owner's preference
• Aesthetics
• Simplicity
• Bidding requirements
• Flow rates and patterns
• Agency approval
• Weather
• Utility relocation
• Conflicts
• Hazardous operations review
• Service requirements
• Safety
• Proximity to site
• Consistent installation requirements
• Vendor availability
• Data transfer

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 After rating the ideas, each highest-ranking recommendation is then further
developed.
The MVO team analyzes the ideas listed in the creative session and selects
the best ideas for further development. The listing of ideas is reviewed with the
design team to discuss the advantages/disadvantages in order to rank the ideas and
to benefit from the design team's point of view on the project. The creative idea
listing/judgment worksheet and the evaluation matrix are used here.
Up to this point in the MVO study, the primary emphasis is on the cost of
each recommendation. Using a weighted analysis, the MVO team may analyze
other pertinent factors including aesthetics, initial and operating costs, safety,
maintainability, operational reliability, and other areas, which are appropriate to
the project.

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 Evaluation Techniques

The team was advised on various methods for evaluating the ideas to determine
the best ones that should be considered for further development. One evaluation
method was to list advantages and disadvantages for each remaining idea, and
determine as a group a score the ideas would have when compared to the original
method or to another high-ranking idea. Options for producing parts may be dis-
cussed for reducing the shortage of parts needed during production in order to
produce a quality product shipped without error to reduce the number of returned
products.
List the criteria for evaluation that were established during the preparation
effort. List each alternative's advantages and disadvantages. Using any gener-
ally accepted ranking procedure, rank each idea on both how well it meets the
criteria and on how well it performs the required functions. Do this evaluation
as a team. If none of the alternatives performs every criterion satisfactorily,
return to the creative phase using the knowledge gained in evaluation and create
new alternatives.
An evaluation matrix works well when comparing competing alternatives.
The team leader may want to devise a matrix that includes all the important
components to be considered during the comparison of the alternatives. For
example, the important features to include on the chart would be ease of mainte-
nance, standard design, error free components, size of package/shipment, and
reduction in hours to monitor production performance. When each competing
alternative is ranked on this matrix, the team has a tool to possibly discard an
idea so that the team can develop the best ones to improve the products.
During the evaluation phase the team selected the most beneficial ideas to be
developed further during the development phase. Out of 100 ideas (all ideas are
not included yet) submitted by the team during the course of this study, the team
used the most significant ones that met the criteria of improving time, perform-
ance, cost to fix the computer, and returns for customer satisfaction.
The new ideas were grouped into four major areas that indicated a value
mismatch from the value index. Those four areas were (1) design, (2) production,
(3) shipping, and (4) accounting.

1. Design: Define product

• Specifications updated quarterly

• Design of parts per new standards
• Parts suppliers in lieu of making own parts
• Use computer aided design

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 2. Production: Form Products

• Consider setting up an independent repair shop and include incentives

for quick, effective repairs.
• Initiate an employee suggestion program to improve design, quality
and durability. Use significant rewards to encourage participation and
establish an effective review board with deadlines and decision-
making powers.
• Consider manufacture of product overseas to enhance profits.
• Augment the QA/QC programs. The existing programs (if any) do not
appear very effective.
• Augment the employee training programs relative to production con-
trols and zero defects.
• Set up an immediate random testing program of items at truck load-
ing. Use program to try and identify the weak links in the total deliv-
ery process.
• Set up a new program that will allow customers to get immediate ac-
tion on returnable units. Consider shipping a new unit as they ship the
old one back. Then you can take time for repairs with a less angry
customer.

3. Shipping: Ship product

• Review contracts with shippers

• Consider bulk discounts with the shippers
• Choose alternate shippers to largest supply areas
• Use direct shipping
• Consider outsourcing the shipping
• Create smaller hub distribution/shipping centers

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 4. Accounting: Control expenditure

• Streamline operations
• Develop new procedures for billing/receiving
• Follow up with customer complaints
• Review methods for financing
• Consider new techniques being used by competitors to attract new
customers

Ideas shown here will be supplemented with ideas received from e-mail
brainstorming session from others.
The writer has e-mailed each VP the list of brainstorming ideas that each are
to rank in their opinion is better than the way the HCC is now performing the work
in each leading category.
After the evaluation phase the team developed selected ideas further dur-
ing the next phase. Most of the study time during the one-week period is spent
during the development phase. The evaluation matrix used by the team during
this experiment is shown in the table.

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 Evaluation table used by team

Modification
0>
o

Accounting
Repair time

Repair cost

Disruption
Weight 1-9 9 = high
Rank 1-5 5 = high PU
3
o
H
,
Weight 9 8 7 8 8 5
Rank
Original design (repair in house) 4 3 3 3 3 2 139
36 24 21 24 24 10
Customer satisfaction 4 4 2 3 3 3 145
36 32 14 24 24 15
CS1 Set up database to de- 5 5 4 2 1 2 147
termine trends of repairs types 45 40 28 16 8 10
CS2 Commitment to quality 5 5 5 5 2 3 176 2
program 45 40 35 40 16 15
CS3 Protection during ship- 3 3 3 2 4 3 135
ping 27 24 21 16 32 15
CS6 Internet customer based 4 4 5 4 3 3 174 3
diagnostics program 36 32 35 32 24 15
Design
Dl Recheck design develop- 3 4 3 1 2 3 119
ment 27 32 21 8 16 15
D2 Check parts causing the 4 4 4 4 5 3 151 4
problems 36 32 28 32 40 15
Manufacturing
Ml Note batch numbers to 3 4 4 4 5 3 142
determine cause 27 32 28 32 40 15
Production
Pr3 Independent repair shop 5 4 5 5 5 4 212 1
with incentives 45 32 35 40 40 20

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 The evaluation matrix may be used for each major category to determine
which ideas are better than the current method and should be developed by the
team in the next phase. Those ideas ranking higher than the original design were
developed.

Copyright 2003 by Marcel Dekker, Inc. All Rights Reserved.

 (a) Cost worksheet
Original estimate New estimate
Item Unit No. units Cost/unit Total Cost/unit Total

(b) Cost validation

Item Original estimate New estimate

Function analysis
Item Function Cost Worth Comments
Verb Noun Kind*

*B = basic, S = secondary
Speculation phase
Charts used in Value Idea Basic function Other functions
Engineering
* List all ideas, evaluate later (brainstorming log)

System/component Fast diagram

How — • <—Why

Idea evaluation
System/component Basic function
Idea Advantages Disadvantages Rank*

*Rank: 1 = excellent (develop), 2 = good (hold), 3 = fair (discard)

Evaluation (g) J Analysis matrix

Phase List best ideas from ranking and comparison techniques
Judge the performance against the criteria
Decision criteria Total
Criteria reference B attribute
Criteria importance rating*

) = excellent, 7 = good, 3 = fair, 1 = poor

<h) Summary and recommendation

System/component
Proposed change

Cost summary
Original - proposed Savings
Total cost (orig.)
Total cost (prop.)
Savings
Present worth - saving