Software Qual J (2010) 18:387–406

DOI 10.1007/s11219-010-9099-2

An integrated framework for supply chain performance

measurement using six-sigma metrics

Lie-Chien Lin • Tzu-Su Li

Published online: 14 May 2010

Ó Springer Science+Business Media, LLC 2010

Abstract Developing a sound performance measurement model is a critical task for a

supply chain and its members in order to examine their current status and identify
improvement opportunities for steering their future direction. This research proposes an
integrated framework for supply chain performance measurement. It adopts the six-sigma
metrics and includes three components (i.e., team structure measurement, supply chain
process measurement, and output measurement) to provide a more complete coverage of
performance requisites, which has not been adequately addressed in relevant literatures
before. Also, a case is applied to demonstrate the feasibility of the model. Complying with
the application of this framework, the whole chain or individual members in a supply chain
can advance the performance in a more effective way.

Keywords Performance measurement  Six-sigma  Supply chain

1 Introduction

Measurement is the language of progress and provides a sense of current status. Thus,
performance measurement is always a vital issue for management. It helps identify
shortfalls and points toward pre-established goals. It is also a powerful behavioral tool,
which can convey to the workforce what is important to the organization and how it
becomes a primary determinant of what is done. In the context of a supply chain and its
associates, performance measurements exist at different levels.
Effective supply chain management (SCM) is treated as a key to building a sustainable
competitive edge through improved inter-firm and intra-firm relationships (Gunasekaran

L.-C. Lin
Department of Logistics Management, National Kaohsiung First University of Science
and Technology, No. 2, Jhuoyue Rd., Nanzih District, Kaohsiung 81164, Taiwan, ROC

T.-S. Li (&)
Graduate Institute of Management, National Kaohsiung First University of Science and Technology,
No 7 Aly 5 Ln 2, Paohua Rd., Rente Township, Tainan County 71751, Taiwan, ROC
e-mail: wjvali@anet.net.tw

123
388 Software Qual J (2010) 18:387–406

et al. 2001; Webster 2002). A magnitude of benefits has been attributed to SCM, including
increased market share, reduced cost, and enhanced customer relationship (Ferguson
2000). However, there is evidence to suggest this may be hyperbole rather than organi-
zational reality (Thomas 1999). Additionally, an international study of modern manufac-
turing practices reported only a moderate perceived effectiveness of SCM (Clegg et al.
2002). Thus, it is critical to develop an effectual measurement system for supply chain
performance since it can facilitate a deeper understanding of the supply chain and improve
its overall performance (Chen and Paulraj 2004; Sharma and Bhagwat 2007).
For the performance measurement, Neely et al. (1995) identified a number of approa-
ches including balanced scorecards (Kaplan and Norton 1992), performance measurement
matrix (Keegan et al. 1989), performance measurement questionnaires (Dixon et al. 1990),
and criteria for measurement system design (Globerson 1985). Beamon (1998) categorized
the performance measures in existing literatures into two groups: qualitative and quanti-
tative, where customer satisfaction and responsiveness, flexibility, supplier performance,
cost, and others used in supply chain modeling were discussed. He also identified three
types of measurements: resource, output, and flexibility in supply chain environments; each
of them encompassed several metrics (Beamon 1999). Gunasekaran et al. (2001) developed
a framework for measuring the supply chain performance in the strategic, tactical, and
operational levels, which dealt mainly with supplier, delivery, customer service, and
inventory and logistics costs. The metrics and measures used for the supply chain activities
included the following: plan, source, make/assemble, and delivery/customer. Gunasekaran
et al. (2004) developed a framework to promote a better understanding of SCM perfor-
mance measurement and metrics using an empirical study of selected British companies.
Confronted with today’s complex and challenging environment, continual improvement
(CI) has been recognized as an effective strategy for the entire supply chain to widen the
competitive advantage gap between competitors (Sharma et al. 2008; Hara 2002; Green
et al. 2007). Yet, any good management approaches for CI must have an effective per-
formance measurement to identify the current status, find improvement opportunities, and
steer toward the future direction. Traditional performance measurements based on
accounting figures such as sales turnover, profit, debt, and ROI might serve well as
warning flags about performance problems, but they do not convey the reasons for the
problems. Kanji (2001) argued that an organization would need a framework that was
comprehensive, flexible, and easy to adopt in order to measure its business excellence.
Shepherd and Gunter (2006) highlighted a range of limitations pertinent to supply chain
performance measurement, including the following: no long-term performance measure-
ment; no focus on strategic issues; little supply chain context; and no systematic approach.
Their studies stressed the need for new measurement systems and metrics to address these
deficiencies. Recently, some researchers have attempted to respond to the limitations by
designing systemic and balanced performance measurement systems. In 1999, the supply
chain operation reference model (SCOR) was introduced. It was known as ‘a systematic
approach for identifying, evaluating, and monitoring the supply chain performance’
(Hakanson 1999; Stephens 2001); and it identified some strategic criteria concerning cost,
cycle time, delivery performance, order fulfillment performance, etc. These metrics are
useful and satisfy the attributes of good metrics as discussed by Rose (1995). Some
researchers augmented the SCOR model by combining it with decision-making tools such
as the analytic hierarchy process (AHP) (Huan et al. 2004; Li et al. 2005). Bhagwat and
Sharma (2007a) developed a balanced scorecard approach for the evaluation and mea-
surement of SCM in a balanced way; they also proposed the use of AHP methodology as
an aid of supply chain decision-making (Bhagwat and Sharma 2007a, b). In addition, Chan

123
Software Qual J (2010) 18:387–406 389

Year 1980 1990 2000 2010

Globerson Dasgupta. Use

1985 2003 six-sigma
metric
Criteria set for
measurement Keegan et al.
1989 Huan et al. SCOR
2004 +AHP

Matrix Dixon et al.

approach 1990 Limitations of
Shepherd & supply chain
Neel et al. Gunter.2006 performance
1995 measurement
Questionnaire
approach
Overview of
measurement Beamon. Bhagwat &
1998 Sharma. Use of
2007 AHP+
FUZZY
Introduce supply
chain concept into SCOR
performance model, Lin&Li
1999 2008

A framework for Combine

measurement in the Gunasekaran six-sigma
strategic, tactical, and et al., 2001 and CMMI
operational levels

Fig. 1 The development of supply chain performance measurement

and Qi (2003) favored fuzzy ratios for selecting metrics; however, they paid little attention
to the performance measurement across the whole supply chain. Figure 1 summarizes the
development of performance measurement methods in the literature.
In summary, previous frameworks related to supply chain performance possess some
weaknesses. First, the majority of articles focused on the study of intra-organizational
performance; they failed to grasp the idea of how the supply chain has performed as a
whole (Lambert and Pohlen 2001). Secondly, the previous research did not consider the
variation of measured values. These values did not present the distribution and uncertainty.
Thus, the decision makers found it difficult to find real performance values, identify weak
areas, take corrective actions, and make continual improvements. Thirdly, there existed no
common metrics for evaluating different processes on the same scale. Different charac-
teristics of associated processes cannot be compared without using the correct metrics. For
example, ‘time’ and ‘cost’ are both critical attributes among different processes, but their
performances are not comparable. Fourthly, the process teams should have motivation,
capacity, and authority to improve processes and their results. An effective team will
enhance the CI performance. Thus, human attributes such as cooperation, skill, commu-
nication, etc. should have been considered as important dimensions of supply chain per-
formance (Dasgupta 2003), but previous researches did not integrate these human
attributes into the supply chain performance measurement model. To overcome these
deficiencies and drawbacks, this paper proposes an integrated framework with a solid
device for continual improvement in supply chain management. The framework can be
utilized to measure the performance at each level across the supply chain, which has not
been adequately addressed in the literature before. As shown in Fig. 2, the supply chain
includes various entities, and each entity encompasses organizational processes, sub-pro-
cesses, and activities. The framework measures the overall performance of supply chain
system processes and cascades down to the lowest level, where those activities of a sub-
process within an individual organization process take place. The framework also evaluates
the performance across the entire supply chain using six-sigma metrics. Six-sigma metrics

123
390 Software Qual J (2010) 18:387–406

Structure, process, output Dimension layer

Sub-dimension for each Sub-dimension layer

Supply dimension
chain
level ...
Supply
the lowest dimension Lowest layer
chain
processes

Organizational processes
Individual
organization
level Organizational sub-processes

Organizational activities

Fig. 2 Multi-level supply chain performance

can easily identify the bias of average measured values; furthermore, they can simplify
comparison to a common denominator (the sigma level) when comparing the performance
of any two processes, irrespective of their nature. Through the implementation of this
framework, an organization will have the capability of monitoring its progress at a given
point of time at each level within a supply chain. This paper is organized into four sections,
the first of which is this introduction; section two proposes the performance measurement
framework and describes its theoretical background; in section three, a case study is used
to demonstrate the application of this model. Finally, we make a conclusion and present
some suggestions in section four.

2 The framework for supply chain performance measurement

With the purpose of solving the deficiencies previously mentioned for supply chain per-
formance measurement, this paper proposes an integrated framework as shown in Figs. 3
and 4. It includes both horizontal measurements and vertical measurements. The horizontal
measurements are for the entire chain and every entity of a chain, while the vertical
measurements are for the organizational processes within each entity. In the horizontal
way, the measurement model for whole chain is presented in this paper; while in the
vertical way, an improvement model for individual organizations or processes is proposed
by Lin and Li (2008). They first developed an improvement model by integrating both Six-
sigma and Capability Maturity Model Integration (CMMI) methods, and a detailed
application procedure in the auto industry was presented, as shown in the lower part of
Fig. 4.
The measurement model uses the six-sigma metrics as an improvement tool for the
supply chain performance. Six-sigma is a systematic problem-solving approach based on
five stages of define–measure–analyze–improve–control (D–M–A–I–C). It targets the root
cause of variation and redefines the process for a long-term outcome. The metrics have
been applied in a variety of settings to solve a broad array of issues including manufac-
turing capabilities, customer service, aircraft design and so forth. Nevertheless, its
uniqueness was primarily attributed to the ‘measure’ stage. Pande et al. (2000) pointed out
that people could better understand the six-sigma measurements which can yield RTY (roll
throughput yield), dpmo (defects per million opportunities), or sigma level, as people can
equate these goals to dollar impact; particularly, RTY has the advantage of calculating

123
Software Qual J (2010) 18:387–406 391

Tier 2 Tier 1 Focal company Customer End-customer

Supplier Supplier
c c s c s c s s c
s s c c c c c s
s c s s c c s c
c Whole Measurement
Chain model
Chain process

Organizational process

c : Core process of an organization

s : Supporting process of an
organization

Fig. 3 The horizontal and vertical measurements for supply chain

Measure
(Six-sigma metric)
Measurement
model
Supply chain processes
Output (process)
Team structure
Process

Output (system)

IDS (integrated Periodic Improvement

CMMI model:
Requirements development assessment
system) An integration of
six-sigma and
CMMI
(Lin & Li, 2008)
Six-sigma engine

Fig. 4 The integrated framework for supply chain performance measurement

overall performance by connecting individual entities to a supply chain. By applying six-

sigma metrics, the enterprises can identify problems of each performance point; the metrics
can also compare any process/activity irrespective of their nature. Another motivation of
using the six-sigma metrics is because both SCM and six-sigma are process-oriented
approaches. The supply chain is a set of key processes, and six-sigma metrics have the
capability to calculate how well a process is performing.

123
392 Software Qual J (2010) 18:387–406

The model includes three components to provide a complete coverage for supply chain
performance measurement, i.e., human team structure, supply chain processes, and output
(process output and system output), as shown in upper part of Fig. 4. Each component is
treated as a dimension and can be deployed down to their respective sub-dimensions. First,
the team structure refers to the characteristics of a performance team, such as communi-
cation, participation, commitment for improvement, etc. These appraisals constitute the
structure measurement. In the literature, several models have been proposed for individual
performance measurement, such as the behavior-based evaluation, ongoing feedback,
multiple raters, and trained appraisers (DeCenzo and Robbins 1996); nevertheless, few
studies examine team performance. Therefore, we will add team measurement to com-
plement previous appraised models. As for team measurements, Janz and Harel (1993)
proposed a model to identify behavioral patterns of teams and figured out what to do in
order to achieve a team’s targets more effectively. However, they did not consider the
measurements of a process and its output. Secondly, the assessment of a supply chain
process indicates whether a process is efficient when producing goods or services to meet
customers’ requirements. Zigon (1994) identified three major points for a performance
measurement system; they were final results, key process steps, and input; however, it only
focused on a ‘‘single’’ process approach for performance measurement. Thirdly, the pro-
cess output indicates the extent to which the process has been implemented. System output
is composed of several single processes; it might be difficult to measure or compute from
individual processes. In the literature, few performance measurement models consider
system output measurement, which is often produced from several single processes in a
supply chain. Thus, system output should be included in the measurement system for
supply chain processes.

2.1 Six-sigma metrics

Six-sigma was introduced in 1987 and adopted by GE to achieve remarkable benefits. It

was defined by Harry (1997) as an improvement approach, a strategy, and also a goal.
Fontenot et al. (1994) discussed how six-sigma could be used to measure customer sat-
isfaction and improve the service process with a case example from Texas Instruments’
Defense System and Electronics Group. A literature review shows that the six-sigma
approach has recently been deployed in areas like human resources (Wyper and Harrison
2000), the health sector (Crago 2000), research and development (Johnson 2002), and
supplier development (Avery 2001; Wang et al. 2004). Most reports have focused on how
the six-sigma practices could be implemented in one organization successfully. Further-
more, Dasgupta (2003) applied six-sigma metrics to measure and improve supply chain
processes, and he suggested that human attributes should be integrated into the perfor-
mance measurement systems.
As for the attribute measurement, assume that we consider a task of fulfilling payments
with n activities made in each payment and each activity may be referred as a defect
opportunity for each payment. If there are k wrong payments out of m payments within
1 week, then there are mn opportunities totally, and the dpo (defects per opportunity) can be
denoted by k/mn. Thus, the yield for fulfilling a payment through all activities is given by
(1-dpo)n, which represents the probability of no defects when a payment is processed. The
yield, using the Poisson distribution, can be approximated by e2dpu where dpu = dpo 3 n.
The z value or sigma value corresponding to the yield can be obtained using normal standard
distribution table. The relationships among sigma level, yield, dpmo, and Cpk (process
capability index) for long-term improvement are shown in Table 1. Another useful metric is

123
Software Qual J (2010) 18:387–406 393

Table 1 Relationships of sigma

Sigma level Dpmo Yield % Cpk
level, dpmo, yield, and Cpk
1r 697,672 30.23 -0.17
2r 308,770 69.13 0.17
3r 66,811 93.32 0.50
4r 6,210 99.379 0.83
5r 233 99.99767 1.17
6r 3.4 99.99966 1.50

the rolled throughput yield (RTY) in the context of supply chain management. It is defined as
the probability of being able to pass a unit of product without defects through the entire chain.
For the chain, RTY = PYi, that is the product of Y1, Y2,.., Yn for n individual entities (each Yi
is given by e2dpu). The rolled throughput yield can identify the potential defects easily in the
entire chain, when compared with the conventional notion of yield. In other words, sigma
level (dpmo or yield) can be obtained from data directly; using it, one can understand the
extent to which the current performance of processes and activities meets targets.

2.2 Team structure measurement

The use of an effective work group in process improvement is gaining popularity recently.
An effective work team could increase efficiency and improve related outcomes (Champion
et al. 1993). Shepherd and Gunter (2006) also hinted at the need to measure human char-
acteristics in performance measurement system to identify problems easily and accelerate
CI effectively; however, few articles in the literature focused on this issue. The eleven most
common group characteristics related to an effective work group will be adapted in this
framework. They include self-management, participation, task interdependence, goal
interdependence, feedback and reward, flexibility, training, managerial support, commu-
nication and cooperation, social support, and workload sharing. The questionnaire of team
structure characteristics, composed of 35 root behavioral patterns given in Champion et al.
(1993), will be adopted for surveying each member of each entity in a supply chain. The
following algorithm, provided by Janz and Harel (1993), will be used for the measurement.
(1) The members of each entity, who are realizing a particular process in a supply chain,
should review the 35 root behaviors and independently assign 100 points for each. (2) The
top management determines the specification of each group characteristic. (3) The work
analyst checks the consistency among team members using Cronbach’s alpha. If the alpha is
greater than 0.75, the process skips to step 5. (4) The analyst leads a discussion of behavioral
patterns that have the greatest variance of weighting by team members. Then, the analyst
gathers team members for possible weight changes and goes back to step 3. (5) The analyst
computes scores for each of the 11 group characteristics. The data is put into MINITAB
software (Minitab Inc. 2000) to attain the yields and sigma levels for these characteristics
and the entire supply chain. (6) For behavioral patterns with a relatively low sigma level, the
team members will investigate why those behaviors occur and how they can be corrected.

2.3 Supply chain process measurement

Supply chain management is the integration of key business processes from end user
through to original suppliers that provide products, services, and information to add value

123
394 Software Qual J (2010) 18:387–406

for customers and other stakeholders. These key processes penetrate function silos within
the company and the various corporate silos across the supply chain (Lamber and Cooper
2000). Each of them has its own chain network structure. For example, the network
structure for the product realization is not completely the same as that of procurement
process. According to Lamber et al. (1998), the major steps of defining a supply chain
network are to identify the entities of the supply chain, define the key processes, and
establish the network structure for each process. Assuming dpuij represents the number of
defects in entity j of tier i, then the yield can be obtained by Y ij ¼ edpuij : The roll
throughput yield with respect to certain characteristic of a particular process of the entire
supply chain equals the product of all individual yields. That is RTY = PiPjYij, and the Z
value or r value can be obtained from the transformation of 2ln(RTY). An approach for
the calculation of a particular process in the entire supply chain is presented below. (1)
Prepare the supply chain network for the specified process. (2) Identify the critical char-
acteristics related to this process from the customer’s viewpoint. (3) Determine the
specification of characteristics identified in step 2. (4) Obtain sample data and compute the
r level, yield, and roll throughput yield.

2.4 Output (process) measurement

The output is the overall outcome of the processes, and it directly affects the customer
satisfaction. Furthermore, some critical characteristics could not be properly measured in
the process until the final product is done. For example, the purity of chemical products
can only be measured at the output stage even though relevant critical process parameters,
such as gradient of SO2 or HCL, have been identified and measured in each process.
There are two approaches for the output measurement, including the customer satisfaction
survey and the performance evaluation of process output. The latter one, which involves
the measurement and monitoring of critical characteristics, will be applied in this
framework since it is a more objective measurement and complements the weakness of a
subjective survey.

2.5 Output (system) measurement

The system output is produced from the relevant individual processes. In general, not all
critical characteristics of final product or service will be completed until all relative
individual processes have been accomplished. That is to say, it would not be enough for
one single process to fulfill all requirements of that product to meet external customer
needs. For example, delivery time performance may be composed of the outputs of pro-
curement process, realization process, order fulfilling process, and so on. Thus, the system
output, which is composed of several key characteristics must be measured and monitored
(Lynch and Cross 1995). The survey approach is a good measurement, which is often
utilized in academic research and industry practice, but it is still subjective. This research
proposes an objective approach for measuring the system output as following. (1) Identify
the characteristics, which are critical to customer satisfaction. (2) Determine the weights
for each characteristic. The analytic hierarchy process (AHP) (Saaty 1980) will be applied
for deciding the weights. (3) Define ‘defect’ of each characteristic. (4) Collect data and
calculate yield and sigma level of each characteristic. (5) Calculate the overall sigma level
of customer satisfaction.

123
Software Qual J (2010) 18:387–406 395

2.6 Result display by radar charts

Since the proposed framework is a multi-dimension measurement system, with the purpose
of comparing the result of each dimension and sub-dimension, we recommend the use of
radar charts to display the performance result. The radar chart is composed of a circle and
equally spaced rays emanating from the center. Each ray represents a performance
dimension as shown in Figs. 6, 7, 8, and 9. Observed sigma values of variables are marked
on the respective rays. The sigma value ‘0’ is set at center and the end of each ray is the point
of 6r. A performance dimension may be composed of several sub-dimensions. Thus, the
number of lower-level radar charts would be as many as the number of sub-dimensions. It
would depend on how the ‘parent’ dimension is cascaded down to the ‘children’ dimension.
For example, the dimension of team structure encompasses several sub-dimensions (e.g.
self-management, participation, task interdependence), and each sub-dimension may
include some relevant sub-sub-dimensions. On the other hand, the measurement frequencies
may vary from one figure to another. This depends on the project needs, availability of data,
and cost incurred. The frequency of team structure measurement suggests at least twice a
year for inexperienced teams and once per year for experienced teams (Storey 1989); the
frequency of process output may be obtained once every 2 months and twice a year for
system output. As for the frequency of radar charts, they can be updated once for every
4–6 months, which is similar to the period of six-sigma improvement program.

3 An illustrative case study

ABC Group, a global manufacturing company, was founded in 1971 and went public in
Taiwan. The headquarters is located in Taiwan, and it owns five interdependent companies
with different functions of sales, manufacturing, and distribution. To meet the customer
demands, these companies established an ISO 9001 process-based management system and
obtained certification registration. The singular performance has the following improvements
including good product quality, time-based logistics, and continuous cost reduction; although
the overall performance of ABC Group still has a lot to improve. This paper will demonstrate
how ABC Group identifies its supply chain performance characteristics, and how the overall
rolled throughput yield can be computed for the supply chain performance measurement.
Figure 5 shows ABC Group’s chain for particular product realization process. It contains

Tier-2 supplier Tier-1 supplier Processing center Sales branch Tier-1 customer

Machining
5 1

4 2
6
Surface
treatment 3

7
Information flow
Material flow
Packaging

Fig. 5 Supply chain network of product realization process

123
396 Software Qual J (2010) 18:387–406

three sales offices, which are entities , `, ´, and they are in USA, China, and Taiwan,
respectively; one processing center (entity ˆ) is in Taiwan; and one manufacturing factory
(entity þ) is in Taiwan. Another two tier-1 suppliers (entity ˜, Þ) and other relative suppliers
are also critical in this chain, although they do not belong to the ABC Group. In order to be a
competitive global supplier, ABC Group sets a 5r level as future objective for the following
2 years and decides to adopt the proposed framework as a performance measurement tool for
continual improvement of time-based logistics. The CEO of ABC Group creates a process
improvement office (PIO), which contains seven top management members from each entity.
The office has the responsibility and authority to implement this framework and monitor the
performance of the supply chain. For simplicity, we confine the scope to tier-1 suppliers and
tier-1 customers. The sales branches are responsible for reviewing orders from customers and
dispatching the information to the processing center. The processing center makes the
decision to outsource the production function to tier-1 suppliers. The tier-1 suppliers will
implement manufacturing functions and accomplish the orders within the stipulated dates.
The final products are inspected at entity þ and delivered to tier-1 customers.

3.1 Team structure measurement

All members of each entity have answered the questionnaire on effective group behavioral
patterns, and then the PIO defines the dpu (defects per unit). Each member is considered as
a unit, and any member’s characteristics that score less than the specified value will be
considered a defect. The data are collected, analyzed, and identified to be reliable as the
alpha is 0.92 (greater than the criterion 0.75). The r level and yield of each behavior
characteristic are calculated for the individual entity and entire supply chain. The statistics
software is utilized to calculate the dpu and r level. The results are shown in Tables 2 and
3. Figure 6 displays the performance for each structure category as dots on the respective
dimension. The higher the sigma level is, the more contributive and effective the team is.
Finally, the overall performance, including both ‘currently’ and ‘after 6 months’, is
calculated and plotted on the structure ray of the radar chart as shown in Fig. 9.

3.2 Supply chain process measurement

The following steps are carried out for the process measurement. (1) The PIO identifies the
supply chain network (as shown in Fig. 5) and prepares a product realization flowchart. In this
process, the sales branch first reviews orders from customers every day and passes these data to
the processing center, which is to schedule the outsourcing for various functions. Then tier-1
suppliers conduct the assigned work according to the stipulated time. (2) The ‘time’ charac-
teristic is considered crucial to our customers. Thus, the key entities involved with this char-
acteristic are identified in this process by the PIO. (3) Determine the specification and define the
‘defects’ of affiliated entities as summarized in Table 2. (4) The PIO then computes both yield
and sigma level for each critical entity, and roll throughput yield is obtained as the overall
performance for the entire process. The results of individual and overall performance are
displayed in Figs. 7 and 9, respectively, and the measured values are shown in Tables 2 and 3.

3.3 Output (process) measurement

The external customer expectations can be measured by two approaches, which are
a questionnaire survey and a fact-based monitor. Since the survey is subjectively biased,

123
 Table 2 Performance measurement results
Dimension Sub-dimension Entity Definition of defect Yield (%) Sigma

Team structure Self-management All entities Score less than 80 points for each member 84.6 2.55
Participation As above Score less than 75 points for each member 88.0 2.65
Task interdependence As above Score less than 70 points for each member 84.0 2.47
Goal interdependence As above Score less than 70 points for each member 91.8 2.80
Feedback and reward As above Score less than 80 points for each member 75.0 2.18
Flexibility As above Score less than 70 points for each member 72.0 2.10
Training As above Score less than 80 points for each member 84.0 2.50
Software Qual J (2010) 18:387–406

Managerial support As above Score less than 80 points for each member 99.4 4.02
Communication and cooperation As above Score less than 80 points for each member 98.2 3.70
Social support As above Score less than 80 points for each member 99.7 3.23
Workload sharing As above Score less than 75 points for each member 94.1 3.05
Overall 88.3 2.71
Process Order review Entity `´ Taking more than three days when reviewing each order 97.1 3.40
Outsourcing Entity ˆ Taking more than two days after receiving each order 94.8 2.90
Machining Entity ˜ Work is not completed within the scheduled time 86.0 1.90
Heat treatment Entity Þ Work taking more than two days after receiving previous 87.0 2.10
product
Final inspection Entity þ Inspection is not completed within one day after receiving final 98.6 3.70
product
Packaging Entity þ Taking more than one day after finishing inspection 99.2 3.90
Overall 67.3 2.64
Process output Delivery time (on time) Entity þ Delivery is not completed within the customer’s demand date 88.2 2.70
Feedback time Entity `´ Response to customer more than two days after receiving 86.6 2.61
service demand
Overall 87.4 2.67
397

123
 Table 2 continued
398

Dimension Sub-dimension Entity Definition of defect Yield (%) Sigma

123
System output Process capability All relative entities Process capability less than 1.33 for each order 97.9 3.55
of this process
and others
Delivery (on time) As above Delivery is not completed within the customer’s 88.2 2.70
demand date
Response (problem solving) As above Solve problem more than three days after receiving 99.4 4.0
customer’s complaint
Cost (failure cost) As above Events of reject or repair happen for each batch 99.6 4.15
Overall 96.5 3.31
Software Qual J (2010) 18:387–406
Software Qual J (2010) 18:387–406 399

Table 3 Yield for each entity in the first period

Dimension Sub-dimension Entity

 ` ´ ˆ ˜ Þ þ

Team structure Self-management 96.2 87.8 90.5 90.4 75.0 68.0 84.5
Participation 97.0 88.3 96.5 96 77.8 70.5 92.0
Task interdependence 93.1 87.1 92.0 91.0 74.3 67.1 84.0
Goal interdependence 96.6 90.3 94.5 94.5 87.5 87.5 92
Feedback and reward 85.0 73.7 78.2 83.8 66.8 67.0 66.6
Flexibility 80.1 70.0 77.0 81.2 65.0 65.9 65.4
Training 93.0 89.0 91.0 93.0 73.0 74.0 73.8
Managerial support 99.7 99.4 99.6 99.8 99.1 98.9 99.0
Communication and cooperation 99.0 98.0 99.0 99.0 97.0 97.8 97.4
Social support 99.8 99.6 99.7 99.8 99.6 99.6 99.5
Workload sharing 96.8 94.5 94.9 96.6 92.0 91.7 92.4
Process Order review 98.5 93.8 99.0 – – – –
Outsourcing – – – 94.8 – – –
Machining – – – – 86.0 – –
Heat treatment – – – – – 87.0 –
Final inspection – – – – – – 98.6
Packaging – – – – – – 99.2
Output (process) Delivery time (on time) – – – – – – 88.2
Feedback time 89.1 83.0 87.8 – – – –
Output (system) Process capability 99.3 99 99.2 99.4 98.3 98.2 98.1
Delivery (on time) 90.0 85.4 89.2 89.7 87.2 87.8 88.0
Response (problem solving) 99.8 99.6 99.7 99.8 98.8 97.8 99.0
Cost (failure cost) 99.8 99.5 99.7 99.8 99.5 99.5 99.1

the PIO adapts the latter approach to collect information as to whether the organization has
met customer requirements. In other words, customer satisfaction is monitored through
continual performance measurement of the realization process. Two performance indica-
tors, on-time deliveries and feedback speed for processing customer service, are deter-
mined by the PIO, and they constitute the output measurements. The PIO sets the lower
limit for each output characteristic, does the data collection, and calculates the sigma value.
The results for the first period are summarized in Tables 2 and 3. Radar chart represen-
tation is not used here since there are only two measurements in this output dimension.

3.4 Output (system) measurement

The following steps are carried out for the system output measurement. (1) The PIO knows
that not all customer requirements are treated equally, nor will customers react to a ‘defect’
in the same way. To identify the crucial factors, a 6-member team is formed for entities ,
`, ´, who contact customers directly and know the customers’ demands well. Then we
refer to 12 performance metrics from the SCOR model (SCC 1999); four critical factors are
identified, including cost, response, process capability, and on-time delivery. The survey
shows how well the customers are satisfied in this supply chain. (2) The analytic hierarchy

123
400 Software Qual J (2010) 18:387–406

Fig. 6 Sigma values in the team

structure measurement

Fig. 7 Sigma values in the

process measurement

process (AHP) (Saaty 1980) is applied for calculating the weights of customer require-
ments. The PIO conducts AHP pairwise comparison through the 6-member team. By
utilizing Expert Choice 11(2004), the weights are assigned for each factor and they are cost
(0.115), response (0.288), process capability (0.386), and on-time delivery (0.211). Fur-
thermore, the CR (consistency ratio) value, less than 10%, indicates that the consistency of
evaluation is acceptable. (3) The PIO decides the definition of ‘defect’ for each factor. (4)
The data for the six-month period is collected, and the sigma levels and yields for each
factor are calculated. (5) Finally, overall yield is obtained by the sum of multiplying each
yield with its assigned weight, which is accomplished in step 2. Figures 8 and 9 display the
results of individual and overall performance measurements accordingly, and the measured
values are shown in Tables 2 and 3.

3.5 Discussion

The top-level radar chart allows one to observe the overall performance at a glance. From
Fig. 9, although the performance of the team structure has advanced from 2.71r to 3.5r,
the top management realizes that after 6 months of six-sigma improvement program, the
overall performance of each dimension still fails to meet the specified target (5 sigma
level). With the information of r levels for each dimension, the top management can utilize
the company resources better and execute continual improvement more effectively. Next,
in order to identify the weak areas, the PIO needs to observe the second-level radar charts.

123
Software Qual J (2010) 18:387–406 401

Fig. 8 Sigma values in the

system output measurement

Fig. 9 A radar chart for overall

performance based on sigma
metric

As shown in Fig. 6 and Table 2, the managerial support, with the highest sigma level
(4.02r), is accomplished well, while the flexibility, with lowest sigma (2.10r), is executed
poorly. It means that the management shows support for the team, yet the team members
have less flexibility with knowing each other’s jobs and job rotation. Thus, ABC Company
may need to use more resources to enhance flexibility. As for process measurement in
Fig. 7, the r levels of machining (1.9r) and heat treatment (2.1r) are low in the process
measurement. Similarly, the management finds that the on-time delivery has the lowest r
value of 2.7 from system output measurement in Fig. 8. It implies that there is room for
progress in all these weak areas for further analysis and improvement.
In order to pinpoint improvement opportunities for deficiencies, the lower-level charts or
tables must be examined carefully. As shown in Table 3, entity  has a yield of 96.2% (that
is 3.27r) with respect to the self-management in the dimension of team structure. After
collecting data from 20 members of entity  and analyzing the data through MINITAB
software (Minitab Inc. 2000), it is clear that there are 37,962 ppm (parts per million) above
the lower limit, 80, as shown in Fig. 10. This figure also shows some important information
including values of the mean, standard deviation, process capability Cpk, and ppm of the
relevant performance. The top management can find room for progress by comparing the
within and overall performance values. The within performance value indicates the potential
process capability for improvement, and the overall performance value is for the current
system performance. In addition, Cpk (at only 0.59) is the crucial index for process capa-
bility breakthrough, since the general business requirement has to be above 1.33.

123
402 Software Qual J (2010) 18:387–406

Input data: 85, 89, 92, 82, 81, 85, 86, 91, 88, 86, 83, 86, 84, 81, 87, 85, 89, 83, 82, 81

LSL
Process Data
USL *
Within
Target *
LSL 80.0 Overall
Mean 85.3
Sample N 20
StDev (Within) 2.98619
StDev (Overall) 3.33795

Potential (Within) Capability

Cp *
CPU *
CPL 0.59
Cpk 0.59

75 80 85 90 95
Cpm *

Overall Capability Observed Performance Exp. "Within" Performance Exp. "Overall" Performance
Pp * PPM < LSL 0.00 PPM < LSL 37962.36 PPM < LSL 56165.47
PPU * PPM > USL * PPM > USL * PPM > USL *
PPL 0.53 PPM Total 0.00 PPM Total 37962.36 PPM Total 56165.47
Ppk 0.53

Fig. 10 Distribution of defects for self-management in the dimension of team structure for entity 

In order to implement continuous improvement more effectively, the top management

decides that sigma values of all dimensions are to be charted for every six months for the
entire supply chain, and the management must carefully monitor their trends. Yield values
for individual entities are also to be monitored, and weak areas are investigated on a
continuous base. Corrective actions are discussed and determined in the management
review meetings and must be accomplished within the stipulated time. In addition, the
appropriate feedback is given to external suppliers whenever their performances show a
downward trend. Following the application of this model, ABC Group effectively identifies
85 poor areas and creates 135 six-sigma improvement projects within 2 years. The com-
pany achieves the target of five-sigma level for each dimension across the entire supply
chain, which was never achieved before. Simultaneously, all employees learn the
knowledge of the sigma-based improvement model, and everyone has confidence that the
company will achieve the six-sigma level in near future.

4 Conclusion

The performance measurement framework proposed in this paper provides a solid device
for continual improvement in supply chain management. It is expected that this framework
will help companies measure and monitor supply chain performance effectively, which
includes team structure, process, and output. The application of this framework contributes
to the management comprehensive and easy-to-understand information regarding the
strengths and weaknesses of the supply chain entities. For example, in this case study, the
sigma level of the system output is 3.31r, whereas that of the team structure dimension is
2.71r. It is clear that there is room for progress in the latter dimension. The top man-
agement will look at the lower-level yields and take corrective actions within the team
structure and its affiliated entities. We can further extend the method to identify

123
Software Qual J (2010) 18:387–406 403

improvement opportunities in second- or third-level yield (that is the internal operations

within an organization) and facilitate teams to accomplish continual improvement.
The case application shows that the integrated framework has several advantages over
traditional ones. First, the framework can determine the overall performance of each
dimension of a supply chain and cascade down to the lowest level, which are those entities
of a sub-dimension. It could evaluate the performance across the entire supply chain using
sigma metrics. Secondly, it would simplify the comparison to a common denominator (that
is sigma level) and compare the performance of any two entities of a supply chain or two
supply chain processes irrespective of their nature. The sigma metrics or yield levels of an
individual entity provide know-how to establish improvement projects. Thirdly, using the
radar charts of sigma level, an organization can monitor its progress at any given point of
time at each level within a supply chain; thus, the organization can keep track of the
records of its progress.
Some important issues should be kept in mind while executing this model. First, we
need to understand that any two entities of a supply chain or two different dimensions may
not be comparable on the sigma scale, unless the proper units and right specifications are
carefully chosen and defined. Secondly, the rolled throughput yield may be a more valu-
able performance indicator from the top management’s perspectives, as the supply chain is
a complex combination of sequential and parallel processes. Thirdly, the application of this
framework is based on facts, thus, the system shall require a sound database to collect data,
identify the corresponding population, and test the normality of each population. Lastly,
this measurement system should not be limited by strict algorithms or never-changing
measurements. The behavioral patterns may evolve or change as the organization matures
after some time, thus the original measurement may not be necessary or meaningful. On
the other hand, the modification of the performance requirements would be necessary if
some progress has been achieved, the possibility for further improvement is little, or if
further investment is not worthwhile.
We must stress that sigma-level ratings have various definitions for defects in each
dimension. While using six-sigma metrics to measure the performance, the supply chain
members must align these dimensions and specifications with both external environmental
and internal organizational changes. In other words, the measurement framework has to be
treated as a dynamic mechanism that responds to competitive market environment
(Waggoner et al. 1999; Kennerley and Neely 2002, 2003). Thus, future research may
integrate this framework with a dynamic monitoring system to ensure that the framework
can be managed and maintained more effectively. Moreover, not all performance
dimensions are employed equally for organizational strategic objectives, nor will every
organization have the same reactions to a defect. The future research may augment this
model by combining it with decision-making tools such as analytic hierarchy process
(AHP) (Huan et al. 2004; Li et al. 2005; Bhagwat and Sharma 2007a, b) or fuzzy ratios
(Chan and Qi 2003).
Finally, while implementing this framework, the top management should not create a
reward mechanism based only on the measured sigma values, as this may tamper with the
measurement process and have destructive effects on team structure. The organization
should create a culture that emphasizes cooperation among entities toward common goals
rather than individual entity’s success. Also, the top management has the duty to facilitate
all entities to achieve the objectives. If the six-sigma philosophy is not widely practiced in
the supply chain or by individual members, the management should provide more training
for the development of their own measurement system. Additionally, it would be helpful to
build a mechanism to motivate employees to achieve process performance targets and to

123
404 Software Qual J (2010) 18:387–406

promote innovative ideas for continual improvement throughout the whole organization.
The mechanism may include communication plans, linkage with human resource devel-
opment, employer’s commitment, resource provision, and so forth.

References

Avery, S. (2001). Linking supply chains saves Raytheon $400 million. Purchasing August 23.
Beamon, B. M. (1998). Supply chain design and analysis: Models and methods. International Journal of
Production Economics, 55(3), 281–294.
Beamon, B. M. (1999). Measuring supply chain performance. International Journal of Operations &
Production Management, 19(3), 275–292.
Bhagwat, R., & Sharma, M. K. (2007a). Performance measurement of supply chain management using the
analytical hierarchy process. Production Planning & Control, 18(8), 666–680.
Bhagwat, R., & Sharma, M. K. (2007b). Performance measurement of supply chain management: A bal-
anced scorecard approach. Computer & Industrial Engineering, 53(3), 43–62.
Champion, M. A., Medsker, G. J., & Higgs, A. C. (1993). Relations between work group characteristics and
effectiveness: Implications for designing effective workgroups. Personnel Psychology, 46(4), 823–847.
Chan, F. T. S., & Qi, H. J. (2003). An innovative performance measurement method for supply chain
management. Supply Chain Management: An International, 8(3), 209–223.
Chen, I. J., & Paulraj, A. (2004). Towards a theory of supply chain management: The constructs and
measurements. Journal of Operations Management, 22(2), 119–150.
Clegg, C. W., Wall, T. D., Pepper, K., Stride, C., Woods, D., Morrison, D., et al. (2002). An International
survey of the use and effectiveness of modern manufacturing practices. Human Factors & Ergonomics
in Manufacturing, 12(2), 171–191.
Crago, M. G. (2000). Patient safety, six sigma & ISO 9000 quality management. Quality Digest
(November).
Dasgupta, T. (2003). Using the six-sigma metric to measure and improve the performance of a supply chain.
Total Quality Management & Business Excellence, 14(3), 355–366.
DeCenzo, D. A., & Robbins, S. P. (1996). Human resource management. NY: Wiley.
Dixon, J. R., Nanni, A. J., & Vollmann, T. E. (1990). The new performance challenge: Measuring oper-
ations for world-class competition. Irwin/Apics Series in Production Management.
Expert Choice Inc. (2004). Expert choice 11.
Ferguson, B. R. (2000). Implementing supply chain management. Production and Inventory Management
Journal March, 22, 64–67.
Fontenot, G., Behara, R., & Gresham, A. (1994). Six sigma in customer satisfaction. Quality Progress
December, 27(12), 73–76.
Globerson, S. (1985). Issues in developing a performance criteria system for an organization. International
Journal of Production Research, 23(4), 639–646.
Green, K. W., Whitten, D., & Inman, R. A. (2007). The impact of timely information on organizational
performance in a supply chain. Production Planning & Control, 18(4), 274–282.
Gunasekaran, A., Patel, C., & McGaughey, R. E. (2004). A framework for supply chain performance
measurement. International Journal of Production Economics, 87(3), 333–347.
Gunasekaran, A., Patel, C., & Tirtiroglu, E. (2001). Performance measures and metrics in a supply chain
environment. International Journal of Operations & Production Management, 21(1–2), 71–87.
Hakanson, B. (1999). SCOR can help analyze your supply-chain operation. Logistics Quarterly, 5(2).
Hara, M. (2002). ‘Kaizen Forever’: Japanese OEMs and suppliers see manufacturing efficiency as key to
future competitiveness. Automotive Industries, 182(10), 44–46.
Harry, M. J. (1997). The vision of six-sigma—A roadmap for breakthrough (5th ed., Vol. 1). Phoenix: Tri
Star Publishing.
Huan, S. H., Sheoran, S. K., & Wang, G. E. (2004). A review and analysis of supply chain operations
reference (SCOR) model. Supply Chain Management: An International Journal, 9(1), 23–29.
Janz, T., & Harel, G. (1993). Performance appraisal for TQM: A team approach. Total Quality Management,
4(3), 275–281.
Johnson, A. (2002). Six sigma in R&D. Research—Technology Management, 45(2), 12–16.
Kanji, G. K. (2001). Focus of excellence in Kanji’s business excellence model. Total Quality Management,
12(2), 259–272.

123
Software Qual J (2010) 18:387–406 405

Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard: Measures that drive performance. Harvard
Business Review, 70(1), 71–79.
Keegan, D. P., Eiler, R. G., & Jones, C. R. (1989). Are your performance measure obsolete? Management
Accounting, 70(12), 45–50.
Kennerley, M., & Neely, A. (2002). A framework of the factors affecting the evolution of performance
measurement systems. International Journal of Operation & Production Management, 22(11), 1222–
1245.
Kennerley, M., & Neely, A. (2003). Measuring performance in a changing business environment. Inter-
national Journal of Operation & Production Management, 23(2), 213–219.
Lamber, D. M., & Cooper, M. C. (2000). Issues in supply chain management. Industrial Marketing Man-
agement, 29(1), 65–83.
Lamber, D. M., Cooper, M. C., & Pagh, J. D. (1998). Supply chain management: Implementation issue and
research opportunities. The International Journal of Logistics Management, 9(2), 1–20.
Lambert, D. M., & Pohlen, T. L. (2001). Supply chain metrics. The International Journal of Logistics
Management, 12(1), 1–19.
Li, S., Rao, S. S., Ragu-Nathan, T. S., & Ragu-Nathan, B. (2005). Development and validation of a
measurement instrument for studying supply chain practices. Journal of Operations Management,
23(6), 618–641.
Lin, L. C., & Li, T. S. (2008). A continual improvement framework with integration of CMMI and six-sigma
model for auto industry. Quality and Reliability Engineering International published online in advance
of print: Nov 12, 2008.
Lynch, R. L., & Cross, K. F. (1995). Measure up!: Yardsticks for continual improvement (2nd ed.). Oxford:
Blackwell publishing.
Minitab Inc. (2000). Minitab statistical software release 13.
Neely, A., Gregory, M., & Platts, K. (1995). Performance measurement systems design: A literature review
and research agenda. International Journal of Operations & Production Management, 15(4), 80–116.
Pande, P. S., Neuman, R. P., & Cavanagh, R. R. (2000). The six sigma way: How GE Motorola, and other
top companies are honing their performance. NY: McGraw Hill Professional (The McGraw-Hill
Companies).
Rose, K. H. (1995). A performance measurement model’’, Quality Progress February.
Saaty, T. L. (1980). Analytical hierarchy process: Planning, priority setting, resource allocation. NY:
McGraw-Hill Professional (The McGraw-Hill Companies).
SCC (Supply-Chain Council). (1999). Overview of the SCOR Model V2.0. http://www.supply-chain.org.
Sharma, M. K., & Bhagwat, R. (2007). An integrated BSC-AHP approach for supply chain management
evaluation. Measuring Business Excellence, 11(3), 57–68.
Sharma, M. K., Bhagwat, R., & Dangayach, G. S. (2008). Performance measurement of information systems
in small and medium sized enterprises: A strategic perspective. Production Planning & Control, 19(1),
12–24.
Shepherd, C., & Gunter, H. (2006). Measuring supply chain performance: Current research and future
directions. International Journal of Productivity and Performance Management, 55(3/4), 242–258.
Stephens, S. (2001). Supply chain operations reference model vision 5.0: A new tool to improve supply
chain efficiency and achieve best practice. Information Systems Frontiers, 3(4), 471–476.
Storey, R. (1989). Team building: A manual for leaders and trainers. London: British Association for
Commercial and Industrial Education, BACIE.
Thomas, J. (1999). Why your supply chain doesn’t work. Logistics Management and Distribution Report,
38(6), 42–44.
Waggoner, D. B., Neely, A. D., & Kennerley, M. P. (1999). The forces that shape organizational perfor-
mance measurement systems: An interdisciplinary review. International Journal of Production Eco-
nomics, 60–61, 53–60.
Wang, F. K., Du, T. C., & Li, E. Y. (2004). Applying six-sigma to supplier development. Total Quality
Management & Business Excellence, 15(9–10), 1217–1229.
Webster, M. (2002). Supply system structure, management and performance: A conceptual model. Inter-
national Journal of Management Reviews, 4(4), 353–369.
Wyper, B., & Harrison, A. (2000). Deployment of six sigma methodology in human resource: A case study.
Total Quality Management, 11(4–6), 720–727.
Zigon, J. (1994). Oil company learns to measure work-team performance. Personnel Journal, 73(11),
46–48.

123
406 Software Qual J (2010) 18:387–406

Author Biographies

Lie-Chien Lin is a professor of logistic management at National

Kaohsiung first University of Science and Technology in Taiwan. He
received his Ph.D. degree of industrial and system engineering from
Georgia Institute of Technology in USA. His research interests include
supply chain management, logistic planning, and knowledge-based
expert systems for warehousing, purchasing, and facility planning.

Tzu-Su Li received M.S. degree of industrial engineering from New

Mexico State University in USA and achieved Ph.D. degree of logistic
management at National Kaohsiung first University of Science and
Technology in Taiwan. He has been a consultant focused on tech-
nology and quality management field in auto industry for many years.
He got six-sigma expert certification from Raytheon Company.

123