Performance Dashboard for a Pharmaceutical Project

Benchmarking Program
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

Sung-Joon Suk1; Bon-Gang Hwang2; Jiukun Dai, M.ASCE3; Carlos H. Caldas, M.ASCE4;
and Stephen P. Mulva, M.ASCE5

Abstract: Performance measurement is essential to controlling and improving capital projects. Increasingly, industry sectors demand an
industry-specific performance measurement system because their unique processes can result in significant differences in performance out-
comes. This paper presents the development of a performance dashboard for a pharmaceutical project benchmarking program. The proposed
approach adopts a relative comparison method and uses weighted key performance indicators (KPIs) to compare project performance. The
dashboard generates an overall performance score both at the project level and at the company level, as shown in tabular and graphical formats.
The dashboard provides flexibility for comparing the performance of projects even when the project types or KPIs are different. Although the
study focuses on a performance dashboard for pharmaceutical industry projects, the dashboard development process described in this paper can
be applied to other industry projects. DOI: 10.1061/(ASCE)CO.1943-7862.0000503. © 2012 American Society of Civil Engineers.
CE Database subject headings: Benchmark; Construction management; Performance characteristics.
Author keywords: Benchmarking; Construction management; Performance dashboard; Performance evaluation; Performance
measurement; Pharmaceutical project.

Introduction Nitithamyong and Skibniewski 2006; Abdel-Razek et al. 2007; Yu

Performance measurement is “the process of quantifying the effi-
ciency and effectiveness of actions” (Neely et al. 1995). In other
words, it is the process of evaluating how successfully a project
is completed or an organization is managed. In a very competitive
global environment, performance measurement becomes critical to
improving and managing performance in the construction industry.
Over the past two decades, a number of performance measurement
frameworks have been introduced to measure and improve different
aspects of performance [Kumaraswamy and Thorpe 1996; Fong
et al. 2001; Palaneeswaran and Kumaraswamy 2000; Sommerville
and Robertson 2000; Key Performance Indicators (KPI) Working
Group 2000; Kagioglou et al. 2001; Li et al. 2001; Cox et al. 2003;
Mohamed 2003; Beatham et al. 2004; Chan et al. 2004; Love et al.
2004; Ramirez et al. 2004; Bassioni et al. 2005; Lee et al. 2005;
1 effectively” (Eckerson 2006). Eckerson stated that a performance
Ph.D. Candidate, Univ. of Texas at Austin, Dept. of Civil, Architectural
and Environmental Engineering, 1 University Station C1752, Austin,
TX 78712 (corresponding author). E-mail: sungjoon.suk@gmail.com
2
Assistant Professor, National Univ. of Singapore, Dept. of Building,
SDE2-03-05, 4 Architecture Drive, Singapore 117566. E-mail: bdghbg@
nus.edu.sg
3 marized graphical view that allows users to monitor aggregated key
Research Engineer, Construction Industry Institute, Benchmarking and
Metrics, Univ. of Texas at Austin, 3925 West Braker Lane (R4500), Austin,
TX 78759. E-mail: jiukun.dai@cii.utexas.edu
4
Associate Professor, Univ. of Texas at Austin, Dept. of Civil,
Architectural and Environmental Engineering, 1 University Station
C1752, Austin, TX 78712. E-mail: caldas@mail.utexas.edu
5
Associate Director, Construction Industry Institute, Benchmarking and
Metrics, Univ. of Texas at Austin, 3925 West Braker Lane (R4500), Austin,
TX 78759. E-mail: smulva@cii.utexas.edu
Note. This manuscript was submitted on April 20, 2011; approved on
October 24, 2011; published online on October 26, 2011. Discussion period
open until December 1, 2012; separate discussions must be submitted
for individual papers. This paper is part of the Journal of Construction
Engineering and Management, Vol. 138, No. 7, July 1, 2012. ©ASCE,
ISSN 0733-9364/2012/7-864–876/$25.00.
et al. 2007; Luu et al. 2008; Toor and Ogunlana 2010; Yeung et al.
2009]. These frameworks can be classified into three categories on
the basis of their area of focus: (1) benchmarking programs;
(2) project performance measurements; and (3) company perfor-
mance measurements. Benchmarking programs focus on compar-
ing and improving the performance of either projects or companies
on the basis of project performance measurements and/or company
performance measurements. Project performance measurements
evaluate the success level of individual projects, whereas company
performance measurements assess the overall business success
level of companies.
Although many performance measurement frameworks have
been developed, few studies have focused on a performance dash-
board, which is defined as “a multilayered application built on a
business intelligence and data integration infrastructure that enables
organizations to measure, monitor, and manage performance more
dashboard needs applications and layers. Applications include
monitoring, to convey information at a glance; analyzing, to let
users analyze exceptional conditions; and managing, to improve
alignment coordination and collaboration. Layers consist of a sum-
performance indicators (KPIs), a multidimensional view to facili-
tate exploration of information from various perspectives, and a de-
tailed reporting view that provides information about performance
in detail. To attain this functionality, a performance dashboard re-
quires a detailed implementation and scoring technique that goes
beyond current research on performance measurement (Bassioni
et al. 2005) and a method to represent performance outputs in a
quantifiable and aggregated fashion.
The purpose of this study is to build a comprehensive
performance dashboard that is practical and relevant to an
industry-tailored benchmarking program. The paper focuses on a
performance dashboard for a pharmaceutical project benchmarking
program, which was built as part of a benchmarking study by the

864 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 Table 1. Specific Benchmarking Programs
Researchers Area
Palaneeswaran and Kumaraswamy (2000) Contractor selection
Sommerville and Robertson (2000) Total quality management
(TQM)
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
Fong et al. (2001) Value management
Li et al. (2001) Partnering
Mohamed (2003) Organizational safety
culture
Love et al. (2004) Information technology
Ramirez et al. (2004) Management practices
Abdel-Razek et al. (2007) Labor productivity
Construction Industry Institute (CII). Data on 198 pharmaceutical
capital projects are used to develop the performance dashboard; the
data were submitted by 12 companies: 11 U.S. companies and one
European company. The development process of the performance
dashboard is presented with a discussion of the underlying rationale
for each process and its results. Furthermore, results of a survey of
16 pharmaceutical industry experts validate the performance dash-
board framework.
Project Performance Measurement Frameworks
Benchmarking Programs
McGeorge et al. (2002) defined benchmarking as “a process of
continuous improvement based on the comparison of an organiza-
tion’s processes or products with those identified as best practice.”
Benchmarking is recognized as a commonly used tool to identify
successful performance of projects and construction companies
(El-Mashaleh et al. 2007). The two most widely known bench-
marking programs are the Benchmarking and Metrics (BM&M)
Program at CII and the Construction Best Practice Programme
(CBPP), which is jointly supported by the Department of the Envi-
ronment, Transport and the Regions and the Construction Industry
Board in the United Kingdom.
Based at the University of Texas at Austin, CII is a consortium
of leading owners, engineering and construction contractors, and
suppliers whose mission is to measurably improve the delivery of
capital facilities (CII 2010b). The CII organized the BM&M Pro-
gram in 1996 and established its first set of performance metrics:
project cost growth, project schedule growth, project budget fac-
tor, project schedule factor, change cost factor, and practice use
(Lee et al. 2005). As the program matured, CII established stan-
dard metric definitions for engineering and construction produc-
tivity from selected disciplines, including concrete, structural
steel, electrical, piping, instrumentation, and equipment (Park
et al. 2005; Kim 2007). Using these metrics, CII provides its par-
ticipants with its web-based benchmarking reports detailing
project performance norms, including values on individual met-
rics with mean, median, quartile cutoffs, and data distribution
(CII 2010b).
The CBPP launched its KPIs for performance measurement
in the United Kingdom in the late 1990s. Ten KPIs were developed
for measuring both project performance and company performance,
comparing a project or company with the other projects or
JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 865
J. Constr. Eng. Manage., 2012, 138(7): 864-876
Result
Process model: A conceptual benchmarking model
Metrics/process model: 18 metrics considered key to the business and a
six-step process toward TQM
Process model: Management commitment, facilitator’s skills,
brainstorming, group effectiveness, customer satisfaction
Process model: Eight-stage process of a cooperative benchmarking
approach
Measurement system: Balanced scorecard across four perspectives,
including management, operational, customer, and learning
Metrics: Benefits, costs, and risks
Measurement system: Application of a management evaluation system
Metrics: Disruption index, performance ratio, and project
management index
companies in the construction industry. Seven of them are related
to project performance: construction cost, construction time, cost
predictability, time predictability, defects, client satisfaction with
product, and client satisfaction for service. The other three KPIs
are related to company performance: safety, profitability, and pro-
ductivity. The 10 KPIs have been used by many construction
companies in the United Kingdom (Beatham et al. 2004).
In addition to CII and CBPP, research efforts have been made to
establish an area-specific benchmarking program. Such efforts gen-
erated more relevant metrics, measurement systems, and/or process
models to benchmark a particular area. Table 1 shows recent efforts
related to benchmarking programs that narrowed down to a
specific area.
Although these benchmarking programs have developed many
useful metrics to measure and evaluate performance, they focus on
individual metrics rather than on aggregated indicators. As a result,
it is difficult to obtain insight into the overall performance of the
company (El-Mashaleh et al. 2007) or project. In addition, a com-
petitive benchmarking program requires an industry-tailored
project hierarchy because processes and facilities vary by industry
(Hwang et al. 2008), and such distinct processes and facilities often
result in significant differences in project performance. Moreover,
each industry needs particular metrics that reflect industry-specific
characteristics. For example, pharmaceutical projects need to meet
regulatory requirements for validation and qualification of facili-
ties. The followings are examples of pharmaceutical industry–
specific metrics to measure and evaluate performance related to
validation and qualification: (commissioning cost + qualification
cost)/total installed cost (TIC); [installation qualification (IQ) thru
operation qualification (OQ) duration]/(# IQ + OQ protocols), and
(process space square foot + process-related space square foot)/
gross square foot (GSF).
Project Performance Measurements
Project performance measurements evaluate the success level of
individual projects. These measurements can have different pur-
poses: business performance and project delivery performance.
Business performance evaluates the success and effectiveness of
projects over the project lifetime. Business performance is usually
measured by the owner because it needs information on the profit-
ability of products or services in addition to the performance of the
project delivery itself. Project delivery performance, on the other
hand, assesses the efficiency of completing facilities constructed
for production or occupancy. Project delivery performance meas-
urement is of major interest to the construction industry and has
 Table 2. Project Performance Indicators
Performance indicators
Client Business Project-team Communi-
Studies Cost Time Quality Safety Environ-ment satisfaction performance satisfaction cation Others
Kumaraswamy and O O O O O O O Technology transfer
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

Thorpe (1996)
Audit Office of New O O O O O O O
South Wales (1999)
Key Performance O O O O O Change orders
Indicators (KPI)
Working Group (2000)
Cox et al. (2003) O O O O Productivity
Chan et al. (2004) O O O O O O O Functionality of
building
Nitithamyong and O O O O Strategic
Skibniewski (2006) performance
risk improvement
Toor and Ogunlana O O O O O O Minimized conflicts
(2010)
Yeung et al. (2009) O O O O O O Innovation and
improvement
This paper O O O O Design/space
efficiency
Note: O = Yes.

been widely adopted for project performance evaluation. Both
business performance and project delivery performance use
performance metrics to evaluate outcomes objectively.
Traditionally, the performance of construction projects was
measured by three performance indicators: cost, time, and quality
(Ward et al. 1991). However, recent studies employed additional
performance indicators to evaluate construction project execution
from a more balanced perspective. Table 2 shows the performance
indicators used by several previous studies. In addition to cost,
time, and quality, safety is included in most of the studies. Addi-
tionally, some studies suggest the use of indicators such as envi-
ronmental friendliness, client satisfaction, business performance,
project-team satisfaction, and communication. Some other indica-
tors found in the literature include technology transfer, change or-
ders, productivity, building functionality, strategic performance,
risk improvements, minimized conflicts and innovation, and
improvement.
The definition of performance indicators were the primary focus
and contribution of the previous studies shown in Table 2, which
primarily dealt with the criteria required for assessing the perfor-
mance of construction projects. However, a generic framework is
required to compare the performance of various construction proj-
ects regardless of their industry or facility type. Specifically, the
framework needs a scoring algorithm for the selected performance
metrics and a comparison method to evaluate the performance of
projects and/or companies.
Company Performance Measurements
For a long time before the 1990s, company performance measure-
ments relied on financial and accounting measurements: return on
investment, sales per employees, cash flow, profit, and more. How-
ever, financial-focused performance measurements were criticized
for their insufficiency in addressing overall business performance
(Eccles 1991; Kaplan and Norton 1992; Parker 2000). To overcome
this deficiency, Kaplan and Norton (1992) introduced the balanced
scorecard (BSC), which added three nonfinancial perspectives:
customers, innovation and improvement, and internal processes,
in addition to the financial perspective. Consequently, the BSC pro-
vided a balanced picture by including operational performance in
addition to economic performance (Amaratunga et al. 2001).
Many studies have been conducted to measure company perfor-
mance by adopting and modifying the BSC. Kagioglou et al.
(2001) presented a conceptual framework that integrated the per-
spectives of project and supplier with the BSC. Bassioni et al.
(2005) established a conceptual framework to measure the business
performance of construction companies by the BSC, business
excellence models, and empirical feedback from expert interviews.
Yu et al. (2007) developed a framework to calculate a company
performance score by combining weighted metrics, which were
categorized by the four BSC perspectives. The weights for metrics
were obtained from a survey of construction companies. Luu et al.
(2008) incorporated a strengths–weaknesses–opportunities–threats
(SWOT) matrix into the BSC so that a typical construction com-
pany can devise their short- and long-term strategies. Stewart and
Mohamed (2001) adapted the BSC concept to develop a framework
to evaluate (IT) performance. The framework measured (IT) per-
formance at three different decision-making tiers: project, business
unit, and enterprise.
These studies contributed to improving company performance
measurement on the basis of the BSC. They developed perfor-
mance metrics and frameworks appropriate for the construction in-
dustry from various perspectives. Nonetheless, more studies need
to be conducted to investigate the relationship between company
performance and project performance (Bassioni et al. 2004). Fur-
thermore, little has been done to measure company performance
from the perspective of aggregated project performance. The most
practical need is a detailed implementation method and scoring al-
gorithm that can be used for company performance measurement
(Bassioni et al. 2005).

866 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 Development of a Performance Dashboard
Background
To develop a more specific benchmarking system for the pharma-
ceutical industry, CII and pharmaceutical industry experts worked
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
metric is within the 25% with the worst performance. More details
on the first stage of the pharmaceutical benchmarking system can
be found in Hwang et al. (2008).
However, the team perceived that the pharmaceutical bench-
marking system should include more aggregated information to as-
sess the project performance from an overall perspective. Although

together as a team to launch the pharmaceutical facility benchmark-
ing program in 2004. The research team developed the hierarchical
structure for categorizing pharmaceutical projects, as shown in
Table 3. They identified three major project types: bulk manufac-
turing, secondary manufacturing, and laboratory, each of which
has subcategories, some of which are divided into more subcate-
gories. The rationale for this categorization is that the distinct
processes and characteristics of pharmaceutical facilities will show
different project performance results. Therefore, benchmarking
requires organization of an appropriate hierarchical structure for
pharmaceutical projects. The team also defined 80 pharmaceutical
industry–specific metrics, which include cost, schedule, dimension,
and quality metrics (Hwang et al. 2008).
The CII constructed a tailored pharmaceutical questionnaire by
modifying and adding questions to the existing questionnaire,
which is generic to capital construction projects in all industries.
The pharmaceutical questionnaire was programmed and released
online to allow the team to submit project data. The submitted data
are analyzed on the basis of the individual metrics categorized by
the project hierarchy. The metrics of a project are compared with
those of other projects by using a statistical method. The compari-
son results are reported by showing a relative performance location
of each metric in a quartile-based distribution. For example, if a
metric for a project belongs to the first quartile, then the perfor-
mance of the metric falls within the top performing 25%. In con-
trast, if a metric is in the fourth quartile, then the performance of the
metrics of a project are individually compared with those of other
projects, the overall performance of projects cannot be evaluated
only with the metrics and the project hierarchy. To objectively as-
sess the overall performance of projects, an approach is required to
reasonably aggregate the performance results of a number of
metrics. The performance dashboard developed in this research
suggests a rational solution to evaluate the overall performance
of projects. The following sections discuss the components and
algorithm of the performance dashboard in detail.
Components of the Performance Dashboard
The proposed performance dashboard framework includes a
tabular-format dashboard and two graphic-format dashboards.
The tabular-format dashboard summarizes the results of the perfor-
mance of projects by numbers, whereas the graphic-format dash-
boards show the deviations of the performance of projects by
distributions. This section describes the components of the
tabular-format dashboard. The outputs of all dashboards are
discussed in the “Performance Dashboard Outputs” section, where
their layout and design are shown in a table and figures.
The tabular-format dashboard consists of aggregated quartile
scores for project-level performance, company-level performance,
safety performance, industrial project definition rate index (PDRI),
project type, project comparison level, number of projects com-
pared, and project priority. Each of these elements is explained
in the following paragraphs.

Table 3. Hierarchical Structure of Pharmaceutical Projects

Level I Level II Level III Level IV
Pharmaceutical bulk Biological (41) Direct derived (2) —
manufacturing (65) Fermentation (14) —
Cell culture (20) Attachment dependent (7)
Stirred tank (13)
Pilot plant (5) —
Chemical (24) Manufacturing (21) —
Pilot plant (3) —
Pharmaceutical secondary Fill finish (50) Parenteral (20) Syringe (2)
manufacturing (68) Delivery device (2)
Vial (15)
Pilot plant (1)
Non-parenteral (30) Inhalants (4)
Solid dosage (22)
Cream/ointment (2)
Pilot plant (2)
Medical devices (3) — —
Nutritional (4) — —
Packaging (3) — —
Pharmaceutical warehouse (8) — —
Pharmaceutical Research (34) Biological (23) —
laboratory (65) Chemical (11) —
Quality control/quality assurance (12) — —
Vivarium (9) — —
Process development (10) — —
Note: Number of projects shown in parentheses.

JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 867

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 The performance of each project is evaluated on the basis of
an overall project performance that is a composite score of four
performance categories: cost, schedule, quality, and dimension.
In addition to the first three most commonly used performance cat-
egories, the performance dashboard includes the dimension perfor-
mance category that measures the design/space efficiency of
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
facilities. The first two performance categories, cost performance
and schedule performance, have subcategories: absolute perfor-
mance and relative performance, each of which has different
key metrics. In contrast, dimension and quality performance cat-
egories consist of key metrics without subcategories.
Apart from the overall project performance, safety performance
is independently assessed. The team decided to exclude it from the
overall project performance because many projects had no accident
record or did not have data, and thus safety performance may not
have enough power of discrimination to calculate the overall
project performance. However, the team still wanted to examine
the results of safety performance because of the importance of
safety itself. Consequently, safety performance was included in
a separate column of the performance dashboard and not as a part
of the overall project performance.
Company-level performance shows the quartiles and rankings
for the overall project performance and the four performance cat-
egories by averaging the performance scores of all the projects in a
company portfolio. Quartiles and ranks are determined by compar-
ing the average performance scores of a company with those of
other companies, which enables each company to know their over-
all relative level of performance. To protect confidentiality, compa-
nies can view only their own performance and relative position in
quartiles; other companies’ data are blinded. To help understand
quartiles and ranks, the distributions of the average performance
scores are examined and graphically represented.
The PDRI score measures how well a project is defined and is
included in the performance dashboard to support examination of
the relationship between project performance and the adequacy of
the definition. Project type indicates the original project type in the
hierarchical structure; comparison level represents the project type
where a project is compared with other projects; numbers of proj-
ects compared shows how many projects are compared in a com-
parison level; and project priority describes the project driver for a
project. Three different project priorities are used: cost-driven,
schedule-driven, and balanced/others.
In addition to this performance dashboard, dashboards are also
provided for the three Level I groups in the hierarchical structure
shown in Table 3 to allow the team to separately examine the per-
formance of bulk manufacturing, secondary manufacturing, and
laboratory.
Performance Categories, Key Performance Metrics,
and Weights
Bulk manufacturing, secondary manufacturing, and laboratory use
the same four performance categories: cost, schedule, dimension,
and quality. First, the cost performance category consists of two
subcategories: the absolute cost performance category and the rel-
ative cost performance category. The absolute cost performance
represents ratios of actual costs to other costs or dimensions,
and the relative cost performance measures cost variation. Second,
the schedule performance category also has two subcategories: the
absolute schedule performance category and the relative schedule
performance category. Absolute schedule performance assesses ac-
tivity durations to the number of protocols for qualification, dimen-
sion, or cost. On the other hand, relative schedule performance
evaluates schedule variation. However, both of the performance cat-
egories for dimension and quality have only key metrics without
868 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012
J. Constr. Eng. Manage., 2012, 138(7): 864-876
subcategories. Third, the dimension performance category evalu-
ates how efficiently a process facility is designed. For bulk and
secondary manufacturing, the proportion of process space is as-
sessed. On the other hand, for laboratory, the length of the benchtop
and the number of persons to be admitted are evaluated. Lastly, the
quality performance category includes planning execution of quali-
fication and validation (PEQV) and Pharmaceutical Project Defi-
nition Index (PPDI). Terms related to PEQV and PPDI are
defined in the CII pharmaceutical owner large questionnaire (CII
2010a) as follows: qualification is the action of providing equip-
ment or ancillary systems that are properly installed, work cor-
rectly, and actually lead to the expected results; validation is a
documented program that provides a high degree of assurance that
a specific process, method, or system will consistently produce a
result meeting predetermined acceptance criteria; and PPDI indi-
cates how well user requirements, design qualification planning,
and validation master plans were defined before the total project
budget authorization. Both PEQV and PPDI measure the level
of activities to meet the requirements of the Current Good Manu-
facturing Practice (CGMP) regulations. On the basis of the CGMP
regulations, the U.S. Food and Drug Administration (FDA) in-
spects the facilities used in manufacturing, processing, and packing
of a drug before it can be approved (FDA 2010). The activities for
qualification and validation are essential to receive an approval
from the FDA and to deliver pharmaceutical projects.
Although bulk manufacturing, secondary manufacturing, and
laboratory use the same performance categories, the key metrics
included in each performance category are not the same. Bulk
manufacturing and secondary manufacturing have the same key
metrics (Fig. 1), but laboratory employs different key metrics
(Fig. 2) to measure aspects of laboratory that are distinct from those
in manufacturing facilities. All of the key metrics for the four per-
formance categories were determined by team discussion.
Weights were defined for the four performance categories, KPIs,
and project priorities to reflect their relative importance. The Delphi
method, an iterative process leading to a group consensus, was used
to determine weights. The iterative process involves experts, a
questionnaire, and a facilitator. Once the experts complete the ques-
tionnaire, the facilitator reports a summary of the answers. Consid-
ering the summary as a group opinion, the experts can revise the
answers. This process can be concluded after a predetermined num-
ber of rounds, when the summary does not change with more
rounds, or when the group reaches consensus on the summary.
The iterative process of the Delphi method enables a group to con-
verge to consensus. In this research, the industry experts of the team
answered the questionnaire for the weights of the KPIs. The CII
facilitated the Delphi method and provided the team a summary
with the averages of the weights. The process stopped after the sec-
ond round because the team achieved consensus on the averages.
Bulk manufacturing and secondary manufacturing use the same
set of weights (Fig. 1), but laboratory uses a different set of weights
(Fig. 2) because its characteristics are different from manufacturing
projects. Each set of weights has three different groups of weights
for the four performance categories depending on project priorities:
cost, schedule, or balanced. The team agreed that the project pri-
orities lead to significant differences in the project performance of
pharmaceutical capital projects. According to the project priority,
the weights of the four performance categories are applied differ-
ently for the weighted percentiles. For example, when cost is the
project priority for a bulk project, the weights of the four perfor-
mance categories are, respectively, 55, 22, 10, and 13. In contrast,
when schedule is the project priority for a bulk project, the weights
of the four performance categories are 22, 55, 10, and 13. The sub-
categories and indicators under the four performance categories
 Cost Performance

Absolute Cost Performance 66 Absolute Cost Performance Relative Cost Performance

Cost Growth: (Actual Cost - Planned Cost) /
Relative Cost Performance 34 TIC / Process Equipment Cost 22 50
Planned Cost
Delta Cost Growth: (Actual Cost - Planned
Facility Construct. Cost / GSF 18 40
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

Cost) / Planned Cost

Change Cost Factor:
Project Priority TIC / GSF 33 10
Change Cost / Actual Cost
Cost Balanced Schedule Soft Cost / Hard Cost 10
Cost (Design Cost + Construct. mgmt.
55 44 22 9
Performance Cost) / TIC
Schedule (Commissioning + Qualification
22 33 55 8
Performance Cost) / TIC
Dimension
10 10 10 Schedule Performance
Performance
Quality
13 13 13 Absolete Schedule Performance 48 Absolute Schedule Performance Relative Schedule Performance
Performance
(IQ thru OQ Duration) / Schedule Growth: (Actual Duration -
Relative Schedule Performance 52 33 70
((# IQ
Q + OQ
Q Protocols)) Planned Duration)) / Planned Duration
Delta Schedule Growth: (Actual Duration -
(Design ~ OQ Duration) / GSF 26 20
Planned Duration) / Planned Duration
Change Schedule Factor:
(Design ~ OQ Duration) / TIC 41 10
Change Duration / Actual Duration
Dimension Performance

(Process Space SF + Process

100
Related Space SF) / GSF

Quality Performance
Planning & Execution of
50
Qualification & Validation
Pharma Project Definition Index 50

Fig. 1. Key performance metrics and weights for bulk and secondary manufacturing

also have weights. All of the weights were determined by the
Delphi method described previously.
Algorithm of the Performance Dashboard
On the basis of the hierarchical structure, components, performance
categories, KPIs, and weights described previously, the following
steps delineate the calculation process of generating the perfor-
mance dashboard.
Step 1: Calculation of the Percentiles of the Key
Performance Metrics
Step 1 calculates the percentiles of the KPIs. Percentiles indicate
relative scores or positions represented by values ranging from
0–100. For example, a metric may have six values that are 1, 2,
3, 4, 5, and 6. If a lower value means better performance, the per-
centiles of the six values are 0, 20, 40, 60, 80, and 100, respectively.
Percentiles are used to represent relative performance scores for
each key metric because percentiles enable the performance dash-
board to be combined with approximately 20 key metrics that mea-
sure different aspects of project performance and thus have various
units and distribution ranges.
In accordance with the project hierarchy, the KPI values of a
project are compared with those of other projects. During this com-
parison, percentiles are calculated and assigned to the KPIs as long
as the number of projects is sufficient. The team established a rule
that the calculation of percentiles for each KPI requires that a
comparison level have more than seven projects submitted by more
than three separate companies. The objective of this rule is to
ensure reasonable comparison and confidentiality. Comparison
with a small number of projects may not have adequate power
to differentiate the performance level of projects, and may reveal
the company that is the source of a metric’s value. The team dis-
cussed that having more than seven projects in a comparison level
ensures at least two projects for each quartile, and the interval of
percentiles is not too wide to calculate their relative performance. In
addition, having more than three companies in a comparison level
prevents a company from easily recognizing the other companies. If
the rule is not satisfied in a comparison level, then percentiles are
calculated and reported at the next higher comparison level in the
project hierarchy that meets the rule.
Step 2: Calculation of the Weighted Percentiles
The percentiles of each key metric are multiplied by their weights,
thereby producing weighted percentiles. Then the sum of the
weighted percentiles in a performance category becomes a perfor-
mance score for the performance category. Each performance cat-
egory also has different weights, which multiply the performance
scores of the four performance categories. Finally, the overall per-
formance scores for each project are calculated by summing up the
weighted percentiles of the four performance categories. Table 4
shows an example of the calculation process of weighted percen-
tiles for the relative cost performance category. In the O1001
project, the three KPIs (cost growth, delta cost growth, and change
cost factor) are in the 80, 40, and 70 percentiles, respectively. The
weights of the three KPIs are 50, 40, and 10, as shown in Table 4.
Because the weighted percentile is the product of percentile and
weight, the weighted percentile of cost growth is 40 (80 × 0:5).

JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 869

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 Cost Performance

Absolute Cost Performance 62 Absolute Cost Performance Relative Cost Performance

Cost Growth: (Actual Cost - Planned Cost) /
Relative Cost Performance 38 Facility Construction Cost / GSF 20 50
Planned Cost
Delta Cost Growth: (Actual Cost - Planned 40
TIC / GSF 31
Cost) / Planned Cost
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

Project Priority Soft Cost / Hard Cost 7 Change Cost Factor: 10

Change Cost / Actual Cost
(Design Cost + Construction
Cost Balanced Schedule 10
Management Cost) / TIC
Cost 59 49 28 (Commissioning + Qualification
6
Performance Cost) / TIC
Schedule 14 24 45 TIC / Lab Population 15
Performance
Dimension 16 16 16 TIC / Total Building Population 11
Performance
Quality
11 11 11
Performance
Schedule Performance

Absolete Schedule Performance 54 Absolute Schedule Performance Relative Schedule Performance

Relative Schedule Performance
Dimension Performance
Benchtop Linear Foot / Lab
Population
GSF / # Total Building Population 51
46 (Design ~ OQ Duration) / GSF 45 Schedule Growth: (Actual Duration - 60
Planned Duration) / Planned Duration
(Design ~ OQ Duration) / TIC 55 Delta Schedule Growth: (Actual Duration - 30
Planned Duration) / Planned Duration
Change Schedule Factor: 10
Change Duration / Actual Duration
49

Quality Performance
Planning & Execution of
36
Qualification & Validation
Pharma Project Definition Index 64

Fig. 2. Key performance metrics and weights for laboratory

Table 4. Example of Calculating the Weighted Percentile

Cost growth Delta cost growth Change cost factor
Weighted
½ðactual cost
planned costÞ∕planned cost ½jðactual cost
planned costÞ∕planned costj ðchange cost∕actual costÞ
percentile of
Weighted Weighted Weighted relative cost
Project Percentile Weight percentile Percentile Weight percentile Percentile Weight percentile performance
O1001 80 50 40.0 40 40 16.0 70 10 7 63.0
(80 × 50 ÷ 100) (40 × 40 ÷ 100) (70 × 10 ÷ 100) (40 þ 16 þ 7)
O1002 60 50 33.3 30 40 13.3 Not 10 Not 46.6
(60 × 50 ÷ 90) (30 × 40 ÷ 90) available available (33:3 þ 13:3)

In the same way, the weighted percentiles of delta cost growth and
change cost factor are 16 and 7. Subsequently, the weighted per-
centile of the relative cost performance category is 63, which is the
sum of 40, 16, and 7.
Weights are adjusted when values are missing in the key metrics.
Although the ground rule is that the weighted percentiles of each
performance category are calculated by every KPI, projects some-
times did not have data for all KPIs. The O1002 project shows an
example of how the weights are adjusted. The O1002 project has
cost growth and delta cost growth but not change cost factor. In this
case, the weighted percentile of the relative cost performance cat-
egory is calculated by only the two KPIs (cost growth and delta cost
growth). Therefore, the weighted percentiles of the two KPIs are
divided by the sum of the weights of the two KPIs. That is, the
weighted percentile of cost growth is 33.3 [ð60 × 0:5Þ∕0:9]. In
the same way, the weighted percentile of delta cost growth is
13.3 [ð30 × 0:4Þ∕0:9]. Finally, the weighted percentile of the rel-
ative cost performance is 46.6, which is the sum of 33.3 and 13.3.
Step 3: Search of the Comparison Level of Projects
Step 3 determines at which level a project is compared in the hier-
archical structure (Table 3). Ideally, projects need to be compared
against their original project category. For instance, comparing an
attachment-dependent project with other attachment-dependent
projects is the most desirable method. However, not all projects
can be compared within their original project category because
of an insufficient number of projects. As described, the team estab-
lished a rule that a comparison level requires eight or more projects
to be submitted by at least four companies. Although 198 projects
have been submitted over 5 years, not every project category in the

870 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 hierarchical structure has a large enough number of projects to meet Table 5. Example of Adjusting Weighted Percentile (Comparison
the rule at the most specific level. If a project category does not Level: Bulk Manufacturing/Biological/Cell Culture/Stirred Tank)
meet the rule, then this category cannot be a comparison level. Ac- Weighted Adjusted
cordingly, projects in such a category cannot be compared in their Project percentile Rank percentile
original project category. Nonetheless, they need to be compared
O2001 74 2 90
somehow so that the overall performance can reflect all of the
O2002 54 7 40
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

projects in a company’s portfolio.

Comparing projects with an upper level project category may be
the next best way to compare projects whose comparison level does
not satisfy the rule. For example, currently the project category of
attachment dependent has seven projects, as shown in Table 3. In
this case, attachment-dependent projects are compared at the
project category of cell culture, which meets the rule and the next
upper level of attachment dependent in the hierarchy. Likewise, if
any project category does not have enough projects, projects are
compared at the next upper category that becomes the comparison
level. Consequently, all projects can be reasonably compared,
which enables the performance dashboard to include all of the
projects in a company’s portfolio.
Step 4: Adjustment of the Weighted Percentiles of the
Performance Categories and the Overall Performance
Step 4 adjusts the weighted percentiles calculated at Step 2 so that
the weighted percentiles for the performance categories and for the
overall performance can range from 0–100 percentiles. Because the
weighted percentiles from Step 2 are the sum of the products of
several metric percentiles and their weights, the performance cat-
egories and the overall performance do not range from 0–100.
Accordingly, the ranges vary depending on comparison levels.
For example, the weighted percentiles in the bulk manufactur-
ing/biological/cell culture level vary from 35–85, whereas those
in the bulk manufacturing/chemical/manufacturing level vary from
25–90. Comparison with these different ranges may lead to a faulty
assessment of project performance. At the bulk manufacturing/
biological/cell culture level, the project whose weighted percentile
is 85 is the best; at the bulk manufacturing/chemical/manufacturing
level, the project whose weighted percentile is 90 is the best. If the
two best projects were compared with only their weighted percen-
tiles, the project with 90 would appear better. However, from the
relative performance perspective, this comparison is unfair to the
project with 85, because this project is the best at the bulk manu-
facturing/chemical/manufacturing level. Similarly, it is not correct
to say that the worst project with 35 is better than a project with 25
because the weighted percentiles come from different comparison
levels.
To avoid such unfair comparisons, the weighted percentiles are
adjusted by Eq. (1). Eq. (1) makes the weighted percentiles range
from 0–100 as relative scores, which enable projects to be com-
pared with each other even though they have different comparison
levels.
Adjusted percentile ¼ 100
100 × ðR
1Þ∕ðN
1Þ ð1Þ
where R = rank of the weighted percentile; and N = total number of
percentiles.
Table 5 shows an example of how the weighted percentiles are
adjusted. The comparison level is bulk manufacturing/biological/
cell culture/stirred tank, with 11 projects. Regarding project
O2007, its weighted percentile for overall project performance is
71, and it ranks third of 11 projects. On the basis of the preceding
formula, the adjusted percentile of the overall performance of
project O2007 is 80, against which the project can be compared
with other projects in different comparison levels. In this way, each
project can be evaluated by using the adjusted percentiles ranging
from 0–100.
O2003 57 6 50
O2004 48 8 30
O2005 64 5 60
O2006 44 9 20
O2007 71 3 80
O2008 16 11 0
O2009 25 10 10
O2010 85 1 100
O2011 67 4 70
Step 5: Comparison of the Performance of Companies
Step 5 compares the performance of companies on the basis of the
averages of the adjusted percentiles of the projects in each company
portfolio. The overall performance of a company is calculated by
averaging the overall performance scores of each project in that
company’s portfolio. The comparison results are described in
the next section.
Performance Dashboard Outputs
Performance dashboards for each company include several dash-
board outputs. Table 6 shows the tabular performance dashboard
of an example company X with 12 projects. The projects have their
adjusted percentiles for the four performance categories and for the
overall project performance. For instance, the adjusted percentiles
for the four performance categories of the O1001 project are 69, 15,
62, and 82, which results in a 69 adjusted percentile and second
quartile for the overall project performance. In addition to these
relative performance scores, the performance dashboard outputs
provide detailed criteria to document how the performance of a
project is evaluated and compared, such as project type, compari-
son level, number of projects compared, and project priority.
The overall performance of each company is evaluated by the
average adjusted percentiles of the projects in their portfolio. In this
example (Table 6), the overall performance on the 12 projects of
company X is 70 on average, which is compared with that of other
companies. Similarly, the averages of the four performance catego-
ries are compared with those of other companies. The comparison
results are reported by rank and quartile in the last two rows. The
overall performance, 70, ranks second of 11 companies, which is
shown in the overall performance column. Consequently, the over-
all performance of company X falls in the first quartile
(2∕11 ≈ 0:18, which is less than 0.25, the first quartile limit). In
the same fashion, the rankings of the four performance categories
are 3/11, 8/11, 2/11, and 7/9, and their quartiles are second, third,
first, and fourth.
In addition to the tabular dashboard (Table 6), the performance
evaluation results of projects in a company portfolio can be exam-
ined by graphical dashboards in a summarized way. The two
graphical dashboards (Figs. 3 and 4) encapsulate the tabular per-
formance dashboard. The objective of the graphical dashboards
is to present a summarized view of the performance results. As
the number of projects increases, the tabular dashboard becomes
difficult to read and interpret. The first graphical dashboard (Fig. 3)
is a doughnut chart that summarizes the distribution of the averages

JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 871

J. Constr. Eng. Manage., 2012, 138(7): 864-876

 Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.

Table 6. Tabular Performance Dashboard for Company X

Overall project Safety Number
performance performance Project of projects
definition in
Cost Schedule Dimension Quality Adjusted rate index Project Comparison compared Project
Project performance performance performance performance percentile Quartile TRIR DART score type level data set priority
O1001 69 15 62 82 69 2 100 100 138 Bulk/bio/ Bulk/bio/ 14 Cost
fermentation fermentation
O1002 77 77 85 89 100 1 100 100 242 Bulk/bio/cell/ Bulk/bio/cell 14 Schedule
attachment
O1003 79 36 67 17 61 2 100 100 — Bulk/bio/pilot Bulk/bio 34 Cost
slant
O1004 91 76 55 Not available 94 1 12 100 — Bulk/Bio/Pilot Bulk/Bio 34 Cost
Plant
O1005 88 8 76 76 67 2 41 N/A — Secondary/ Secondary 52 Cost
warehouse
O1006 100 25 92 18 67 2 Not Not — Secondary/fill Secondary/fill 13 Schedule
available available finish/parental/ finish/
vial parental/vial
O1007 56 50 94 80 75 1 100 100 76 Secondary/fill Secondary/fill 17 Cost
finish/parental/ finish/
delivery parental/
delivery
O1008 78 80 78 11 71 2 Not 34 331 Secondary/ Secondary 52 Balanced
available nutritional
O1009 20 100 25 38 50 2 63 100 — Laboratory/ Laboratory/ 11 Cost
quality control quality
and quality control and

J. Constr. Eng. Manage., 2012, 138(7): 864-876

assurance quality

872 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012

assurance
O1010 15 Not available 100 13 40 3 0 0 — Laboratory/ Laboratory/ 21 Cost
research/bio research/bio
O1011 95 10 61 0 80 1 13 100 414 Laboratory/ Laboratory/ 21 Cost
research/bio research/bio
O1012 85 Not available Not available 50 70 2 100 Not — Laboratory/ Laboratory/ 21 Schedule
available research/bio research/bio
Average 71 48 72 43 70 63 82
Company-level Rank/number of 3∕11 8∕11 2∕11 7∕9 2∕11 3∕9 3∕9
performance companies
Quartile 2 3 1 4 1 2 2
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
Fig. 3. Graphical performance dashboard I for company X
for the overall performance and the four performance categories.
This chart provides information on how the overall performance
of a company is derived from the performance results of the indi-
vidual projects. In Fig. 3, the number in the center of the top circle
shows that the overall performance of company X is in the first
quartile. The percentages in the ring of the top circle indicate that
33% are in the first quartile, 59% are in the second quartile, 8% are
in the third quartile, and 0% is in the fourth quartile. Because com-
pany X has 12 projects in its portfolio, three projects are in the first
quartile, seven projects are in the second quartile, one project is in
the third quartile, and no projects are in the fourth quartile. This
distribution of the overall performance averages of 12 projects re-
sults in the first quartile for the performance of company X. The
circles at the bottom illustrate the distribution of the performance
averages of the four performance categories. Because the doughnut
chart shows the performance results by distribution, it provides a
summary of the performance of projects regardless of the number
of projects.
To examine the performance of projects in detail, the team de-
veloped similar performance dashboards for bulk manufacturing,
secondary manufacturing, and laboratory. These performance dash-
boards further investigate the performance of projects as classified
by the three primary project types in Level I of the hierarchical
structure of pharmaceutical projects (Table 3). The performance
dashboards for the three primary project types have the same tabu-
lar and graphical format described in the previous paragraphs. They
assist in conveying company portfolio performance at a more
detailed level.
The second graphical dashboard (Fig. 4) provides the distribu-
tions of the average adjusted percentiles for the overall performance
and the four performance categories. This dashboard enables com-
panies to investigate a relative level of their performance. The dis-
tributions help them understand how their performance is different
from other companies in terms of the average adjusted percentile of
the projects in their portfolio. This first bar graph from left indicates
the overall performance that summarizes the next four bar graphs
for the performances of cost, schedule, dimension, and quality. For
JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 873
J. Constr. Eng. Manage., 2012, 138(7): 864-876
example, the average adjusted percentiles for company X are de-
picted by large diamonds in Fig. 4. A significant difference is
shown in the average adjusted percentiles of the overall perfor-
mance between the first ranked company and company X. In con-
trast, little difference is shown in the average adjusted percentiles of
the cost performance between the first or second ranked companies
and company X. The distributions imply that company X falls be-
hind the first ranked company from the overall performance per-
spective, although it ranked second, and that company X is
similar to the first and second companies from a cost performance
perspective. Consequently, company X can have a better under-
standing regarding their performance.
Validation
The developed performance dashboards were reported to each com-
pany through the reporting system. To validate the overall frame-
work of the performance dashboards, pharmaceutical industry
experts were surveyed on whether the performance dashboards
are applicable to their needs as a decision support system. The sur-
vey solicited feedback on the rationale underlying the comparison
algorithm and on the structure of the developed dashboard frame-
work from the user’s standpoint. The respondents were 16 pharma-
ceutical industry experts from 10 pharmaceutical companies who
are participating in the CII pharmaceutical BM&M Program. They
were asked to rate different aspects of the framework on 5-point
Likert scales (strongly agree, agree, neutral, disagree, and strongly
disagree). Fifteen of 16 answered agree or strongly agree to the
question “The comparison algorithm consisting of the relative im-
portance of the indicators, adjusted composite index scores, and
level of data references is appropriate.” The same number of the
respondents also answered agree or strongly agree to the question
“The structure of the performance dashboard is well organized.”
Almost 94% of the respondents agreed that the rationale of the per-
formance dashboard framework and the structure of the perfor-
mance dashboard are appropriate for providing appropriate
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
Fig. 4. Graphical performance dashboard II for company X
levels of information on the performance of pharmaceutical proj-
ects. The performance dashboard has been implemented as a regu-
lar component of the CII benchmarking program, which further
validates its applicability.
Discussion
The purpose of this study was to develop a performance dashboard
for a pharmaceutical project benchmarking system that could rea-
sonably assess the overall performance of different types and char-
acteristics of pharmaceutical projects. The performance dashboard
used weights that were assigned to the project priorities, the per-
formance categories, and the KPIs.
Depending on whether the project priority was cost, schedule, or
balanced, the different sets of weights were applied for the cost
performance category and the schedule performance category in
evaluating the overall project performance. When the project prior-
ity was cost, the weights showed that cost performance was con-
sidered more important than schedule as much as 2.5 times
(cost∕ schedule ¼ 55∕22) for manufacturing projects (Fig. 1)
and 4.2 times (cost ∕ schedule ¼ 59∕14) for laboratory projects
(Fig. 2). On the other hand, when the project priority was schedule,
the weights indicated that schedule performance was more
important than cost performance as much as 2.5 times
(schedule∕ cost ¼ 55∕22) for manufacturing projects and 1.6 times
(schedule∕ cost ¼ 45∕28) for laboratory projects. When the project
priority was balanced, cost performance was still considered more
important than schedule performance as much as 1.3 times
(cost ∕ schedule ¼ 44∕33) for manufacturing projects and 2.0 times
(cost ∕ schedule ¼ 49∕24) for laboratory projects. Meanwhile, the
weights implied that cost performance was more emphasized on
laboratory projects than on manufacturing projects.
For cost performance that consists of absolute cost performance
and relative cost performance, the former was regarded as approx-
imately twice as important as relative cost performance for both
manufacturing projects and laboratory projects. However, for
874 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012
J. Constr. Eng. Manage., 2012, 138(7): 864-876
schedule performance, absolute schedule performance and relative
schedule performance had almost equal importance in their
weights.
The weights for the KPIs were also determined to relatively
measure each cost, schedule, dimension, and quality performances.
The weights for the KPIs differed between manufacturing projects
and laboratory projects because they had different KPIs. For abso-
lute cost performance, manufacturing projects have TIC/process
equipment cost, whereas laboratory projects had TIC/lab popula-
tion and TIC/total building population. Because of the FDA’s in-
spection criteria, manufacturing projects included (IQ ∼ OQ
duration)/(# IQ + OQ protocols) in evaluating absolute schedule
performance, which was not necessary for laboratory projects.
For dimension performance, manufacturing projects used (process
space square foot + process-related space square foot)/GSF,
whereas laboratory projects involved linear foot benchtop/lab pop-
ulation and GSF/total building population.
Regarding the comparison method used for the dashboard, a rel-
ative comparison approach was adopted so that different types of
the KPIs could be aggregated, and distinct types of projects could
be compared. As a result, the performance of every project could be
evaluated with the adjusted weighted percentile ranging from
0–100, which indicated a relative performance score. In addition,
the overall performance of each company portfolio could be as-
sessed, showing how successfully a company completed their proj-
ects compared with their competitors. Because projects in each
company’s portfolio were equally weighted in calculating the over-
all performance of a company regardless of the project cost, the
overall performance evaluated a company’s effectiveness in com-
pleting the projects in their portfolio. If the overall performance was
weighted by the cost of projects, then the dashboard could evaluate
a company’s efficiency in completing the projects in their portfolio.
The authors recognized that the results from the effectiveness
evaluation were different from those from the efficiency evaluation
by a sensitivity analysis.
On the basis of the relative project or company performance
scores, the performance dashboards enabled companies to facilitate
 communication regarding where the strong or weak points of their
projects are and to understand how they performed on their projects
in delivering the pharmaceutical capital projects compared with
their competitors. Furthermore, the two graphical dashboards effec-
tively summarize the performance of all projects in each company’s
portfolio regardless of the number of projects in the portfolio. Be-
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
cause the performance dashboard has been developed as part of the
CII benchmarking Program, high-level managers with pharmaceut-
ical companies participating in the CII pharmaceutical BM&M Pro-
gram have used the dashboards to compare the performance of their
projects with other projects in the CII database. However, the per-
formance dashboard is open to any company who joins the CII
pharmaceutical BM&M Program. Currently, the dashboards show
the comparison results on the basis of 198 projects from 12 com-
panies. This comparison results will be updated at the end of every
year with newly submitted projects.
Although this study focuses on pharmaceutical industry proj-
ects, it may be possible to apply the process used for developing
a performance dashboard to other industries. The process involved
is as follows: (1) develop a hierarchical structure for categorizing
projects in a certain industry on the basis of project characteristics,
such as project type and project priority; (2) identify KPIs appro-
priate to the type of projects; (3) group the identified KPIs into
relevant performance categories, such as cost, schedule, quality, di-
mension, and others; (4) define weights for the KPIs and the per-
formance categories; (5) calculate the percentiles of the KPIs and
the performance categories; (6) calculate the weighted percentiles
of the KPIs and the performance categories; (7) adjust the weighted
percentiles for each project; (8) determine the relative performance
level of each project by the adjusted weighted percentiles; (9) cal-
culate the average adjusted weighted percentiles for both the per-
formance categories and the overall project performance for each
company; and (10) determine the relative performance level of each
company by the average adjusted percentiles of the projects in their
portfolio.
Conclusions
This study described the development of a performance dashboard
for a pharmaceutical project benchmarking system. The developed
dashboard integrated four performance categories: cost, schedule,
dimension, and quality. Each performance category consisted of
KPIs. The weights of the project priorities, the performance catego-
ries, and the KPIs were assigned through the Delphi method. The
dashboard was formulated with a relative comparison approach. As
a result, the performance of different types of projects and the per-
formance of a company’s portfolio could be compared and evalu-
ated. Although the dashboard was developed for pharmaceutical
industry projects, the development process for the dashboard could
be applicable to other industries.
The results of this paper open several areas for future research.
The proposed dashboard can facilitate the study of the relationships
between the results and the causes of project performance. To im-
prove project performance, the causes or factors resulting in good
or bad project performance need to be identified. Possible ap-
proaches to investigate the causes and factors include the following
analyses: the relationships between project performance and mana-
gerial factors, such as delivery methods, contract types, project
organization, and others, and the relationships between project per-
formance and use of practices such as front-end planning, align-
ment, change management, partnering, and others If identified,
the causes and factors could be used as leading indicators. The
strategies or performance goals for each company can be different;
JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012 / 875
J. Constr. Eng. Manage., 2012, 138(7): 864-876
thus, a flexible performance dashboard framework needs to be
studied so that a company can dynamically assign preferred
weights to the dashboard.
References
Abdel-Razek, R. H., Elshakour, M. H., and Abdel-Hamid, M. (2007).
“Labor productivity: Benchmarking and variability in egyptian
projects.” Int. J. Proj. Manage., 25(2), 189–197.
Amaratunga, D., Baldry, D., and Sarshar, M. (2001). “Process improvement
through performance measurement: The balanced scorecard
methodology.” Work Study, 50(5), 179–189.
Audit Office of New South Wales. (1999). Performance audit report: Key
performance indicators, Sydney, Australia.
Bassioni, H. A., Price, A. D. F., and Hassan, T. M. (2004). “Performance
measurement in construction.” J. Manage. Eng., 20(2), 42–50.
Bassioni, H. A., Price, A. D. F., and Hassan, T. M. (2005). “Building a
conceptual framework for measuring business performance in construc-
tion: An empirical evaluation.” Constr. Manage. Econ., 23(5), 495–507.
Beatham, S., Anumba, C., Thorpe, T., and Hedges, I. (2004). “KPIs: A
critical appraisal of their use in construction.” Benchmarking, 11(1),
93–117.
Chan, A. P. C., Scott, D., and Chan, A. P. L. (2004). “Factors affecting the
success of a construction project.” J. Constr. Eng. Manage., 130(1),
153–155.
Construction Industry Institute (CII). (2010a). “Owner pharmaceutical
questionnaire version 10.2.” 〈https://www.construction-institute.org/
scriptcontent/BM_GUEST/downloads/Owner_Pharma_v10_2.pdf〉 (Jan.
28, 2010).
Construction Industry Institute (CII). (2010b). “About CII.” 〈https://www
.construction-institute.org/scriptcontent/aboutcii.cfm?section=aboutcii〉
(Jan. 28, 2010).
Cox, R. F., Issa, R. R. A., and Ahrens, D. (2003). “Management’s percep-
tion of key performance indicators for construction.” J. Constr. Eng.
Manage., 129(2), 142–151.
Eccles, R. G. (1991). “The performance measurement manifesto.” Harv.
Bus. Rev., 69(1), 131–137.
Eckerson, W. W. (2006). Performance dashboards: Measuring, monitor-
ing, and managing your business, Wiley, New York.
El-Mashaleh, M., Minchin, R. E., Jr., and O’Brien, W. J. (2007).
“Management of construction firm performance using benchmarking.”
J. Manage. Eng., 23(1), 10–17.
Fong, P. S., Shen, Q., and Cheng, E. W. L. (2001). “A framework for
benchmarking the value management process.” Benchmarking, 8(4),
306–316.
Hwang, B. G., Thomas, S. R., Degezelle, D., and Caldas, C. H. (2008).
“Development of a benchmarking framework for pharmaceutical capital
projects.” Constr. Manage. Econ., 26(2), 177–195.
Kagioglou, M., Cooper, R., and Aouad, G. (2001). “Performance manage-
ment in construction: A conceptual framework.” Constr. Manage.
Econ., 19(1), 85–95.
Kaplan, R. S., and Norton, D. P. (1992). “The balanced scorecard measures
that drive performance.” Harv. Bus. Rev., 70(1), 71–79.
Key Performance Indicators (KPI) Working Group. (2000). KPI report for
the Minister for Construction, Dept. of the Environment, Transport and
the Regions, London.
Kim, I.-H. (2007). “Development and implementation of an engineering
productivity measurement system (EPMS) for benchmarking.” Ph.D.
dissertation, Univ. of Texas at Austin, Austin, TX.
Kumaraswamy, M. M., and Thorpe, A. (1996). “Systematizing construction
project evaluations.” J. Manage. Eng., 12(1), 34–39.
Lee, S. H., Thomas, S. R., and Tucker, R. L. (2005). “Web-based bench-
marking system for the construction industry.” J. Constr. Eng. Manage.,
131(7), 790–798.
Li, H., Cheng, E. W. L., Love, P. E. D., and Irani, Z. (2001). “Co-operative
benchmarking: a tool for partnering excellence in construction.” Int. J.
Proj. Manage., 19(3), 171–179.
Love, P. E. D., Irani, Z., and Edwards, D. J. (2004). “Industry-centric
benchmarking of information technology benefits, costs and risks for
 small-to-medium sized enterprises in construction.” Autom. Constr.,
13(4), 507–524.
Luu, T. V., Kim, S. Y., Cao, H. L., and Park, Y. M. (2008). “Performance
measurement of construction firms in developing countries.” Constr.
Manage. Econ., 26(4), 373–386.
McGeorge, W. D., Palmer, A., and London, K. (2002). Construction man-
agement: New directions, Wiley-Blackwell, Hoboken, NJ.
Downloaded from ascelibrary.org by "National Institute of Technical Teachers Training & Research, Chennai" on 10/03/23. Copyright ASCE. For personal use only; all rights reserved.
Mohamed, S. (2003). “Scorecard approach to benchmarking organizational
safety culture in construction.” J. Constr. Eng. Manage., 129(1), 80–88.
Neely, A., Gregory, M., and Platts, K. (1995). “Performance measurement
system design.” Int. J. Oper. Prod. Manage., 15(4), 80–116.
Nitithamyong, P., and Skibniewski, M. J. (2006). “Success/failure factors
and performance measures of web-based construction project manage-
ment systems: Professionals’ viewpoint.” J. Constr. Eng. Manage.,
132(1), 80–87.
Palaneeswaran, M. M., and Kumaraswamy, M. M. (2000). “Benchmarking
contractor selection practices in public-sector construction: A proposed
model.” Eng. Constr. Archit. Manage., 7(3), 285–299.
Park, H. S., Thomas, S. R., and Tucker, R. L. (2005). “Benchmarking of
construction productivity.” J. Constr. Eng. Manage., 131(7), 772–778.
Parker, C. (2000). “Performance measurement.” Work Study, 49(2), 63–66.
Ramirez, R. R., Alarcon, L. F. C., and Knights, P. (2004). “Benchmarking
system for evaluating management practices in the construction
industry.” J. Manage. Eng., 20(3), 110–117.
876 / JOURNAL OF CONSTRUCTION ENGINEERING AND MANAGEMENT © ASCE / JULY 2012
J. Constr. Eng. Manage., 2012, 138(7): 864-876
Sommerville, J., and Robertson, H. W. (2000). “A scorecard approach to
benchmarking for total quality construction.” Int. J. Qual. Reliab. Man-
age., 17(4/5), 453–466.
Stewart, R. A., and Mohamed, S. (2001). “Utilizing the balanced scorecard
for IT/IS performance evaluation in construction.” Constr. Innovation-
London, 1(3), 147–164.
Toor, S. R., and Ogunlana, S. O. (2010). “Beyond the ‘iron triangle’:
Stakeholder perception of key performance indicators (KPIs) for
large-scale public sector development projects.” Int. J. Proj. Manage.,
28(3), 228–236.
U.S. Food and Drug Administration (FDA). (2010). “Drug applications and
current good manufacturing practice (CGMP) regulations.” 〈http://www
.fda.gov/Drugs/DevelopmentApprovalProcess/Manufacturing/ucm090016
.htm〉 (Jan. 28, 2010).
Ward, S. C., Curtis, B., and Chapman, C. B. (1991). “Objectives and
performance in construction projects.” Constr. Manage. Econ., 9(4),
343–353.
Yeung, J. F., Chan, A. P. C., and Chan, D. W. C. (2009). “Developing a
performance index for relationship-based construction projects in
Australia: Delphi study.” J. Manage. Eng., 25(2), 59–68.
Yu, I., Kim, K., Jung, Y., and Chin, S. (2007). “Comparable performance
measurement system for construction companies.” J. Manage. Eng.,
23(3), 131–139.