Professional Documents
Culture Documents
Abstract. The authors report the findings of a survey of hos- astounding advances in recent years, “the packaging
pital managers on the utilization of various project selection and delivery of that treatment are often inefficient,
and evaluation methodologies. The focus of the analysis was
the empirical relationship between a portfolio of project evalu- ineffective and consumer unfriendly” while concur-
ation1 methods actually utilized for a given project and several rently being quite costly (Herzlinger 2006, p. 58).
measures of perceived project success. The analysis revealed It has been projected, for example, that spending
that cost–benefit analysis and top management support were
the two project evaluation methods used most often by the
on healthcare-related activities may account for ap-
hospital managers. The authors’ empirical assessment provides proximately 18% of the U.S. gross domestic product
evidence that top management support is associated with over- in the 2009 fiscal year. This number is expected to
all project success. continue to rise in coming years, increasing to 20%
Keywords: hospital, project management, project selection, in 2018 (Elmendorf 2009). Although our “nation’s
project success aging hospital infrastructure attempts to meet an
increasing demand for services, deciding how to
spend scarce dollars has never been more difficult”
Janet R. Buelow, PhD, is an associate professor of health services administration at the College of Health Professions at Armstrong
Atlantic State University. Kathryn M. Zuckweiler, PhD, is an assistant professor of management at the University of Nebraska at
Kearney. Kirsten M. Rosacker, PhD, CMA, is an assistant professor of accountancy at the University of Wisconsin–La Crosse.
10
HOSPITAL TOPICS: Research and Perspectives on Healthcare 11
Hospital administrators responsible for oversee- equally essential. This latter focus may be in recog-
ing projects within their organizations cannot and nition that projects involve group-oriented activ-
should not assume that effective project selection ities subject to the benefits and issues associated
and evaluation methods in other industries apply with group interactions and communication. Thus,
within the hospital industry. Even if believing this qualitative approaches often override quantitative
assertion to be true, such a hypothesis should be approaches, which may require more detailed data
empirically assessed. than is reasonably available.
This article has four primary purposes. First, we
used a survey to determine whether hospital man- Quantitative Project Evaluation Methods
agers utilize standard project methodologies in se- Several quantitative project evaluation method-
lecting among various project opportunities and ologies have been reported in the research litera-
controlling the projects selected for implementa- ture. These methods are directed at and focused
tion. Second, the survey documented the preferred on evaluating project investments using financial
project analysis methodologies utilized by a cross criteria. They include cost–benefit analysis (CBA),
section of project managers in evaluating and con- budgetary constraint, payback, net present value
trolling recently completed projects within health- (NPV), internal rate of return (IRR), net project
care environments. We expected an ordered ranking execution cost (NPEC), and net project operation
of project methodology usage to flow directly from value (NPOV).
the survey results. Third, we developed a description CBA involves determining the anticipated costs
of the rationales employed by hospitals in ranking and benefits of investing in a project. In general,
and ultimately selecting among multiple project op- for a project to be acceptable under this technique
portunities. Fourth, we present empirical findings its benefits should exceed its costs. When utilizing
surrounding the relationships that exist between the the budgetary constraint method, an organization
project evaluation methods chosen and perceived simply selects the number of projects that can be en-
project success. The success measures evaluated were tertained within its present budget—an extremely
those reported in prior empirical research regard- simplistic methodology. The payback approach to
ing project methodologies and measures of project project investment analysis measures the time it
success (Belout and Gauvreau 2004). This research takes to recover the initial investment and ranks the
adds to the literature on project management and projects based on this criterion (Boyle and Gutherie
ultimately its findings should be of assistance to 2006). Although “the payback method ignores both
hospital administrators as they select among various the time value of money and cash flows that oc-
projects and manage their selected projects through cur subsequent to payback” (Boyle and Gutherie
to successful implementation. 2006, 1), the literature reports widespread use re-
vealing that 40%–90% of U.S. firms use payback as
PROJECT EVALUATION METHODS a project evaluation technique (Gilbert and Reichert
Lasting success for an organization is frequently 1995; Gitman and Vandenberg, 2000; Stanley and
determined by the effectiveness of its project eval- Block 1984; Visscher and Stansfield 1997).
uation process (Sun, Zhi-Ping, and Wang 2008). Advanced financial analysis techniques view a
Organizations formally appraise their project invest- project as a series of primarily, although not exclu-
ments to justify their investments, enable and sup- sively, initial cash outflows and future cash inflows
port a decision between alternative projects, control composed of two important components: (1) the
expenditures, improve the investment selection pro- recovery of the initial investment and (2) income
cess, and facilitate project management (Garrison, (Garrison, Noreen, and Brewer 2010). The NPV
Noreen, and Brewer 2010; Ghasemzadeh, Archer, method, also referred to as the discounted cash flow
and Lyogun 1999; Olson 2004; Sun, Zhi-Ping, and method, employs financial evaluation formulas to
Wang 2008). compute the values for all cash flows—inflows and
Two broad categories of project appraisal tech- outflows—that are expected to accompany a poten-
niques—quantitative (financial) and qualitative— tial investment based on an organization’s expected
commonly used in evaluating and selecting among minimum rate of return at a stipulated common
investment opportunities have been identified point in time. Effectively, all cash inflows and out-
(Ghasemzadeh, Archer, and Lyogun 1999). Al- flows are discounted for presentation at a fixed point
though financial evaluation is important and in time. The use of NPV in projects has been widely
relatively straightforward, qualitative metrics are addressed and reported in the literature (Flaig 2005;
12 Vol. 88, no. 1 2010
Mullen and Donnelly 2006). The NPV method 2000, p. 237). Instinctive behavior is widely dis-
evaluates the present value of estimated future cash cussed in biological settings and its presence is so
flows on the assumption that future benefits are sub- well known and defined that it simply cannot be
ject to a discount factor that takes into account the ignored when making most decisions.
cost of money and project risk. “While the NPV A legal requirement, demands of an external cus-
measure is theoretically valid, it lacks the intuitive tomer or supplier, and competitive pressures may
appeal of the IRR in communicating the value of mandate an organization to undertake an invest-
investment alternatives” to an organizations man- ment (Fitzgerald 1998). Past experience provides
agement (Kennedy and Plath 1994, p. 38). The an initial decision framework for projects that fol-
IRR method, although based in NPV techniques, low. It is often from this starting point that project
returns a metric that may be more understandable techniques are selected for use in future manage-
for some project managers. The outcome of an IRR ment decision making. Many projects may not be
analysis is the rate of return that equates project cash permitted to proceed without the presence of top
inflows and outflows to a NPV of zero. An organi- management support (Jacobsen 2008). The project
zation’s management determines the minimum ac- management literature indicates that top manage-
ceptable IRR for a project—generally referred to as ment support is mission critical and that the degree
the hurdle rate. Those projects that exceed the min- of its presence significantly influences project suc-
imum hurdle rate remain in consideration, whereas cess (Jha and Iyer 2008; Young and Jordan 2008).
those that do not are eliminated from the evaluation Additionally, only those projects that have a high
process. Kennedy and Plath (1994) explained that probability of reaching completion are to be under-
IRR is easily understood by organizational decision taken.
makers.
NPEC and NPOV differ from the other two METHOD
quantitative methodologies in that they overtly In the present study, we used a survey method-
consider opportunity costs (Yu, Flett, and Bowers ology, incorporating a standardized survey instru-
2005). NPEC is the excess of all project costs over ment that has been widely used in project man-
the project benefits incurred during the execution agement research (Rosacker and Olson, 2008)
phase of a project. NPOV considers all the benefits a and, to our knowledge, has never been used in
client obtains from the project during operation and healthcare organizations. Survey participants were
reduces these by any related operational costs. More identified using snowball sampling with the au-
advanced financial analysis variants have been pro- thors identifying hospital chief operating officers
posed in the literature (Patton and Shechet 2007; (COOs) in 11 Midwestern and Southern states (Al-
Liesiö, Mild, and Salo 2007); however, although abama, Colorado, Connecticut, Georgia, Kansas,
theoretically appealing, these advanced techniques Michigan, Minnesota, Missouri, South Carolina,
have not been reported as widely utilized. South Dakota, and Wisconsin). Hospital COOs
were contacted via e-mail and asked to either com-
Qualitative Project Evaluation Methods plete the online survey themselves or forward the
Qualitative attributes represent business or e-mail containing the survey link to individuals in
project characteristics, issues, or concerns that can their organizations who were in charge of a hospital
be identified but are not easily quantified (Klein and project. Hospital managers in all 11 states partici-
Beck 1987). Qualitative characteristics include but pated in the survey.
are not limited to such things as instinct, gut feel,
mandatory requirements, past experience, proba- RESULTS
bility of completion, and top management support.
Essentially, they represent anything that impacts the Managers and Hospital Project Sample
desirability or feasibility of a project from the broad A total of 199 hospital managers answered the
perspective of any of the project stakeholders. The survey link, providing 186 usable responses. Nearly
project management literature provides evidence half (48%) of the managers identified themselves
that managers occasionally base investment deci- as senior hospital administrators, 29% identified
sions on instinct (Powell, 1992). Instinct is a subtle themselves as department managers, and 23%
form of reasoning that takes into account how the identified themselves as various clinicians (e.g.,
world really appears rather than merely “relying on physicians, nurses). Table 1 presents the breakdown
what the spreadsheet says” (Bannister and Remenyi of responding hospital managers.
HOSPITAL TOPICS: Research and Perspectives on Healthcare 13
Hospital Characteristics
Hospital managers were asked to identify the Table 2 shows characteristics of the hospitals the
types of projects that provided their framework for surveyed managers represented. In terms of urban
completing the survey. Managers, although select- and rural facilities, the hospital sample was similar to
ing only one project as their frame of reference, the distribution of American Hospital Association-
were permitted to characterize projects under more registered U.S. hospitals, including 59% urban and
than one project type. For example, one respondent 41% rural facilities (American Hospital Association
identified her chosen project as involving infor- 2009). The majority of the sampled hospitals were
mation technology, process improvement, quality of not-for-profit status (81%), with 7% proprietary
improvement, and efficiency/financial. Another re- and 12% public hospitals. In total, 26% of the hos-
spondent identified his chosen project as involving pitals were critical-access hospitals. The bed size of
facilities construction or renovation and new service the hospitals differed considerably from U.S. hos-
development. This multiple-classification method- pitals. The present sample had more large hospitals
ology permitted managers to more fully describe with 400 or more beds (28% vs. 8%) as well as
projects without having to choose between equally more small hospitals (6–24 beds), at 21% versus
applicable project types. Through the use of this 10% nationally.
project-type classification, the managers were able
to provide data for 290 project types. The project Project Type and Evaluation Methods
types were categorized as facilities construction or Hospital managers were asked to select the
renovation (24%), process improvement (17%), in- methods used to justify their project. Over-
formation technology (15%), efficiency/financial all, managers reported using an average of 3.88
improvement (14%), quality improvement (13%), evaluation methods per project (SD = 2.17).
new service development (10%), human resource CBA was used most frequently (76%). Other
development (5%), and other (2.07%). A summary project evaluation methods commonly used in-
of the project types is provided in Table 1. cluded top management support (64%), bud-
Project budgets ranged from less than $100,000 getary constraints (40%), mandatory requirements
to greater than $10,000,000. Half of the budgets fell (39%), IRR (33%), payback method (28%), NPEC
within the lowest two categories (less than $100,001 (29%), NPOV (29%), probability of completion
and $100,000–$500,000). In all, 27 projects had (29%), and instinct (28%). For about 9% of
budgets greater than $10,000,000. projects, managers identified another evaluation
14 Vol. 88, no. 1 2010
method not listed on the survey instrument and Success metric 1—Time. Four project evaluation
entered their method. No additional quantitative methods (IRR, NPV, payback, and top manage-
methods were recorded by the managers. Additional ment support) were statistically significantly (p =
methods used included alignment with mission and .05) associated with the time dimension of project
strategic vision, community need, equipment ob- success. IRR, NPV, and payback represented quan-
solescence, increased productivity, market compe- titative project evaluation measures. Top manage-
tition, physician support, rate of patient satisfac- ment support was a qualitative project evaluation
tion, space issues, and quality improvement. For methodology. No other project evaluation method
only three projects did managers not know the type displayed a statistically significant relationship with
of evaluation used. Table 3 presents a summary of the time success metric.
project justification methods used by type of project.
Process improvement 50 37 25 17 22 16 18 14 11 11 11 4 6 0
Information technology 44 41 31 18 15 11 12 14 15 10 9 8 7 1
Efficiency/financial improvement 41 31 23 23 18 14 14 12 11 9 12 5 4 0
Quality improvement 37 28 25 12 21 13 13 13 12 11 10 4 3 0
New service 28 23 21 11 9 15 8 7 10 11 8 6 1 0
HR development 15 13 10 5 5 5 5 5 4 3 4 0 0 0
Other 6 2 2 1 1 1 1 1 0 2 2 0 2 0
Percentage of projects using 76% 64% 40% 39% 33% 29% 29% 29% 28% 28% 14% 9% 1%
method
Note. CBA = cost–benefit analysis; IRR = internal rate of return; NPOV = net project operation value; NPEC = net project execution cost; NPV = net present value.
15
16 Vol. 88, no. 1 2010
Success Metric
Success Metric Success Metric Success Metric Success Metric 5—Overall
Method 1—Time 2—Cost 3—Use 4—Impact success
Note. CBA = cost–benefit analysis; IRR = internal rate of return; NPOV = net product operation value; NPEC = net
project operation cost; NPV = net present value; df = 185 for every F .
∗
p = .10. ∗∗ p = .05. ∗∗∗ p = .001.
Austin, C., K. Homberg, and J. Shmerling. 2000. Managing in- Jha, K. N., and K. C. Iyer. 2008. Critical factors affecting
formation resources: A study of ten healthcare organizations. quality performance in construction projects. Total Quality
Journal of Healthcare Management 45:229–239. Management 17:1155–1170.
Bannister, F., and D. Remenyi. 2000. Acts of faith: instinct, Kennedy, W. F., and D. A. Plath. 1994. A return-based alter-
value, and IT investment decisions. Journal of Information native to IRR evaluations. Healthcare Financial Management
Technology 15:231–241. 48(3):38–45.
Belout, A., and C. Gauvreau. 2004. Factors influencing project Klein, G., and P. Beck. 1987. A decision aid for selecting among
success: The impact of human resource management. Inter- information system alternatives. MIS Quarterly 11:177–185.
national Journal of Project Management 22:1–11. Knight, G. 2005, August. Healthcare IT needs project man-
Boyle, G., and G. Gutherie. 2006. Payback without apology. agement. Healthcare IT News.
Accounting and Finance 46:1–10. Langabeer, J. 2008. Hospital turnaround strategies. Hospital
Butcher, L. 2007. So many projects, so little money. Healthcare Topics 86(2):3–10.
Strategic Management 25 (3):7. Liesiö, J., P. Mild, and A. Salo. 2007. Preference programming
Carden, L., and T. Egan. 2008. Does our literature support for robust portfolio modeling and project selection. European
sectors newer to project management? The search for qual- Journal of Operational Research 181:1488–1505.
ity publications relevant to nontraditional industries. Project Mullen, R., and J. T. Donnelly. 2006. Keeping it real—Building
Management Journal 39 (3):6–27. an ROI model for an ambulatory EMR initiative that the
Elmendorf, D. 2009. Options for controlling the cost and physician practices espouse. Journal of Healthcare Information
increasing the efficiency of health care. Washington, DC: Management 20(1):42–52.
Congressional Budget Office. http://www.cbo.gov/ftpdocs/ Olson, D. L. 2004. Information systems project management. 2nd
100xx/doc10016/Testimony.1.1.shtml (accessed April 30, ed. New York: McGraw-Hill/Irwin.
2009). Patton, N., and A. Shechet. 2007. Earned value management:
Flaig, J. J. 2005. Improving project selection using expected Are expectations too high? CrossTalk 20 (1):10–15.
net present value analysis. 17:535–38. Powell, P. 1992. Information technology evaluation: Is it differ-
Fitzgerald, G. 1998. Evaluating information systems projects: ent? Journal of the Operational Research Society 43(1):29–42.
A multidimensional approach. Journal of Information Tech- Rosacker, K., and D. Olson, 2008. An empirical assessment of
nology 13(1):15–27. IT project selection and evaluation methods in state govern-
Garrison, R. H., E. W. Noreen, and P. C. Brewer. in press. Man- ment. Project Management Journal 39(1):49–59.
agerial accounting. 13th ed. New York: McGraw-Hill/Irwin. Sa Couto, J. 2008. Project management can help reduce cost
Ghasemzadeh, F., N. Archer, and P. Lyogun.1999. A zero-one and improve quality in health care services. Journal of Eval-
model for project portfolio selection and scheduling. Journal uation in Clinical Practice 14:48–52.
of Operational Research 50:745–755. Stanley, M. T., and S. B. Block. 1984. A survey of multinational
Gilbert, E., and A. Reichert 1995. The practice of financial capital budgeting. Financial Review 19(1):36–54.
management among large U.S. corporations. Financial Prac- Sun, Y., J. Zhi-Ping, and J. Wang. 2008. A group decision
tice and Education 5(1):16–23. support approach to evaluate experts for project selection.
Gitman, L. J., and P. A. Vandenberg. 2000. Cost of capital IEEE Transactions on Engineering Management 55:158–170.
techniques used by major US firms: 1997 vs. 1980, Financial Visscher, S. L., and T. C. Stansfield. 1997. Illustrating capital
Practice and Education 10:53–68. budgeting complexities with JIT justification data. Financial
Glaser, J. 2004. Back to basics managing IT projects. Healthcare Practice and Education 7:29–34.
Financial Management 58(7):34–38. Wood, J. R. 2008. June. Effective project management. Health
Hair, J., R. Anderson, R. Tatham, and W. Black. 1998. Mul- Facilities Management Magazine p. 49.
tivariate data analysis, 5th ed. Upper Saddle River, NJ: Young, R., and E. Jordan. 2008. Top management support:
Prentice-Hall. Mantra or necessity? International Journal of Project Manage-
Herzlinger, R. E. 2006. Why innovation in healthcare is so ment 26:713–725.
hard. Harvard Business Review 84 (5):58–66. Yu, A. G., P. D. Flett, and J. A. Bowers. 2005. Developing a
Jacobsen, J. 2008. Avoiding the mistakes of the past. Journal value-centered proposal for assessing project success. Inter-
for Quality and Participation 31(2):4–8. national Journal of Project Management 23:428–436.
Copyright of Hospital Topics is the property of Taylor & Francis Ltd. and its content may not be copied or
emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.
Copyright of Hospital Topics is the property of Taylor & Francis Ltd. and its content may not be copied or
emailed to multiple sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.