This action might not be possible to undo. Are you sure you want to continue?
Key Performance Indicators for Innovation Implementation: Perception vs. Actual Usage
School of Management, Queensland University of Technology, Australia Received 18 April 2009; Received in revised form 11 June, 2009; Accepted 7 September 2009
Abstract This paper aimed to explore the proportion associated with the perceived importance and the actual use of performance indicators from manufacturing and non manufacturing industries. The sample was 86 small and medium sized-organizations in Thailand. The perceived importance and the actual use of financial and non-financial indicators were found to be significantly related among manufacturing and non-manufacturing industries. Three key performance indicators (KPIs) (i.e. sales and sales growth, quality of products and/or services, and process time) were perceived the most importance among manufacturing managers (85.3%, 79.4% and 76.5% respectively). Three KPIs (i.e. customer satisfaction, quality of products and/or services; and process time) were perceived the most importance among nonmanufacturing managers (84.8%, 93.5%, and 84.8% respectively). Interestingly, the most used KPIs for manufacturing were sales and sales growth (64.7%); profit margins (61.8%); and customer satisfaction (84.8), while non-manufacturing used quality products/services (60.9%); sales and sales growth (54.3%) and employee development (54.3%) respectively. Limitation and implication were also discussed. Keywords: Key performance indicator, innovation, SME, performance 1. Introduction This study aims to provide a tentative framework for exploring a proposed relationship between perceived importance and the actual use of financial and non-financial indicators capturing technical and administrative innovation performances among small and mediumsized organizations in Thailand. In this study, an innovation is considered to be “A technology or practice that an organization is using for the first time, regardless of whether other organisations have previously used the technology or practice” (Klein et al., 2001). Key performance indicators (KPIs) help an organization define and measure progress toward organizational goals. KPIs are quantifiable measurements to examine the improvement in performing an innovation implementing activity that is critical to the success of a business (Cox et al., 2003). Innovation adopters often assume that investments in innovation will lead to productivity improvements. However, investment in innovation does not guarantee the effective implementation. Previous research has often shown that adopted innovations (e.g. Lean production, Customer Relation Management (CRM), Enterprise Resource Planning (ERP), and Rapid Prototyping) fail to successfully complete the implementation phase and perceive the improvements (Dennis, 2003). Shiba et al.(1993) indicated that the implementation effort must be monitored and diagnosed. This is essential because it enables individuals and groups to assess where they stand in comparison to their
Corresponding author : E-mail: firstname.lastname@example.org
Hudson et al. These companies have adopted and implemented technical and/or administrative innovations in the past three years.. and (b) non-finance-based. productivity. There are organisations that attempt to evaluate the worth and effectiveness of innovation implementation by adopting financial measurement (e. construction. The sample organizations (40%) had the average organizational gross value during year 2003-2004 of 50-200 million baht. Subsequent research examined the effectiveness of balancing financial and non-financial measurement on performance measurement system (Bremser and Barsky. organizations must be capable of capturing objective (e. In order to be successful in today’s world-wide competitive environment. 1992). based mainly in Bangkok and suburban areas. 2003. Questions were asked as to whether organization realized the importance of this issue and whether organizations implement non-financial indicator as well as financial indicator to measuring innovation performance. Moreover. such as return on investment. assessing innovation performance provides the opportunity of recognising problems and weaknesses and taking corrective action before these problems escalate (Kueng. The previous studies indicated the positive relationship between managerial perception and the use of key performance indicators within organization (e. Twenty eight percent of sample organizations had the range of employees of 101-200 people. such as customer retention. automotive supplies. quality.g. By examining the innovation literature. 1993). 2. unit cost. The mail survey sent to the CEO or Managing Director of each organization. which are used for capturing technical and administrative innovation performance. the aim of the questionnaire was to explore the measurements. 1999. In addition. Thus. et al. Kanji and Sa'. the present study hypothesized that there is a positive correlations between perceived importance and the actual usage of performance indicator such that organizations which perceive the importance of performance indicators would utilize such indicators to measure innovation performance. 2004. 2001. and how managers perceived the importance of these performance indicators.g. return on investment). Many of the measurements used today are those developed by accountants and are financially oriented (Barbosa. Kerssens-van Drongelen and Bilderbeek. satisfaction) performances.. These traditional evaluations do not provide feedback on the effectiveness of internal process.. 1972). and employee development. profit) and subjective (e. Methodology 2. 2004.g. Of these.S. pharmaceuticals. Savioz and Blum. 34% were in the manufacturing industry. No single performance indicator can capture the full complexity of an innovation performance (Amaratunga et al.2 Performance metrics The performance metrics which are commonly used in business fall into two categories: (a) finance-based. 2001). the common 14 performance metrics (Table 1) were employed in the past research to measure 24 . Cox. and telecommunication). Kaplan and Norton. Norman and Bahiri. assessing innovation performance enables organisation claiming their implementation effectiveness accurately. 2002. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 competitors.1 Participant The sample was 86 small and medium sized-organizations in Thailand. 2002). perhaps.g. This paper aimed to explore the proportion associated with the perceived importance and the actual use of performance indicators from manufacturing and non-manufacturing industries. 2004. 2000). has been growing and productive because of the innovation. Kaplan and Norton. 46% were in the non-manufacturing industries (e. organisation.g. and employees perspectives (AbdelMaksound. 2. As a result. but traditional measurements have failed to capture that growth.
The perceived importance and the actual use of financial indicators were found to be significantly related among manufacturing and nonmanufacturing industries. 25 . Aycock et al. 3. participants were also asked to identify the actual measurement systems used to collect information on each metric. The Thai version was then back-translated into English by a bilingual volunteer.. Results The two-way cross tabulation chi-square analysis was conducted to evaluate whether organizations which perceived the importance of performance indicators would utilize such indicators to measure innovation performance. List of the studied KPIs. Following this. 46) = 78. Prajogo and Sohal. Definition Return on investment Various profit margin measures Sales and sales growth Payback and payback period Cash flow Customer satisfaction Customer retention rate Labour productivity Quality of products and/or services Lead time Delivery reliability and/or speed Process time Employee development Employee knowledge KPI KPI 1 KPI 2 KPI 3 KPI 4 KPI 5 KPI 6 KPI 7 KPI 8 KPI 9 KPI 10 KPI 11 KPI 12 KPI 13 KPI 14 Respondents were asked whether they think these metric were important for their organisation and whether their organisation used these metrics to measure innovation effectiveness. Pearson χ2(25.001. p < 0 . and differences resolved through discussion. First. 34) = 54. the original questionnaire was compared with the back-translated English version. Bremser and Barsky. and Pearson χ2(25. Luria and Wiarda. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 the effectiveness of both product and processes innovation (Ahmad and Dhafr. This process ensured an accurate. the current study employed the same set of performance metrics which derived from the previous innovation studies. we employed a technique designed to decrease social desirability effects. respondents who reported that they measured employee morale may mention a measurement record system such as an employee survey as the tool used to obtain this information.000 respectively. To ensure accurate reporting. 2004.58. The translation of the questionnaire into Thai language was accomplished through a twostage translation-back translation procedure. 2002. 1996. literal translation of the original English language version of the questionnaire. I translated the questionnaire from English into Thai. p < 0. 2001). For example. Thus. Because some respondents may report using these metrics to measure innovation effectiveness but may not actually use them.S. who was not aware of the purpose of the study. Table 1.27. 1999.
p < 0. Follow up pairwise comparisons were conducted to demonstrate the different proportion of each indicator. Perceived importance and the actual use of indicators to measuring the innovation performance (Manufacturing) 90 80 70 60 Percentage 50 40 30 20 10 0 Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14 Actual Use No Actual Use Yes Perceieved importance of KPIs Figure 1.S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 Correspondingly. the perceived importance and the actual use of non-financial indicators were also found to be significantly related among manufacturing and non-manufacturing industries. Perceived importance and the actual use of indicators to measuring the innovation performance (Non manufacturing) 100 90 80 70 60 50 40 30 20 10 0 Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14 Actual Use No Actual Use Yes Percentage Perceieved importance of KPIs Figure 2. 26 . and Pearson χ2(80. Perceived importance and the actual use of indicators to measuring the innovation performance (Non-manufacturing). Pearson χ2(81.001. Perceived importance and the actual use of indicators to measuring the innovation performance (Manufacturing). 45) = 173.001 respectively. p < 0.50. 34) = 133.80. Figures 1 and 2 show the results of these analyses.
they should always be interested in two questions: Does an organisational performance improve after implementing the innovation? To what degree are the target values fulfilled? (Kueng. profit margin. Surprisingly. in particular. return on investment. 9. the quality or products/services indicators. They found that financial measurement. did not employ those KPIs for the actual usage. 1997). such as profit. cash flow..9%). the most used KPIs for manufacturing were sales and sales growth (64. Additionally.S.3%) respectively (Figures 1 and 2). Empirical evidence indicates that using a proper performance measurement system is critical to answer those two questions (Chiesa et al. who highly recognized the importance of customer satisfaction.e. quality of products and/or services. and process time) were perceived the most importance among nonmanufacturing managers (84. Implications Assessing the innovation outcome is to ensure that the change initiative is successful et al. 2001. Further research could be conducted to capture the moderating affect. the stability of our findings is uncertain. did not use this KPI. Mostly. 93. quality of products and/or services. while KPIs 6.e. Nevertheless. 41% of non-manufacturing managers. 1996. we have to acknowledge that there may be other potential moderators existing between the managerial perception and the actual use of non-financial indicators.7%). Although. sales and sales growth (54.g. process time and employees’ development and knowledge). 5. Hudson et 27 .8%.3%. Chi-square analyses of the actual use of KPIs yielded statistically significant differences among the level of perceived importance. and 12 (i.5% respectively). traditional financial indicator (e. the results show that 38-40% of manufacturing managers. some indicators can be difficult to make a good comparison with other organizations. Conclusion We anticipated that organizations which perceived the importance of KPIs would employ those KPIs to measuring the innovation performance.8%).. Interestingly.g. As a result. Griffin and Page. The present study demonstrated the important of managerial perception toward the performance indicators. Therefore. profit margins (61. but they had used those indicators (e. customer satisfaction. pay back and cash flow) for their innovation project. they used. or even to be a world-class level.8% respectively). However. and customer satisfaction (84. Likewise. and 12 (i. organizations which did not perceived the importance of some financial indicators. sales and sales growth.e. organizations realized the importance of those financial and non-financial KPIs and used those KPIs for capturing innovation progress. Tushman and O'Reilly. and process time) were perceived the most importance among manufacturing managers (85. were mentioned significantly among managers whom they interviewed with. some organizations may not perceive the importance of some indicators.3%) and employee development (54. 4. 79. 2000).(2000).8%) while non-manufacturing used quality products/services (60. As a result. 9. This means the actual gain of can be lagged a year later after the initial innovation implementation. it has been realized that not all performance indicators are equally reliable.4% and 76. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 KPIs 3. Other limitations in our study are our sample of organization was small with the low response rate (10%).5%. This result can be explained by Jarvis et al. who highly recognized the importance of non-financial KPIs (i. and turnover. return on investment. and 84. pay back and cash flow) to measure the innovation performance. it has been recommended for organizations using multiple indicators in order to track an improvement from the innovation project. 1993. profit margin. As organizations which are implementing innovation in order to survive in a world-wide competitive market. Some indicators such as ROI can be considered as a lag indicator.
(2003) Management's perception of key performance indicators for construction. M.H. G. Based on our results. Thomas. J. M.. R&D Management. Kaplan. Conn. Page. Voss. D. (1993) Using the balanced scorecard as a strategic management system. Ahrens. Work Study.P. 71-79. R&D Management. (2001) Theory and practice in SME performance measurement systems. G. Lightfoot.C.P. P.. Kitching. Hudson. 171-176.. Ahmad. 811-824..K.F. 39. R. D. D. Unpublished PhD Thesis. 34(3). Kerssens-van Drongelen. R.. C.B. R.B. 50(5).. (2004) Manufacturing in the UK: Contemporary characteristics and performance indicators. Curran. 1096-1115.. Kanji. Journal of Small Business and Enterprise Development. Cox. 29(1). 13-27. Robotics and Computer Integrated Manufacturing. Burchill. Barbosa. 13(2). 123-134. 229-238. the more they will use them to identify the innovation performance. 41. M. Bremser.J. V.. N.S. J. Matthews.. 179-188. (2001) Process improvement through performance measurement: The balanced scorecard methodology. J.. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 al. (1999) R&D performance measurement: More than choosing a set of metrics. 129(2)...R. Sa'. Barsky. W. D. Bilderbeek. 18(3-4). Verhaeghe and Kfir.. I. G. Chiesa. 71(5). 35-46. M.L. 13(1). (1996) Development of a technological innovation audit. A. Jean-Normand. Sorra.P. Klein.G. Kaplan. Kanji's business scorecard. University of Puerto Rico.W. 98(4). Journal of Construction Engineering and Management. Smart. R. 51-76.M. Journal of Production Innovation Management. Journal of Product Innovation Management.. P.A. K. Dhafr. increasing awareness of balancing financial and non-financial indicators among managers would enhance the actual utilization of those indicators. M.. N.S. D. (2002). 7(2).M. (2003) What if your feaf plan isn't working? Canadian Machinery and Metalworking. 105-136.S. 291-308. Total Quality Mangement. Sarshar. 15(2). (1999) TL 9000 performance metrics to drive improvement. Norton.. Journal of Applied Psychology.. Dennis. 10(4). Journal of Manufacturing Technology Management.. Bourne. 2001.A. (2004) Utilizing the balanced scorecard for R&D performance measurement. Providing training. D. R. J. Harvard Business Review. (1992) The balanced scoredcard-measures that drive performance. F. J. Ford. A. P. References Abdel-Maksound..P. Coughlan. D. Griffin.. International Journal of Operations & Production Management.. Issa.. A. 32(7). (2004) Assessment of PR manufacturing readiness with respect to communication and information technology and its impact on productivity. seminar or discussion network regarding to these issues will develop the higher level of perceived importance of financial and non-financial KPIs. Aycock. (2000) The use of quantitative and qualitative criteria in the measurement of performance in small firms. J. 155-171. Quality Progress. (2001) Theoretical foundation of the seven infrastructures model.. Harvard Business Review. 86(5).. A. 134-148. 28 . (1993) An interim report on measuring product development success and failure. (2002) Establishing and improving manufacturing performance measures. Evans. 70(1).E.G. (2001) Implementing computerzied technology: An organization analysis. The more managers perceive the significance benefits of KPIs. Amaratunga.R. C. Norton. 142-151.S.. Baldry. 10(2). 21(8). 2002)..Y. Jarvis. Center of Quality of Management Journal. A.
233. Kfir. D..A. Shiba. 22(2). Verhaeghe. Research Policy. 539-558. MA. A. Oregon Productivity Press. P..S. D. Savioz. Total Quality Management.I.. O'Reilly. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29 Kueng. (2002) Managing innovation in a knowledge intensive technology organisation (KITO). 32(5).. Norman. 67-85. R&D Management. S. Technovation. Bahiri. C. 29 . Blum. 11(1).. M. (1996) Performance benchmarking and measuring program impacts on customers: Lessons from the Midwest Manufacturing Technology Center. A. Tushman. Technovation. Wiarda. 21(9). M. 25(2).. A. (1972) Productivity Measurement and Incentives. P. (2001) TQM and innovation: A literature review and research framework. Luria. 409-417. R. London. Prajogo. Portland.S. (1993) New American TQM.. D.G. (2000) Process performance measurement system: A tool to support process-based organizations. Graham. Butterworths. (1997) Winning through innovation: A practical guide to leading organizational change and renewal Boston. 91-100. Harvard Business School Press. Walden. (2002) Strategic forecast tool for SMEs: How the opportunity landscape interacts with business strategy to anticipate technological trends. S. Sohal. E.. R.