Asia Pacific Management Review 16(1) (2011) 23-29

www.apmr.management.ncku.edu.tw

Key Performance Indicators for Innovation Implementation: Perception vs. Actual Usage
Sukanlaya Sawang*
School of Management, Queensland University of Technology, Australia Received 18 April 2009; Received in revised form 11 June, 2009; Accepted 7 September 2009

Abstract This paper aimed to explore the proportion associated with the perceived importance and the actual use of performance indicators from manufacturing and non manufacturing industries. The sample was 86 small and medium sized-organizations in Thailand. The perceived importance and the actual use of financial and non-financial indicators were found to be significantly related among manufacturing and non-manufacturing industries. Three key performance indicators (KPIs) (i.e. sales and sales growth, quality of products and/or services, and process time) were perceived the most importance among manufacturing managers (85.3%, 79.4% and 76.5% respectively). Three KPIs (i.e. customer satisfaction, quality of products and/or services; and process time) were perceived the most importance among nonmanufacturing managers (84.8%, 93.5%, and 84.8% respectively). Interestingly, the most used KPIs for manufacturing were sales and sales growth (64.7%); profit margins (61.8%); and customer satisfaction (84.8), while non-manufacturing used quality products/services (60.9%); sales and sales growth (54.3%) and employee development (54.3%) respectively. Limitation and implication were also discussed. Keywords: Key performance indicator, innovation, SME, performance 1. Introduction This study aims to provide a tentative framework for exploring a proposed relationship between perceived importance and the actual use of financial and non-financial indicators capturing technical and administrative innovation performances among small and mediumsized organizations in Thailand. In this study, an innovation is considered to be A technology or practice that an organization is using for the first time, regardless of whether other organisations have previously used the technology or practice (Klein et al., 2001). Key performance indicators (KPIs) help an organization define and measure progress toward organizational goals. KPIs are quantifiable measurements to examine the improvement in performing an innovation implementing activity that is critical to the success of a business (Cox et al., 2003). Innovation adopters often assume that investments in innovation will lead to productivity improvements. However, investment in innovation does not guarantee the effective implementation. Previous research has often shown that adopted innovations (e.g. Lean production, Customer Relation Management (CRM), Enterprise Resource Planning (ERP), and Rapid Prototyping) fail to successfully complete the implementation phase and perceive the improvements (Dennis, 2003). Shiba et al.(1993) indicated that the implementation effort must be monitored and diagnosed. This is essential because it enables individuals and groups to assess where they stand in comparison to their
*

Corresponding author : E-mail: s.sawang@qut.edu.au

23

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

competitors. In addition, assessing innovation performance provides the opportunity of recognising problems and weaknesses and taking corrective action before these problems escalate (Kueng, 2000). Moreover, assessing innovation performance enables organisation claiming their implementation effectiveness accurately. As a result, organisation, perhaps, has been growing and productive because of the innovation, but traditional measurements have failed to capture that growth. Many of the measurements used today are those developed by accountants and are financially oriented (Barbosa, 2004; Norman and Bahiri, 1972). There are organisations that attempt to evaluate the worth and effectiveness of innovation implementation by adopting financial measurement (e.g. return on investment). These traditional evaluations do not provide feedback on the effectiveness of internal process, and employees perspectives (AbdelMaksound, 2004; Kaplan and Norton, 1992). No single performance indicator can capture the full complexity of an innovation performance (Amaratunga et al., 2001). In order to be successful in todays world-wide competitive environment, organizations must be capable of capturing objective (e.g. unit cost, profit) and subjective (e.g. quality, satisfaction) performances. Subsequent research examined the effectiveness of balancing financial and non-financial measurement on performance measurement system (Bremser and Barsky, 2004; Hudson et al., 2001; Kanji and Sa', 2002; Kerssens-van Drongelen and Bilderbeek, 1999; Savioz and Blum, 2002). Questions were asked as to whether organization realized the importance of this issue and whether organizations implement non-financial indicator as well as financial indicator to measuring innovation performance. This paper aimed to explore the proportion associated with the perceived importance and the actual use of performance indicators from manufacturing and non-manufacturing industries. The previous studies indicated the positive relationship between managerial perception and the use of key performance indicators within organization (e.g. Cox, et al., 2003; Kaplan and Norton, 1993). Thus, the present study hypothesized that there is a positive correlations between perceived importance and the actual usage of performance indicator such that organizations which perceive the importance of performance indicators would utilize such indicators to measure innovation performance. 2. Methodology 2.1 Participant The sample was 86 small and medium sized-organizations in Thailand, based mainly in Bangkok and suburban areas. These companies have adopted and implemented technical and/or administrative innovations in the past three years. Of these, 34% were in the manufacturing industry, 46% were in the non-manufacturing industries (e.g. automotive supplies, construction, pharmaceuticals, and telecommunication). The sample organizations (40%) had the average organizational gross value during year 2003-2004 of 50-200 million baht. Twenty eight percent of sample organizations had the range of employees of 101-200 people. The mail survey sent to the CEO or Managing Director of each organization; the aim of the questionnaire was to explore the measurements, which are used for capturing technical and administrative innovation performance, and how managers perceived the importance of these performance indicators. 2.2 Performance metrics The performance metrics which are commonly used in business fall into two categories: (a) finance-based, such as return on investment, and (b) non-finance-based, such as customer retention, productivity, and employee development. By examining the innovation literature, the common 14 performance metrics (Table 1) were employed in the past research to measure

24

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

the effectiveness of both product and processes innovation (Ahmad and Dhafr, 2002; Aycock et al., 1999; Bremser and Barsky, 2004; Luria and Wiarda, 1996; Prajogo and Sohal, 2001). Thus, the current study employed the same set of performance metrics which derived from the previous innovation studies. Table 1. List of the studied KPIs. Definition Return on investment Various profit margin measures Sales and sales growth Payback and payback period Cash flow Customer satisfaction Customer retention rate Labour productivity Quality of products and/or services Lead time Delivery reliability and/or speed Process time Employee development Employee knowledge

KPI KPI 1 KPI 2 KPI 3 KPI 4 KPI 5 KPI 6 KPI 7 KPI 8 KPI 9 KPI 10 KPI 11 KPI 12 KPI 13 KPI 14

Respondents were asked whether they think these metric were important for their organisation and whether their organisation used these metrics to measure innovation effectiveness. Because some respondents may report using these metrics to measure innovation effectiveness but may not actually use them, we employed a technique designed to decrease social desirability effects. To ensure accurate reporting, participants were also asked to identify the actual measurement systems used to collect information on each metric. For example, respondents who reported that they measured employee morale may mention a measurement record system such as an employee survey as the tool used to obtain this information. The translation of the questionnaire into Thai language was accomplished through a twostage translation-back translation procedure. First, I translated the questionnaire from English into Thai. The Thai version was then back-translated into English by a bilingual volunteer, who was not aware of the purpose of the study. Following this, the original questionnaire was compared with the back-translated English version, and differences resolved through discussion. This process ensured an accurate, literal translation of the original English language version of the questionnaire. 3. Results The two-way cross tabulation chi-square analysis was conducted to evaluate whether organizations which perceived the importance of performance indicators would utilize such indicators to measure innovation performance. The perceived importance and the actual use of financial indicators were found to be significantly related among manufacturing and nonmanufacturing industries, Pearson 2(25, 34) = 54.27, p < 0.001; and Pearson 2(25, 46) = 78.58, p < 0 .000 respectively.

25

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

Correspondingly, the perceived importance and the actual use of non-financial indicators were also found to be significantly related among manufacturing and non-manufacturing industries, Pearson 2(81, 34) = 133.80, p < 0.001; and Pearson 2(80, 45) = 173.50, p < 0.001 respectively. Follow up pairwise comparisons were conducted to demonstrate the different proportion of each indicator. Figures 1 and 2 show the results of these analyses.
Perceived importance and the actual use of indicators to measuring the innovation performance (Manufacturing)
90 80 70 60 Percentage 50 40 30 20 10 0 Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14 Actual Use No Actual Use Yes

Perceieved importance of KPIs

Figure 1. Perceived importance and the actual use of indicators to measuring the innovation performance (Manufacturing).
Perceived importance and the actual use of indicators to measuring the innovation performance (Non manufacturing)
100 90 80 70 60 50 40 30 20 10 0 Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14 Actual Use No Actual Use Yes

Percentage

Perceieved importance of KPIs

Figure 2. Perceived importance and the actual use of indicators to measuring the innovation performance (Non-manufacturing).

26

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

KPIs 3, 9, and 12 (i.e. sales and sales growth; quality of products and/or services; and process time) were perceived the most importance among manufacturing managers (85.3%, 79.4% and 76.5% respectively), while KPIs 6, 9, and 12 (i.e. customer satisfaction, quality of products and/or services; and process time) were perceived the most importance among nonmanufacturing managers (84.8%, 93.5%, and 84.8% respectively). Interestingly, the most used KPIs for manufacturing were sales and sales growth (64.7%); profit margins (61.8%); and customer satisfaction (84.8%) while non-manufacturing used quality products/services (60.9%); sales and sales growth (54.3%) and employee development (54.3%) respectively (Figures 1 and 2). Although, some organizations may not perceive the importance of some indicators, they used, in particular, traditional financial indicator (e.g. return on investment, profit margin, pay back and cash flow) to measure the innovation performance. 4. Conclusion We anticipated that organizations which perceived the importance of KPIs would employ those KPIs to measuring the innovation performance. Chi-square analyses of the actual use of KPIs yielded statistically significant differences among the level of perceived importance. Mostly, organizations realized the importance of those financial and non-financial KPIs and used those KPIs for capturing innovation progress. Surprisingly, organizations which did not perceived the importance of some financial indicators, but they had used those indicators (e.g. return on investment, profit margin, pay back and cash flow) for their innovation project. This result can be explained by Jarvis et al.(2000). They found that financial measurement, such as profit, cash flow, and turnover, were mentioned significantly among managers whom they interviewed with. However, the results show that 38-40% of manufacturing managers, who highly recognized the importance of non-financial KPIs (i.e. the quality or products/services indicators, process time and employees development and knowledge), did not employ those KPIs for the actual usage. Likewise, 41% of non-manufacturing managers, who highly recognized the importance of customer satisfaction, did not use this KPI. As a result, we have to acknowledge that there may be other potential moderators existing between the managerial perception and the actual use of non-financial indicators. Further research could be conducted to capture the moderating affect. Other limitations in our study are our sample of organization was small with the low response rate (10%). Therefore, the stability of our findings is uncertain. The present study demonstrated the important of managerial perception toward the performance indicators. Nevertheless, it has been realized that not all performance indicators are equally reliable. Some indicators such as ROI can be considered as a lag indicator. This means the actual gain of can be lagged a year later after the initial innovation implementation. Additionally, some indicators can be difficult to make a good comparison with other organizations. As a result, it has been recommended for organizations using multiple indicators in order to track an improvement from the innovation project. 5. Implications Assessing the innovation outcome is to ensure that the change initiative is successful et al., 2001; Tushman and O'Reilly, 1997). As organizations which are implementing innovation in order to survive in a world-wide competitive market, or even to be a world-class level, they should always be interested in two questions: Does an organisational performance improve after implementing the innovation? To what degree are the target values fulfilled? (Kueng, 2000). Empirical evidence indicates that using a proper performance measurement system is critical to answer those two questions (Chiesa et al., 1996; Griffin and Page, 1993; Hudson et

27

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

al., 2001; Verhaeghe and Kfir, 2002). Based on our results, increasing awareness of balancing financial and non-financial indicators among managers would enhance the actual utilization of those indicators. The more managers perceive the significance benefits of KPIs, the more they will use them to identify the innovation performance. Providing training, seminar or discussion network regarding to these issues will develop the higher level of perceived importance of financial and non-financial KPIs. References Abdel-Maksound, A.B. (2004) Manufacturing in the UK: Contemporary characteristics and performance indicators. Journal of Manufacturing Technology Management, 15(2), 155-171. Ahmad, M.M., Dhafr, N. (2002) Establishing and improving manufacturing performance measures. Robotics and Computer Integrated Manufacturing, 18(3-4), 171-176. Amaratunga, D., Baldry, D., Sarshar, M. (2001) Process improvement through performance measurement: The balanced scorecard methodology. Work Study, 50(5), 179-188. Aycock, D.G., Jean-Normand, D., Thomas, F.Y. (1999) TL 9000 performance metrics to drive improvement. Quality Progress, 32(7), 41. Barbosa, J.P. (2004) Assessment of PR manufacturing readiness with respect to communication and information technology and its impact on productivity. Unpublished PhD Thesis, University of Puerto Rico. Bremser, W.G., Barsky, N.P. (2004) Utilizing the balanced scorecard for R&D performance measurement. R&D Management, 34(3), 229-238. Chiesa, V., Coughlan, P., Voss, C.A. (1996) Development of a technological innovation audit. Journal of Production Innovation Management, 13(2), 105-136. Cox, R.F., Issa, R.R.A., Ahrens, D. (2003) Management's perception of key performance indicators for construction. Journal of Construction Engineering and Management, 129(2), 142-151. Dennis, P. (2003) What if your feaf plan isn't working? Canadian Machinery and Metalworking, 98(4), 39. Ford, M.W., Evans, J.R., Matthews, C.H., Burchill, G. (2001) Theoretical foundation of the seven infrastructures model. Center of Quality of Management Journal, 10(2), 51-76. Griffin, A., Page, A.L. (1993) An interim report on measuring product development success and failure. Journal of Product Innovation Management, 10(4), 291-308. Hudson, M., Smart, A., Bourne, M. (2001) Theory and practice in SME performance measurement systems. International Journal of Operations & Production Management, 21(8), 1096-1115. Jarvis, R., Curran, J., Kitching, J., Lightfoot, G. (2000) The use of quantitative and qualitative criteria in the measurement of performance in small firms. Journal of Small Business and Enterprise Development, 7(2), 123-134. Kanji, G.K., Sa', P.M.E. (2002). Kanji's business scorecard. Total Quality Mangement, 13(1), 13-27. Kaplan, R.S., Norton, D.P. (1992) The balanced scoredcard-measures that drive performance. Harvard Business Review, 70(1), 71-79. Kaplan, R.S., Norton, D.P. (1993) Using the balanced scorecard as a strategic management system. Harvard Business Review, 71(5), 134-148. Kerssens-van Drongelen, I.C., Bilderbeek, J. (1999) R&D performance measurement: More than choosing a set of metrics. R&D Management, 29(1), 35-46. Klein, K.J., Conn, A.B., Sorra, J.S. (2001) Implementing computerzied technology: An organization analysis. Journal of Applied Psychology, 86(5), 811-824.

28

S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

Kueng, P. (2000) Process performance measurement system: A tool to support process-based organizations. Total Quality Management, 11(1), 67-85. Luria, D., Wiarda, E. (1996) Performance benchmarking and measuring program impacts on customers: Lessons from the Midwest Manufacturing Technology Center. Research Policy, 25(2), 233. Norman, R.G., Bahiri, S. (1972) Productivity Measurement and Incentives. Butterworths, London. Prajogo, D.I., Sohal, A.S. (2001) TQM and innovation: A literature review and research framework. Technovation, 21(9), 539-558. Savioz, P., Blum, M. (2002) Strategic forecast tool for SMEs: How the opportunity landscape interacts with business strategy to anticipate technological trends. Technovation, 22(2), 91-100. Shiba, S., Graham, A., Walden, D. (1993) New American TQM. Oregon Productivity Press, Portland. Tushman, M., O'Reilly, C.A. (1997) Winning through innovation: A practical guide to leading organizational change and renewal Boston, Harvard Business School Press, MA. Verhaeghe, A., Kfir, R. (2002) Managing innovation in a knowledge intensive technology organisation (KITO). R&D Management, 32(5), 409-417.

29