Qual Quant (2015) 49:1945–1967

DOI 10.1007/s11135-014-0083-y

Influences of knowledge spillover and utilization on the

NIS performance: a multi-stage efficiency perspective

Chia-Chin Chang

Received: 22 May 2014 / Accepted: 18 July 2014 / Published online: 3 August 2014
© Springer Science+Business Media Dordrecht 2014

Abstract The innovation process cannot be completed independently. Research on the

national innovation system (NIS) has attracted considerable attention. Such research reveals
that it is critically important to understand that the operation of the NIS is both multi-stage
and multi-dimensional, and ultimately shows its results in terms of national economic per-
formance. From the multi-stage efficiency perspective, we built a three-stage network data
envelopment analysis model to assess the R&D efficiency of knowledge creation fostered by
the NIS, the diffusion efficiency of knowledge spillover and the economic efficiency of the
economic achievements resulting from knowledge utilization. The study subjects consisted
of 41 countries, including China, Brazil and OECD member countries. Finally, the efficiency
results of NISs involve both cluster analysis and multidimensional scaling to make efficiency
analysis concrete and vivid, as well as to serve as benchmark targets for inefficient NISs.

Keywords Innovation system · Knowledge spillover · Cluster analysis · Innovative

capacity

1 Introduction

In a knowledge-based economy, enterprises convert knowledge into commercial operations

that drive economic growth; however, the innovation process cannot be finished indepen-
dently. Studies have shown that many factors related to the emergence, development and
diffusion of science and technology are mostly related to the country where the enterprise
is located; it is considered that a good national innovation system (NIS) can provide enter-
prises with a favorable innovation environment and allow the enterprises to develop global
competitive advantages (Kravchenko 2011; Lundvall 2007; Motohashi 2005).

C.-C. Chang (B)

Institute of Resource Management and Decision Science, National Defense University, No. 70, Sec. 2,
Zhongyang N. Road, Beitou, Taipei City 112, Taiwan
e-mail: yaemailtw@gmail.com

123
1946 C.-C. Chang

In the last three decades, the NIS concept has become the focus in numerous studies
on innovation activity (e.g., Intarakumnerd 2011; Mowery 2011; Marxt and Brunner 2013).
Kravchenko (2011) argued that there are two main reasons for this: first, NIS makes it possi-
ble to compare different countries and regions; second, it can serve as a tool for national and
regional policies. In addition, economic evolutionary theorists have argued that institutions
play a central role in supporting and molding efforts to advance technology. For example,
Nelson and Nelson (2002) emphasized that an innovation system is an institutional concept
par excellence, articulated at the national, regional and local levels, and characterized by the
interaction of different actors, including firms, universities, and technology transfer organiza-
tions. Therefore, the emergence and diffusion of innovation is affected by social subsystems,
actors, institutions and economic structures that affect the rate and direction of change in
society. Nelson and Nelson (2002) revealed that the innovation of a country depends on the
close cooperation of government, enterprises and academia; only by establishing an effec-
tive interactive network can the various key players successfully promote the creation and
diffusion of new knowledge and convert it into the fruits of economic growth.
On the other hand, the NIS is also considered a multi-dimensional operating system;
the heterogeneous organizations within the system form a network system of knowledge
flow, diffusion and value creation through interconnections with each other (OECD 1999;
Edquist 2004). The OECD (1999, p. 24) defined the NIS according to Metcalfe (1995), who
stated that a NIS is “distinct institutions which...set jointly and individually contribute to
the development and diffusion of new technologies and which provide the framework within
which governments form and implement policies to influence the innovation process. As such
it is a system of interconnected institutions to create, store and transfer the knowledge, skills
and artefacts which define new technologies.” Moreover, Edquist (2004, p.182) also argued
that a NIS includes ”all important economic, social, political, organizational, institutional,
and other factors that influence the development, diffusion, and use of innovations.”
Therefore, if the performance of a NIS is evaluated only according to a single aspect or to
a one-stage operation mode, it cannot reflect the overall perspective of the system, such as the
interactive links among enterprises, educational institutions, research institutions and gov-
ernments; nor can it reflect a series of continuous activities, such as the generation, diffusion,
transformation and utilization of knowledge. In order to systematically evaluate the perfor-
mance of a NIS, it is necessary to consider not only the R&D efficiency of knowledge (i.e.
the efficiency of the knowledge generation process), but also the influence of the diffusion
efficiency of knowledge spillover and economic efficiency after the value-added application
of knowledge. Moreover, traditional performance evaluation neglects the context and con-
nectivity between various knowledge activities in the innovation system in the measuring of
the overall efficiency of input-output; it cannot effectively present the potential management
implications and information as a consequence (e.g., Kravchenko 2011). Therefore, given
the importance of the connectivity between knowledge activities for a NIS, the appropri-
ate multi-stage performance evaluation model deserves greater scrutiny. Finally, a certain
length of time is required in regard to the creation, diffusion and transformation activities
of knowledge. The time lag between inputs and outputs needs to be taken into account in
conducting the evaluation of NIS efforts. In order to overcome the above problems, this study
established a longitudinal, multi-aspect and multi-stage performance measurement model as
the reference for NISs’ performance evaluation. This evaluation model not only considers the
staged and overall performance of a NIS, but also provides decision makers with guidance
for further improvement.
Due to its multi-stage and multi-dimensional operation structure, the system and network
concepts were considered simultaneously in this study in order to re-examine and explore the

123
Influences of knowledge spillover and utilization 1947

operating efficiency of NIS. In order to exactly evaluate the NIS performance, it is necessary
to integrate the knowledge-diffusion performance and economic performance of knowledge
utilization and value-added into the NIS performance evaluation, to more systematically
present the management implications and information of the system, as well as to further
analyze various important factors influencing the staged operation of the system. This could
provide a reference for the future improvement and perfection of NISs. To sum up, this study
compared the performance of 41 countries, including China, Brazil and OECD member
countries, at different stages, analyzed the benchmarking objects and clarified the important
variables affecting the performance at each stage through the application of a multi-stage
Network Data Envelopment Analysis (Network DEA).

2 Literature review

2.1 National innovation system

Innovation literature abounds, with each study emphasizing a different aspect of the term.
Schumpeter (1934, 1942) was among the first to emphasize the role of innovation in the
entrepreneurial process and outlined an economic process of “creative destruction” (Lumpkin
and Dess 1996). He stressed the novelty aspect and novel outputs, which could be summarized
as ‘doing things differently’.
Innovation research, however, has changed considerably, and the traditional Schum-
peterian approach has been found to be inadequate, based on current views concerning
the innovation process. Innovation is not an exclusive internal activity undertaken by firms
to achieve monopolistic advantages. Some scholars (e.g. Kline and Rosenberg 1986; Dosi
1988; Kaufmann and Tödtling 2001) argued that innovation is an evolutionary, non-linear
and interactive process between a firm and its environment. For example, Kaufmann and
Tödtling (2001) argued that the concept of non-linearity implies that innovation is stimulated
and influenced by many actors and information sources, both inside and outside the firm; it is
determined not only by scientists and engineers working on R&D, but also by the feedback
on the interactions among production, marketing and customers in the early phases of the
innovation process. Thus, innovations are embedded in specific social, economic, political
and cultural contexts; they are context-dependent and have a systemic character.
Increasingly, the concept of innovation systems is being applied at the national level.
The term NIS was used first by Freeman (1987) and later employed by Lundvall (1992) to
emphasize the importance of knowledge and learning. Freeman (1987) defined NIS as a set of
public and private institutions whose activities and interactions generate, import, change, and
diffuse new technologies. Lundvall (1992, p. 2) emphasized the importance of knowledge and
learning as part of “ a system of innovation... constituted by elements and relationships which
interact in the production, diffusion and use of new, and economically useful, knowledge.”
Lundvall (1992) further differentiated between narrow and broad innovation systems. A
narrow definition of the innovation system primarily incorporates the R&D functions of
universities, public and private research institutes and corporations, reflecting the acquisition
and dissemination of knowledge. A broader concept includes all parts and aspects of the
economic structure and the institutional set-up affecting learning, as well as the searching
and exploring activities (Lundvall 1992, p. 12).
Based on the mechanism of innovation generation, the two major and complementary
“organizational” and “institutional” dimensions of the innovation process are elaborated
(Coriat and Weinstein 2002). The former goes inside the black box of organizational struc-

123
1948 C.-C. Chang

tures, with the aim of capturing the determinants of a firm’s capacity to innovate; the latter
emphasizes the characteristics of the global structure of economic systems, and of the inter-
actions between actors, thus leading to the insistence on the role of institutions. Malerba
(2004) used three dimensions to characterize and analyze sectoral systems of innovation:
technological knowledge, actors or networks, and institutions.
In addition, economic evolutionary theorists (e.g. Freeman 1987; Nelson 1993) also argued
that a variety of national institutions have greatly affected the relative rates of technical change
and, hence, of economic growth in various countries. These institutions provide the core for
the analyses of NISs. For example, Nelson (1993) presented a comparative study between
various NISs, and concluded that factors such as: strong core competencies, high-quality
education and training, and stable and facilitative economic and trade policies, all have an
influence on national innovative performance. In sum, these studies have shown that NISs are
multi-dimensional operation systems which differ significantly between countries because
countries have different economic structures, knowledge bases and institutional specificities.
On a different note, the literature on innovation systems generally defines organizations and
institutions according to their main components, as they constitute the structure of such sys-
tems. However, this “structure-based” approach includes a fundamental weak point in regard
to the determinants of innovation; Kitanovic (2007) pointed out that the main activities of the
innovation process could be interpreted as the determinants when trying to analyze NIS. This
“process–based” approach is appropriate for describing the actual processes that take place
in an innovation system. The departure point is the main function of innovation systems,
a function that is considered to be creative, diffusive and innovative (Edquist 2004). In the
context of this article, we will not try to argue about the relevance and importance of certain
activities, where activities are defined as determinants serving the main function of innovation
systems in terms of creation, diffusion and utilization of innovations. Metcalfe (1995) stated
that a NIS is a set of unique organizations that promote the development and diffusion of
new technologies, and that they put pressure on governments to form and implement a policy
framework related to innovation. Therefore, a NIS is composed of interconnected systems
that create, save and transfer knowledge, skills and new technical products. Cooke and Gomez
Uranga (1997) pointed out that innovation is not only an interactive social process, but also
the process of converting knowledge into practical commodities, involving feedback at dif-
ferent points in the innovation process, as it involves knowledge development, diffusion and
deployment. Nevertheless, a NIS has to be looked at from two angles: organizational elements
and the linkages between them. “University research, research institutes, technology-transfer
agencies, consultants, skills-development organizations, public and private funding organi-
zations... involved in innovation are the main elements. Linkages can be specified in terms
of flows of knowledge and information, flows of investment funding, flows of authority and
even more informal arrangements such as networks, clubs, fora and partnerships” (Cooke
and Gomez Uranga 1997, p. 478). Conceivably, the tripartite academia-industry-government
relations provide a neo-evolutionary model of the process of innovation. This model states
that universities can be a factor for socio-economic development within trilateral interactions
among academia, industry and government (Etzkowitz and Leydesdorff 2000)
In short, a NIS is an interactive network for the creation, diffusion and application of
knowledge and technology between the organizations in the system (Freeman 1987; Lundvall
1992; OECD 1999). Its links consist not only of knowledge flows, but also financial flows,
human flows and regulation flows. Figure 1 shows the innovation system proposed by the
OECD in 1999, including its linked structure and complexity.
In 1996, the OECD pointed out that in a knowledge-based economy, the NIS structure
is an important determinant of the economy. The OECD argues that “the knowledge-based

123
Influences of knowledge spillover and utilization 1949

Fig. 1 Actors and linkages in the innovation system (Source OECD 1999 p. 23)

economy places great importance on the diffusion and use of information and knowledge as
well as its creation. The determinants of success of enterprises, and of national economies
as a whole, is ever more reliant upon their effectiveness in gathering and utilizing knowl-
edge (OECD 1996, p. 14)”. Subsequently, the Asia Pacific Economic Cooperation forum
(APEC 2000) referred to a knowledge-based economy as an economy in which the produc-
tion, distribution and use of knowledge is the main driver of growth, wealth creation and
employment across all industries. The definitions presented by both the OECD and APEC
show that knowledge could be isolated as an input factor, and that its importance exceeds
real production elements such as land and capital.
Michael E. Porter’s book The Competitive Advantage of Nations (Porter 1998) pointed
out that in a world with fierce competition, traditional natural resources and capital are no
longer the main factors of economic advantage, and that the creation, diffusion and applica-
tion of new knowledge is more important. Therefore, in a knowledge economy, the economic
strength of a country can almost be represented by its knowledge stock and degree of dif-
fusion. As a result, in a NIS, the interaction between organizations in the system, such as
industry-academia cooperation, promotes the diffusion, commercialization and combination
of technical knowledge in regard to contemporary economic development; this interaction
promotes the so-called capitalization of knowledge. Its output will not only affect the techno-
logical industry and high-tech exports, but will also directly affect the economic development
(such as GDP) and competitive power of a country (Etzkowitz and Leydesdorff 2000; Park
and Lee 2006). The operation of a NIS allows new knowledge to develop from scratch, and

123
1950 C.-C. Chang

this development will eventually be reflected in the economic growth of a country. According
to the observations of OECD regarding the development of technological innovation policies
in global industries, technology and innovation are the major driving forces promoting eco-
nomic growth in global and regional competition (OECD 2001). The main members of the
NIS create new knowledge through interaction, diffuse it throughout the system, and then
further use it and add value to it, causing it to achieve economic growth.
Various scholars (e.g., Cooke and Gomez Uranga 1997; Etzkowitz and Leydesdorff 2000;
Edquist 2004) have argued that the interactions among different organizations, such as enter-
prises, educational institutions, research institutions and governments, as well as the proper
use of national resources, are bound to have an influence on the R&D, diffusion, transforma-
tion and utilization of technical knowledge and other activities. Therefore, in order to evaluate
the performance of a NIS, it is necessary to consider not only the R&D and output efficiency
of technological knowledge, but also the influence of the diffusion efficiency of knowledge
spillover and economic efficiency after the value-added application of the knowledge. It is
thus necessary to reexamine the NIS using the multi-stage efficiency evaluation because
the operation of a NIS plays a decisive role in national economic development, as well as
enterprise innovation and transformation. A country must first develop various components
and form an efficient NIS before it can display excellent competitiveness in a knowledge
economy.

2.2 Efficiency analysis on NISs

In recent years, efficiency analysis issues concerning NISs have become increasingly impor-
tant, primarily because of international economic competition, especially in regard to indus-
trial innovation. Nasierowski and Arcelus (2003) distinguished the national productivity
and R&D contribution using the one-stage DEA method. They adopted 45 countries in the
Global Competitiveness Report as their research objects, in order to measure the competence
of various countries in converting input to output. The variables used included input, such as
the national R&D expenditure and proportion of educational expenditure in GDP, and out-
put, such as the number of residents’ patents in foreign countries, the number of residents’
patents in their own countries and national productivity. The research findings showed that
most OECD member countries have low investments in R&D, but that most non-member
countries of the OECD in Eastern Europe, Latin America and Asia have excessive invest-
ments in R&D, which undermines the overall efficiency, suggesting that the investment of
such countries in R&D is still at the early development stage.
Lai and Shyu (2003), using the one-stage DEA method, measured the performance of the
NISs of OECD members. The national R&D expenditure, input degree of private sectors in
R&D, the number of R&D personnel in the whole country, gross educational expenditure and
direct investment flowing from foreign countries were all used as the input; the number of
national patent applications, the number of patents registered in the United States, frequency
of citation of scientific periodicals, and number of dissertations in scientific periodicals and
productivity were used as the output. The results showed that among 23 countries, Belgium,
Finland, Ireland, Norway, Poland, Switzerland and the United States were efficient for two
consecutive years. Wang and Huang (2007) analyzed the relative R&D efficiency of the
innovation system of 23 OECD member countries and seven non-member countries, using
the one-stage DEA. R&D expenditure and human expenditure were used as the input, and
patent and academic publications were used as the output. The influence of the external
environment was also considered. The research findings showed that less than half of the
countries were efficient, that more than 2/3 of the countries had increasing returns to scale,

123
Influences of knowledge spillover and utilization 1951

and that most countries had more advantages in academic publications than in patent output.
Pan et al. (2010) studied the operating performance of the NISs in 33 Asian and European
countries using the one-stage bilateral DEA model. The result revealed that the overall tech-
nical inefficiencies of the NIS activities across countries were primarily due to pure technical
inefficiencies rather than scale inefficiencies.
As can be seen from the above studies, the initial input elements of a NIS can be summa-
rized in two categories: capital and labor; the former is related to technology or knowledge
creation, and the latter reflects the state of human factors related to R&D. However, while all
of the related studies use the one-stage DEA method, it cannot fully reflect all of the infor-
mation in a series of production processes of a NIS, especially the economic results created
by knowledge or technology diffusion. A NIS includes a continuous process from the input
in R&D activities to the output of knowledge or technology, which affects the economic
growth of a country. Accordingly, this study proposed the multi-stage Network DEA model
to investigate the R&D efficiency of knowledge/technology outputs, as well as to consider
the diffusion efficiency of knowledge spillover and subsequent economic efficiency obtained
using the innovation technology.

3 Research design

3.1 Research framework

At present, the multi-index framework is used in most performance evaluations and analyses.
This framework may also be combined with qualitative and quantitative indexes. Traditional
performance evaluation and analysis neglect the context and connectivity between various
activities in the system when measuring the overall efficiency of input-output and, conse-
quently, it cannot effectively present the potential management implications and information.
Therefore, in order to overcome the above problems, this study established a three-stage
performance measurement model as the performance evaluation tool of the NIS using the
performance analysis method, which can evaluate multiple input-outputs. Based on the above
description, this study reexamined the definition of NISs by integrating the studies of OECD
(1999) and Edquist (2004), and argued that in order to clearly evaluate the performance of
a NIS, it is necessary to consider not only the R&D efficiency of technological knowledge
but also the diffusion efficiency of knowledge and economic efficiency after value-added
application, in order to more integrally present the implication and information of the inno-
vation system with respect to management and operation. This study further analyzed various
important factors influencing staged operations of the system, so that it could serve as a ref-
erence for the future improvement and perfection of NISs. Figure 2 shows the details of the
research model in this paper.

3.2 Selection of the inputs and outputs

In this study, the input and output were selected by referring to the related literatures of
previous scholars. Table 1 lists the description and data sources of the input and output
variables in the three stages. Input elements affecting the NIS of a country are nothing
more than the two major production elements of capital and labor; the former is related to
the source of the R&D funds, and the latter reflects the state of human factors of R&D.
Therefore, in the inputs of the first stage (R&D efficiency), the capital inputs included the
total expenditure on R&D (TERD), Business expenditure on R&D (BERD), and total public

123
1952 C.-C. Chang

Fig. 2 A multi-stage performance measurement model of NIS

expenditure on education (EDU) (Balzat and Hanusch 2004; Nasierowski and Arcelus 2003;
Wang and Huang 2007). The labor inputs included the total R&D personnel nationwide per
capita (RDPN) and the total R&D personnel in business per capita (RDPB) (Lai and Shyu
2003; Wang and Huang 2007). An additional input was the technological regulation (TR) that
the purpose of the index is to measure if TR supports business development and innovation.
(Kneller and Manderson 2012). Moreover, since a certain length of time is required regarding
to the creation, diffusion and transformation activities of knowledge, a time lag between inputs
and outputs needs to be taken into account in conducting a deterministic DEA evaluation
of NIS efforts (Cullmann et al. 2009). This study sets the time lag between stages to be at
least one year. For example, the input data of first stage set for 2007 is thus matched with the
output data set for 2008
Outputs of the R&D efficiency in the first stage of this research included the total number
of patents granted to residents (PGTR), the number of scientific articles (SA), and the effect of
basic research (BR) on long-term economic development of a country. A patent not only has
the legal effect of protecting the patent rights of the patentees but is also regarded as an index
to measure the innovation capacity of a country, an organization or an enterprise (Cantwell
and Fai 1999; Zander 1997). Since the publication of the book Invention and Economic
Growth from the economist Schmookler (1966), which links patent statistics and economic
growth, patent statistics have been widely used by scholars as an index to measure R&D
and innovation. Examples include Pavitt (1982, 1985), Basberg (1987), Griliches (1990),
Archibugi (1992), OECD (1994) and Archibugi and Pianta (1996). In addition, in previous
studies, variables such as the SCI (Science Citation Index), the SSCI (Social Sciences Citation
Index) and the EI (Engineering Index) have often been used as the indexes to represent basic
research-oriented achievements. Therefore, the number of scientific articles and whether
basic research (BR) could strengthen the degree of long-term economic development were
taken as the second and third output index of the R&D efficiency. At the same time, the above
three outputs were also used as the inputs in the next stage.
The second stage was the knowledge diffusion stage. In addition to the three outputs
in the first stage, the internet is the main tool for knowledge diffusion at present (Whelan
et al. 2010); therefore, internet bandwidth speed (IBS) was increased as a new input in

123
Influences of knowledge spillover and utilization 1953

Table 1 Explanation of input and output variables

Variable Definition (unit) Sources

TERD Total expenditure on R&D. (Percentage of GDP) WCY(2009) Data(2007)

BERD Business expenditure on R&D. (US$ millions) WCY(2009) Data(2007)
EDU Total public expenditure on education. (Percentage of WCY(2009) Data(2007)
GDP)
RDPN Total R&D personnel nationwide per capita. (FTE per WCY(2009) Data(2007)
1,000 people)
RDPB Total R&D personnel in business per capita. (FTE per WCY(2009) Data(2007)
1,000 people)
TR Technological regulation. The purpose of the index is to WCY(2007) Data(2007)
measure if TR supports business development and
innovation. (scale 0–10, survey data)
SA Number of scientific articles published by origin of WCY(2012) Data(2009)
author.
PGTR Number of patents granted to residents. (Average WIPO
2008–2010)
BR Basic research (BR) is the basis for many commercial WCY(2008) Data(2008)
products and applied research. The purpose of the
index is to measure if BR does enhance long-term
economic development. (scale 0–10, survey data)
IBS Internet bandwidth speed. (Per internet user, kbps) ITU(2008)
IUSER Internet users. (Number of internet users per 1,000 CIA (2009)
people)
KT Knowledge transfer. The purpose of the index is to WCY(2009) Data(2009)
measure if KT is highly developed between
companies and universities.
TC Technological cooperation. The purpose of the index is WCY(2009) Data(2009)
to measure if TC between companies is developed.
(scale 0–10, survey data)
IC Innovative capacity. The purpose of the index is to WCY(2010) Data(2010)
measure if IC of firms (to generate new products,
processes and/or services) is high.
GDP Gross domestic product. (US$ billions) WCY(2011) Data(2010)
HTEX High-tech exports. (US$ millions) World Bank (2010)

this stage. The output index included the number of internet users (IUSER), the degree of
knowledge transfer between companies and universities (KT), and the degree of technological
cooperation between companies (TC). Finally, the inputs of economic efficiency in the third
stage included the above two outputs of the second stage, KT and TC, with the addition of the
innovation capacity (IC) of firms. IC was used to represent the capacity of firms to convert
knowledge/technology to new products, processes and service modes. The outputs included
the gross domestic product (GDP) and high-tech exports (HTEX) (Balzat and Hanusch 2004;
Nasierowski and Arcelus 2003; Niosi 2002). The GDP reflects the national income of a
country and is the index of economic growth, while the HTEX shows the strength of the
industrial competitiveness of a country.
In addition, Golany and Roll (1989) pointed out that the DEA input and output should
be selected as follows. The number of decision-making units (DMUs) must be at least more
than twice as much as the number of staged inputs and outputs. In this study, the number of

123
1954 C.-C. Chang

Table 2 Descriptive statistics for the 41 countries

Input/output factors Minimum Maximum Mean Std. Dev. Valid N

TERD (percentage of GDP) 0.10 3.47 1.51 0.91 41

BERD (US$ millions) 38 265,193 16, 529.59 44, 768.37 41
EDU (percentage of GDP) 2.3 7.6 4.85 1.29 41
RDPN (FTE per 1,000 people) 0.16 1.65 4.57 2.72 41
RDPB (FTE per 1,000 people) 0.06 7.71 2.42 1.95 41
TR (scale 0–10, survey data) 4.49 8.50 6.19 1.02 41
SA (number) 137 208,601 17, 435.34 34, 718.77 41
PGTR (number) 18 167,820 11, 694.83 31, 326.28 41
BR (scale 0–10, survey data) 3.34 8.50 5.76 1.32 41
IBS (kbps) 1.29 9043.06 258.58 1, 406.80 41
IUSER (users per 1,000 people) 129 824 609.07 209.97 41
KT (scale 0–10, survey data) 2.63 7.01 4.75 1.26 41
TC (scale 0–10, survey data) 3.03 7.83 5.64 1.17 41
IC (scale 0–10, survey data) 3.14 8.02 5.59 1.32 41
GDP (US$ billions) 19.20 14,660.40 1, 293.64 2, 525.28 41
HTEX (US$ millions) 1.93 67.82 17.04 13.80 41

DMU was 41, and the highest number of inputs and outputs was (6 + 3) in the first stage. As
41 > 2∗ (6+3), this met the requirements, as it was more than three times the requirements.
For example, Bowlin (1998) suggested that the number of DMUs should be at least triple to
the number of inputs and outputs considered to get good discriminatory power. In addition,
in this study, there was also a positive relationship between the inputs and outputs selected
as per the verification of the Pearson correlation coefficient, which was also in line with the
regulation that the DEA input-output data must have an isotonic relationship, that is, the
increase of an input should not reduce an output.

3.3 Data source

During the DEA performance evaluation, the evaluated units (DMU) should be comparatively
homogeneous (Golany and Roll 1989). Hence, in this study, the national level was taken as
the research object, which not only was homogeneous but also had comparative signifi-
cance. The research samples mainly included OECD members, but they also included BRIC
countries such as China, Brazil and Russia, etc. After deducting the countries with incom-
plete data, there were a total of 41 countries, namely: Australia, Austria, Belgium, Brazil,
Bulgaria, Canada, China, Croatia, the Czech Republic, Denmark, Estonia, Finland, France,
Germany, Greece, Ireland, Italy, Japan, Korea, Lithuania, Luxembourg, Malaysia, Mexico,
Netherlands, New Zealand, the Philippines, Poland, Portugal, Romania, Russia, Singapore,
Slovakia, Slovenia, Spain, Switzerland, Taiwan, Thailand, Turkey, Ukraine, the U.K. and the
USA. Variable data sources included the World Competitiveness Yearbook (WCY) issued
by the Institute for Management Development IMD ( 2007–2012) between 2007 ∼ 2012,
the World Intellectual Property Organization (WIPO), the International Telecommunication
Union (ITU) and the Computer Industry Almanac Inc. (CIA Inc.). Table 2 presents descriptive
statistics for our dataset.

123
Influences of knowledge spillover and utilization 1955

3.4 Network DEA

DEA is an analytical technique for measuring the relative efficiency of organizational or pro-
duction units. It has been widely acknowledged for its strength of (1) capturing multiple inputs
and outputs, (2) combining multiple performance dimensions, and (3) computing perfor-
mance measures that integrate data/information across multiple dimensions and input/output
resources (Gattoufi et al. 2004). DEA identifies the empirical frontier that is constituted by
the best practice units, and gives an inefficiency/eficiency score to each individual DMU.
Since the innovative work by Charnes et al. 1978, DEA has been used in many applications,
e.g. education, agriculture, healthcare, and banking industry (Cheng et al. 2007; O’Neill et
al. 2008). Meanwhile, the Network DEA (Färe and Grosskopf 2000) identifies inefficiency
sources embedded in the interactions among various components of the evaluated unit. This
approach allows one to further investigate the structure and processes inside the DMU, to
identify the misallocation of inputs among sub-processes and generate insights about the
sources of inefficiency within the DMU. Consequently, this approach allows one to investi-
gate what is transpiring within the “black box” input/output representation of the traditional
DEA analysis.
In DEA, excesses in inputs and shortfalls in outputs are called slacks. Hereafter, the slacks
based Network DEA model will be referred as the SBMN model. The slacks-based measure
(SBM) of efficiency has resulted from the additive models (Cooper et al. 2000). Expanding
on the basic SBM model proposed by Tone Tone (2001), Tone and Tsutsui (2009) devel-
oped a slacks-based Network DEA model. Färe and Grosskopf (2000) use radial efficiency
measurement in their Network DEA model. In contrast, the SBM model uses a non-radial
approach. The radial approach assumes proportionate reduction/increase in inputs/outputs.
The SBM model relaxes the proportionate change assumption and aims at obtaining max-
imum rate of reduction/increase in inputs/outputs. Therefore, the SBM model captures the
non-radial slacks directly. The fractional program of non-oriented, variable returns to scale
Network DEA has been proposed by Tone and Tsutsui (2009). We employ the following
notations to describe Network SBM:

n: The number of DMUs

K: The number of divisions
mk : The number of inputs to Division k
rk : The number of outputs from Division k
D: The set of divisions in the model. The divisions are numbered from 1 to K.
S: The set of divisions which have no incoming links, i.e. starting divisions
T: The set of divisions which have no outgoing links, i.e. terminal divisions
L: The set of links
(k,h): The link from Division k to Division h
t(k,h) : The number of items in Link (k,h)
mk
x kj ∈ R+ : Input resources to DMU j at Division k (k = 1, . . . , K )
rk
y kj ∈ R+ : Output products from
DMU j at Division k (k = 1, . . . , K )
(k,h) t
zj ∈ R+(k,h) : Linking input resources to DMU j at Division h from Division k ((k, h) ∈ L)
= Linking output products from D MU j at Division k to Division h ((k, h) ∈ L)

The production possibility set {(x k y k z (k,h) )} is defined by

n
xk ≥ x kj λkj (k = 1, . . . ., K)
j=1

123
1956 C.-C. Chang

n
yk ≥ y k λk (k = 1, . . . ., K)
j=1 j j
n (k,h) h
z (k,h) = z λ j (∀(k, h))(as inputs to h)
j=1 j
n (k,h) h
z (k,h) = z λ j (∀(k, h))(as outputs from k)
j=1 j
n
λkj = 1(∀k), λkj ≥ 0(∀ jk)
j=1

where λk ∈ R+ n is the intensity vector corresponding to Division k(k = 1, . . . K),

D MU o (o = 1, . . . , n) can be represented by
xok = X k λk + sok− (k = 1, . . . , K )
yok = Y k λk − sok+ (k = 1, . . . , K )
eλk = 1(k = 1, . . . , K )
λk ≥ 0, sok− ≥ 0, sok+ ≥ 0, (∀k)
Where
X k = (x1k , . . . , xnk ) ∈ R m k ×n
Y k = (y1k , . . . , ynk ) ∈ R rk ×n
As regard to the linking constraints, we have several options of which we present two possible
cases.
(a) The “fixed” link value case.
The linking activities are kept unchanged:
z o(k,h) = Z (k,h) λh (∀(k, h))
z o(k,h) = Z (k,h) λk (∀(k, h))
(b) The “free” link value case.
The linking activities are freely determined while keeping continuity between input and
output:
Z (k,h) λh = Z (k,h) λk (∀(k, h))
where
(k,h)
Z (k,h) = (z 1 , . . . , z n(k,h) ) ∈ R t(k,h) ×n
We employ the following objective function for Non-Oriented case.
  
K m k siok−
k=1 w 1 − mk
k 1
i=1 x k
ρo∗ = min 
io
  k+


K r s
k=1 w 1 + rk
k 1 k ro
r =1 x k ro

In this case we define the efficiency score of Division k by

  
m k siok−
1 − mk
1
i=1 x k
io
ρk =   k+


rk sro
1 + rk
1
r =1 x k ro

Thus, the overall efficiency is neither the arithmetic nor the harmonic mean of divisional
efficiencies.

123
Influences of knowledge spillover and utilization 1957

4 Empirical results

This study evaluated the innovation system efficiency of 41 countries around the world,
including OECD member countries, China and Brazil, using DEA Solver software. First
of all, the overall efficiency and staged efficiency of the innovation system were obtained
through Network SBM performance measurement model, then the observation samples were
grouped in combination with cluster analysis and multidimensional scaling analysis, so that
the efficiency analysis would be more specific. The benchmarking objects were analyzed
to provide various countries with suggestions on performance improvements. Finally, the
influence of inputs in various stages on the relative efficiency was respectively investigated
through sensitivity analysis using the super-efficiency model.

4.1 Efficiency analysis

This study measured the R&D efficiency, diffusion efficiency and economic efficiency of the
knowledge innovation programs of the NISs according to the data from 2007 to 2010 and
using the Network SBM performance measurement model, in order to understand the links
of economic activities within a NIS. The results of the analysis are shown in Table 3.
The NISs in Austria, China, Finland, Netherlands, Switzerland and Japan had higher
overall efficiency or staged efficiency than the other countries, suggesting that the input
resource of these six countries in each stage fully exerted their roles and created the best
possible output under the conditions. In other words, in terms of the creation, diffusion and
transformation activities of knowledge technology, these countries could set a benchmark
example to other countries in respect of decision management, connection and development.
For example, the 10th five-year plan promoted in China in 2001 proposed that the driving
force of economic development mainly comes from three major internal factors, namely,
structural adjustment, reform and opening-up, and technological progress.
In order to further understand the performance of economic activities within the NIS, the
R&D efficiency, diffusion efficiency and economic efficiency were respectively analyzed. The
average R&D efficiency in this stage was 0.645, and the efficiency was one in eight countries,
which indicated that 33 countries (about 80 %) still had room for improvement. For example,
in respect of patent applications, it was necessary to develop valuable key technologies and
patents, strengthen patent layouts and accelerate the creation of intellectual property. In
respect of the publication of dissertations in periodicals, it was necessary to deeply develop
academic researches such as technical reports and dissertations in periodicals, in order to show
the achievement degree of researches and improve the quality of basic researches. In other
words, the innovation system of various countries should actively and deeply investigate key
niche technologies required in the industry, patent layout, intellectual property right creation
and academic research in universities, and effectively implement industrial applications of
technical achievements, thereby driving a rush of industrial upgrading or transformation and
automatically improving the contribution and value of R&D efficiency.
The average diffusion efficiency in this stage was 0.608. The efficiency was lower than
1 (about 68 %) in 28 countries and was low in the majority of countries, showing there
was much room for improvement. In terms of the network environment, these countries
should strengthen their basic network equipment, improve the network bandwidth speed and
reduce the use cost, so as to popularize the diffusion platform. In terms of the knowledge
transfer, they should provide the industries with research findings through holding activities
such as technical seminars, training workshops and technical release conferences, so as to
help manufacturers improve their technical ability and create technical value from the R&D

123
1958 C.-C. Chang

achievements. In terms of technical cooperation, they should actively use existing technical
services, international standard certification and technical exchange platform mechanisms,
and provide industries with consultation services such as technical consultations, repair and
maintenance, in order to promote the upgrading of industrial technology. During this stage,
they may establish the mechanisms and strategies such as technology transfers, patent licens-
ing and industrial services through strengthening the diffusion effect of the R&D achieve-
ments, mechanisms and strategies to industrial applications, thereby helping manufacturers
find their survival niche and driving development of the overall industry, in order to highlight
the knowledge spillover achievements of the diffusion efficiency.
The average economic efficiency of knowledge was 0.633 in this stage. The efficiency
was lower than 1 (about 80 %) in 33 countries, showing there was room for improvement.
In terms of high-tech exports, these countries should diffuse technological achievements to
industrial applications or new product development through the strategies such as technology
transfer, patent licensing and industry-university cooperation. This could bring the substantial
effect of directly facilitating or indirectly converting derived knowledge and technology into
economic income, such as the increase of output value and export volume of high-tech
products, which could drive the industry to develop towards high added value. In addition,
they should also attach importance to fostering the enterprise innovation capacity, so as to
improve the successful commercialization rate of new technology transfer products. In other
words, in this stage, they may actively use existing industrial alliances, technical exchange
platforms and technical cooperation mechanism through enhancing the industrial value gain,
so as to enhance mutual innovation energy and create new industrial niche and profitable
value.
Table 3 shows that the average scores of R&D efficiency economic efficiency and diffu-
sion efficiency are 0.645, 0.633 and 0.608, which indicates that in general, although the R&D
efficiency of countries was the best the three were not greatly different. The NISs of each
country try to be engaged in the output of knowledge and technology, but they are generally
unable to be effectively popularized to related organizations such as industries, governments,
universities and research institutions in the knowledge diffusion stage; hence, they can nei-
ther motivate more possible knowledge and technology achievements through exchange and
diffusion nor enhance the innovation benefit of new knowledge and technology. Therefore,
various countries are suggested to expand the quality (such as fast network environments) and
quantity (such as popular knowledge network access points and network platform database
etc.) of relevant knowledge exchange platforms, in order to allow the masses and institu-
tions to efficiently diffuse and receive new knowledge and technology at any time, as well as
strengthen the technical cooperation schemes and platforms for various parties including the
media, industries, governments, universities and research institutions. In this way the overall
knowledge innovation efficiency of NISs could be effectively enhanced.

4.2 Multivariate analysis and benchmarking

Multivariate analysis, including cluster analysis and multidimensional scaling, is utilized

to determine the contributions of each stage to the overall efficiency of NIS. In order to
further understand the common features and differences of efficiency in each stage, this
study combined cluster analysis and multidimensional scaling analysis, so as to allow the
efficiency analysis to be more specific, allow various countries to more clearly understand
the benchmarking objective and improvement direction, and improve the national innovation
performance.

123
Influences of knowledge spillover and utilization 1959

Table 3 Overall efficiency and efficiency decomposition

No. Country Overall efficiency∗ Three-stage efficiency

R&D-stage Diffusion-stage Economic-stage

1 Australia 0.881 0.846 1.000 0.808

2 Austria 1.000 1.000 1.000 1.000
3 Belgium 0.299 0.342 0.223 0.351
4 Brazil 0.454 0.561 0.318 0.523
5 Bulgaria 0.227 0.358 0.104 0.314
6 Canada 0.577 0.528 0.598 0.608
7 China 1.000 1.000 1.000 1.000
8 Croatia 0.531 0.634 0.434 0.544
9 Czech 0.423 0.393 0.405 0.474
10 Denmark 0.677 0.445 0.997 0.699
11 Estonia 0.332 0.768 0.116 0.409
12 Finland 1.000 1.000 1.000 1.000
13 France 0.304 0.295 0.167 0.571
14 Greece 0.361 0.344 0.196 0.698
15 Germany 0.828 1.000 1.000 0.568
16 Ireland 0.911 1.000 1.000 0.757
17 Italy 0.494 0.516 0.247 0.948
18 Japan 0.998 0.996 1.000 0.998
19 Korea 0.814 0.995 1.000 0.542
20 Lithuania 0.764 1.000 1.000 0.446
21 Luxembourg 0.541 0.479 0.997 0.332
22 Malaysia 0.323 0.483 0.155 0.451
23 Mexico 0.305 0.394 0.144 0.502
24 Netherlands 1.000 1.000 1.000 1.000
25 New Zealand 0.610 0.626 1.000 0.363
26 Philippines 0.312 0.504 0.193 0.312
27 Poland 0.326 0.322 0.256 0.421
28 Portugal 0.390 0.362 0.534 0.306
29 Romania 0.493 0.724 0.300 0.552
30 Russia 0.572 0.529 0.572 0.619
31 Singapore 0.793 0.784 0.757 0.841
32 Slovak 0.563 0.748 0.530 0.451
33 Slovenia 0.544 0.532 0.855 0.353
34 Spain 0.463 0.421 0.236 1.000
35 Switzerland 1.000 1.000 1.000 1.000
36 Taiwan 0.557 0.561 0.461 0.667
37 Thailand 0.301 0.316 0.169 0.510
38 Turkey 0.619 0.600 0.396 1.000
39 Ukraine 0.545 0.586 0.869 0.318

123
1960 C.-C. Chang

Table 3 continued
No. Country Overall efficiency∗ Three-stage efficiency

R&D-stage Diffusion-stage Economic-stage

40 U.K. 0.883 1.000 1.000 0.689

41 USA 0.686 0.456 0.709 1.000
Average score 0.602 0.645 0.608 0.633

∗ Overall efficiency = Geometric mean (R&D eff. x Diffusion eff. x Economic eff.)

Cluster analysis was provided to classify the observed values of the samples (Johnson and
Wichern 2007) and effectively understand the characteristics of each group. Some charac-
teristics were different between groups (Sharma 1996). As a result, this study divided the
hierarchical tree results into four groups through cluster analysis of the staged efficiency, such
as the creation efficiency, the diffusion efficiency and the economic efficiency of the inno-
vation systems of the aforementioned 41 countries according to the characteristics of cluster
analysis. According to the characteristics and difference analysis of each group, this study
respectively named the four groups as the leading knowledge innovation group (including the
benchmarking group), the knowledge innovation diffusion group, the knowledge innovation
application group and the knowledge innovation catching-up group. In addition, in order
to further understand the hidden characteristic structure behind the 41 countries, this study
visualized the efficiency of the NISs of three stages to a two-dimensional space structure
diagram using multidimensional scaling analysis (Torgerson 1952). The four groups were
drawn (as shown in Fig. 3) by integrating the grouping results of the cluster analysis, and the
stress coefficient of 0.08944 and the RSQ (R Square) of 0.96801 were obtained based on the
two-dimensional analysis (X, Y axis). According to the interpretation of Kruskal (1964), the
stress coefficient and the RSQ are indexes for the degree of adaptability. The lower the stress
coefficient is, the higher the RSQ will be, and the more probable that the obtained struc-
tural diagram can represent the original data. The stress coefficient obtained in this study
was 0.050 ∼ 0.100, and was in line with the requirements of the degree of adaptability in
multidimensional scaling analysis.
This study established a benchmarking group, as shown in Table 4, through grouping
the DMUs based on the efficiency in each stage, in order to provide a reference for vari-
ous countries in performance improvement and resource allocation. The leading knowledge
innovation group included 13 countries, all of which had better efficiency in each stage. The
countries with efficiency in all three stages approaching 1.000 were regarded as the bench-
marking group, which covered six countries, including Austria, China, Finland, Netherlands,
Switzerland and Japan
In addition, the countries with a diffusion efficiency higher than the economic efficiency
and R&D efficiency were classified into the knowledge innovation diffusion group, which
also covers six countries, including Denmark. In contrast, the countries with an economic
efficiency higher than the diffusion efficiency and R&D efficiency were classified into the
knowledge innovation application group, which covered four countries, including Spain. The
remaining countries generally had low efficiency in each stage. These countries were classi-
fied into the knowledge innovation catching-up group, which covered a total of 19 countries.
The countries in the knowledge innovation diffusion group attached more importance to the
diffusion and spillover activities of technical knowledge, while the countries in the technical
knowledge application group attached more importance to the transformation and utilization

123
Influences of knowledge spillover and utilization 1961

Fig. 3 Multidimensional analysis of NIS efficiency

of technical knowledge and attached less importance to the R&D, innovation and diffusion
of technical knowledge. Therefore, the overall efficiency of the NIS could not be effectively
promoted due to respective bias.
In order to further understand whether the above grouping was distinguishable and could
show the common features and differences of the innovation efficiency of each country in each
stage, the Kruskal-Wallis test of non-parametric statistical methods was used in this study for
verification (Brockett and Golany 1996). At the significance level of 0.001, the test results
were all significantly different, showing that this grouping method was distinguishable, as
shown in Table 5

4.3 Sensitivity analysis

Sensitivity analysis was provided to investigate the influence of the increase or decrease in
the number of input or output variables, as well as the change of the number of evaluated units
on the relative efficiency. This study investigated the efficiency changes in each stage through
reducing the number of input variables (reducing one input variable each time), namely, by
investigating the influence of an input variable on the overall efficiency by removing it, so as
to understand the relative importance of each input variable on the overall efficiency in each
stage, in order to provide a reference for various countries to improve efficiency.
In addition, the relative efficiency of the DMU was 1 in the efficient frontier, meaning that
the efficiency changes of these efficient DMUs could fail to be measured during sensitivity
analysis. Therefore, this study implemented sensitivity analysis using the super efficiency
model proposed by Anderson and Petersen (1993), in order to solve the problem that relative
efficiency changes of the DMU could not be measured.

123
1962 C.-C. Chang

Table 4 The three-stage efficiency of four groups

Groups No. Country R&D efficiency Diffusion efficiency Economic efficiency

Leading group 1 Australia 0.846 1.000 0.808

2 Austria 1.000 1.000 1.000
7 China 1.000 1.000 1.000
12 Finland 1.000 1.000 1.000
15 Germany 1.000 1.000 0.568
16 Ireland 1.000 1.000 0.757
18 Japan 0.996 1.000 0.998
19 Korea 0.995 1.000 0.542
20 Lithuania 1.000 1.000 0.446
24 Netherlands 1.000 1.000 1.000
31 Singapore 0.784 0.757 0.841
35 Switzerland 1.000 1.000 1.000
40 U.K. 1.000 1.000 0.689
Average 0.971 0.981 0.819
Diffusion group 10 Denmark 0.445 0.997 0.699
21 Luxembourg 0.479 0.997 0.332
25 New Zealand 0.626 1.000 0.363
33 Slovenia 0.532 0.855 0.353
39 Ukraine 0.586 0.869 0.318
Average 0.533 0.943 0.413
Application group 17 Italy 0.516 0.247 0.948
34 Spain 0.421 0.236 1.000
38 Turkey 0.600 0.396 1.000
41 USA 0.456 0.709 1.000
Average 0.498 0.397 0.987
Catching-up group 3 Belgium 0.342 0.223 0.351
4 Brazil 0.561 0.318 0.523
5 Bulgaria 0.358 0.104 0.314
6 Canada 0.528 0.598 0.608
8 Croatia 0.634 0.434 0.544
9 Czech 0.393 0.405 0.474
11 Estonia 0.768 0.116 0.409
13 France 0.295 0.167 0.571
14 Greece 0.344 0.196 0.698
22 Malaysia 0.483 0.155 0.451
23 Mexico 0.394 0.144 0.502
26 Philippines 0.504 0.193 0.312
27 Poland 0.322 0.256 0.421
28 Portugal 0.362 0.534 0.306
29 Romania 0.724 0.300 0.552
30 Russia 0.529 0.572 0.619
32 Slovak 0.748 0.53 0.451

123
Influences of knowledge spillover and utilization 1963

Table 4 continued
Groups No. Country R&D efficiency Diffusion efficiency Economic efficiency

36 Taiwan 0.561 0.461 0.667

37 Tailand 0.316 0.169 0.51
Average 0.482 0.309 0.489

Bold values signify the average staged efficiency score is above 0.85
Table 5 Kruskal-Wallis test of non-parametric statistical methods

Groups Numbers Average Kruskal-Wallis Test (P-Value)

R&D efficiency Leading group 13 0.971 26.730***

Diffusion group 5 0.533
Application group 4 0.498
Catching-up group 19 0.482
Diffusion efficiency Leading group 13 0.981 32.085***
Diffusion group 5 0.944
Application group 4 0.397
Catching-up group 19 0.309
Economic efficiency Leading group 13 0.819 22.733***
Diffusion group 5 0.413
Application group 4 0.987
Catching-up group 19 0.489

** α =0.01; *** α =0.001

As can be seen from the sensitivity analysis results (see Table 6), in the R&D efficiency
of the first stage, the R&D expenditure of the enterprises (BERD) was the most sensitive
followed by the technical regulations (TR), suggesting that the R&D expenditure of private
enterprises has the most important influence on the R&D efficiency of NIS. In the diffu-
sion efficiency of the second stage, network bandwidth speed (IBS) was the most sensitive,
suggesting that the input degree and quality of a country in its information dissemination
infrastructure has the most direct influence on the knowledge diffusion efficiency. The number
of published dissertations in scientific periodicals (SA) was more sensitive than the residents
with granted patents (GERD), suggesting that a country should attach more importance to
publishing dissertations in periodicals instead of excessively focusing on patent protection,
so as to facilitate knowledge diffusion and popularization. Finally, in the staged economic
efficiency, the innovation capacity of enterprises (IC) was the most sensitive, suggesting
that in a knowledge economy, enterprises are still the subjects converting knowledge into
commercial operation, and that national leaders should create favorable policies and envi-
ronments as well as cultivate and strengthen the innovation capacity of enterprises, in order
to effectively commercialize the existing knowledge and technology and promote national
economic development and achievements.

5 Conclusions

The operation of a NIS is multidimensional and multistage, and its operation performance has
a critical influence on national economic development and competitiveness. However, since

123
1964 C.-C. Chang

Table 6 Sensitivity analysis with super efficiency model

First stage Rank Second stage Rank Third stage Rank

R&D eff. Diffusion eff. Economic eff.

Original efficiency 0.6631 1.008 0.2906

Removing inputs TERD 0.6489 5
BERD 0.5075 1
EDU 0.6333 4
RDPN 0.6278 3
RDPB 0.6687 6
TR 0.5853 2
SA 0.9392 3
PGTR 0.9669 4
BR 0.8391 2
IBS 0.6353 1
KT 0.2740 2
TC 0.2746 3
IC 0.2549 1

traditional performance analysis focuses on the overall efficiency of the system’s input and
output, it neglects the context and connection of various activities at different stages within the
system. Moreover, since a certain length of time is required regarding the creation, diffusion
and transformation of knowledge activities, a time lag between input and output needs to be
taken into account in conducting the evaluation of NIS efforts. Mindful of this, this study
established a longitudinal and multi-stage network performance measurement model of the
NIS and advocated that, in order to clearly evaluate the performance of a NIS, it is necessary
to consider not only the R&D efficiency in the knowledge and technology output stage, but
also the diffusion efficiency in the knowledge spillover stage and the economic efficiency
in the conversion and application stage, so as to more integrally present the management-
related implications and information. This section collects the results of empirical analyses,
proposes the research conclusions, shows the limitations and provides suggestions for future
research.
Firstly, the results show that the innovation systems in Austria, China, Finland, the Nether-
lands, Switzerland and Japan have higher overall efficiency and staged efficiency than those
in other countries, showing that the input resource of these six countries at each stage fully
exerts its role and creates the best output under possible conditions. In other words, in terms
of the creation, diffusion and transformation activities of knowledge and technology, these
countries can be set as benchmark examples to other countries in respect of decision man-
agement, connection and development.
Secondly, in terms of the staged efficiency, the results show that the countries with high
R&D efficiency may not have high diffusion efficiency, and that those with high diffusion
efficiency may not have high economic efficiency. For example, there were 13 countries with
high diffusion efficiency, but only nine countries with high R&D efficiency, and only eight
countries with a high degree of economic efficiency. As a result, while most countries have
the advantages of NIS, they fail to effectively connect them to the activity context of the inno-
vation system. For example, a country that is proficient at technical innovation or production
knowledge may fail to effectively use and diffuse its output; as a result, it will be unable to

123
Influences of knowledge spillover and utilization 1965

achieve the desired economic results. This result confirmed those by Pandza et al. (2011),
and strengthened the arguments made by Guan and Chen (2012) that the overall innovation
efficiency of a NIS is mainly subject to downstream commercial efficiency performance,
and that improving commercial efficiency should thus be a primary consideration in future
innovation policy-making in most OECD countries. This finding and concept correspond
with research by Galanakis (2006)
Thirdly, in order to further understand the common features and differences of efficiency
at each stage, this study combined cluster analysis and multidimensional scaling analysis of
multivariate analysis, and classified the observed values of the samples, respectively, as the
leading knowledge innovation group (including the benchmarking group), the knowledge
innovation diffusion group, the knowledge innovation application group and the knowledge
innovation catching-up group, to allow various countries to better understand the benchmark-
ing objective and direction of improvement. Since the countries in the leading knowledge
innovation group had favorable relative efficiency at each stage, they can set an example to
the other groups.
Fourthly, the sensitivity analysis showed that the R&D expenditure of enterprises (BERD)
has the most significant influence on the R&D efficiency; IBS has the most significant influ-
ence on the diffusion efficiency; and the IC has the most significant influence on economic
efficiency. This shows that the BERD is most important for the R&D efficiency of a NIS. It
also shows why the R&D funds of private enterprises have always been regarded by various
countries as an important index in terms of revitalizing the industry and the economy. In
terms of the diffusion efficiency at the second stage, IBS is the most sensitive factor, showing
that the input degree and quality of a country, in regard to the information dissemination
infrastructure, most directly impacts the knowledge diffusion efficiency.
Fifthly, in terms of economic efficiency, the innovation capacity of enterprises (IC) is
the most sensitive factor, showing that in a knowledge economy, enterprises are converting
knowledge into commercial operation. National leaders should create favorable policies and
environments, to cultivate and strengthen the IC in order to effectively commercialize the
existing knowledge and technology, as well as to promote national economic development
and achievements.
Finally, this study performed analyses using cross-sectional data. An actual NIS will
change over time as environmental factors change; this cross-sectional study and analy-
sis failed to capture the vertical sectional information. Subsequent studies should consider
extending the period of study, not only to investigate and evaluate the changes of the produc-
tion frontier caused by the changes of evaluated units over time through the intertemporal
analysis of different NISs using the Malmquist productivity index in order to calculate the
catching-up in efficiency (CIE), but also to generate a more comprehensive discussion.

Acknowledgments The author gratefully acknowledges the constructive comments and suggestions pro-
vided by the reviewers and editors. The authors would like to thank Lieh-Ting Lan for assistance of the research
This research was supported by the National Science Council of Taiwan under the Grant NSC 101-2410-H-606
-011.

References

Anderson, P., Petersen, N.C.: A procedure for ranking units in data envelopment analysis. Manage. Sci. 39,
1261–1264 (1993)
APEC.: Towards Knowledge Based Economies in APEC. APEC Economic Committee, Singapore (2000)

123
1966 C.-C. Chang

Archibugi, D.: Patenting as an Indicators of Technological Innovation, a Review. Sci. Pub. Policy 19, 357–368
(1992)
Archibugi, D., Pianta, M.: Measuring technological change through patents and innovation surveys. Techno-
vation 19, 451–468 (1996)
Balzat, M., Hanusch, H.: Recent trends in the research on national innovation systems. J. Evolut. Econ. 14,
197–210 (2004)
Basberg, B.L.: Patents and the measurement of technological change: a survey of the literature. Res. Policy
16, 131–141 (1987)
Bowlin, W.F.: Measuring performance: an introduction to data envelopment analysis (DEA). J. Cost Anal. 7,
3–27 (1998)
Brockett, P., Golany, B.: Using rank statistics for determining programmatic efficiency differences in data
envelopment analysis. Manage. Sci. 42(3), 466–472 (1996)
Cantwell, J., Fai, F.: Firms as the source of innovation and growth: the evolution of technological competence.
J. Evolut. Econ. 9(3), 331–366 (1999)
Charnes, A., Cooper, W.W., Rhodes, E.: Measuring the efficiency of decision making units. Eur. J. Op. Res.
2(6), 429–444 (1978)
Cheng, E.W.L., Chiang, Y.H., Tang, B.S.: Alternative approach to credit scoring by DEA: evaluating borrowers
with respect to PFI projects. Build. Environ. 42(4), 1752–1760 (2007)
Cooke, P., Gomez Uranga, M.: Regional innovation systems: institutional and organisational dimensions. Res.
Policy 26(4), 475–491 (1997)
Cooper, W.W., Seiford, L.M., Tone, K.: Data Envelopment Analysis: A Comprehensive Text with Models,
Applications, References and DEA-Solver Software. Kluwer Academic Publishers, Boston (2000)
Coriat, B., Weinstein, O.: Organizations, firms and institutions in the generation of innovation. Res. Policy
31(2), 273–290 (2002)
Cullmann, A., Schmidt-Ehmcke, J., & Zloczysti, P. (2009). Innovation, R&D Efficiency and the Impact of
the Regulatory Environment: A Two Stage Semi-Parametric DEA Approach: Discussion papers/German
Institute for Economic Research.
Dosi, G.: Sources, procedures, and microeconomic effects of innovation. J. Econ. Lit. 26(3), 1120–1171 (1988)
Edquist, C.: Systems of innovation: perspectives and challenges. In: Fagerberg, J., Mowery, D., Nelson, R.
(eds.) The Oxford Handbook of Innovation, pp. 181–208. Oxford University Press, Oxford (2004)
Etzkowitz, H., Leydesdorff, L.: The dynamics of innovation: from national systems and “Mode 2” to a triple
helix of University–Industry–Government relations. Res. Policy 29(2), 109–123 (2000)
Färe, R., Grosskopf, S.: Network DEA. Socio-Economic Planning Sciences 34, 35–49 (2000)
Freeman, C.: Technology Policy and Economic Performance: Lessons from Japan. Pinter, London (1987)
Galanakis, K.: Innovation process. Make sense using systems thinking. Technovation 26(11), 1222–1232
(2006)
Gattoufi, S., Oral, M., Kumar, A., Reisman, A.: Content analysis of data envelopment analysis literature and
its comparison with that of other OR/MS fields. J. Op. Res. Soc. 55, 911–935 (2004)
Golany, B., Roll, Y.: An application procedure for DEA. Omega 17, 237–250 (1989)
Griliches, Z.: Patent statistics as economic indicators: a survey. J. Econ. Lit. 28, 1661–1797 (1990)
Guan, J., Chen, K.: Modeling the relative efficiency of national innovation systems. Res. Policy 41(1), 102–115
(2012)
IMD.: IMD world competitiveness yearbook. IMD, Lausanne (2007–2012)
Intarakumnerd, P.: Thaksin’s legacy: thaksinomics and its impact on Thailand’s national innovation system
and industrial upgrading. Int. J. Inst. Econ. 3(1), 31–60 (2011)
Johnson, R.A., Wichern, D.W.: Applied Multivariate Statistical Analysis, 6th edn. Pearson International Edi-
tion, London (2007)
Kaufmann, A., Tödtling, F.: Science—industry interaction in the process of innovation: the importance of
boundary-crossing between systems. Res. Policy 30, 791–804 (2001)
Kitanovic, J.: The applicability of the concept of national innovation systems to transition economies. INNO-
VATION Manage. Policy Pract. 9(1), 28–45 (2007)
Kline, S.J., Rosenberg, N.: An overview on innovation. In: Landau, R., Rosenberg, N. (eds.) The Positive Sum
Strategy: Harnessing Technology for Economic Growth. National Academy Press, Washington, Strategy
(1986)
Kneller, R., Manderson, E.: Environmental regulations and innovation activity in UK manufacturing industries.
Resour. Energy Econ. 34, 211–235 (2012)
Kravchenko, N.: The problem of measuring and assessing national innovation systems. Probl. Econ. Trans.
53(9), 61–73 (2011)
Kruskal, J.B.: Nonmetric multidimensional scaling: a numerical method. Psychometrika 29(2), 115–129
(1964)

123
Influences of knowledge spillover and utilization 1967

Lai, H.C., Shyu, J.Z.: Using data envelopment analysis to evaluate performance of national innovation systems:
the case of major OECD member. Chung Yuan Manage. Rev. 1(2), 221–238 (2003)
Lumpkin, G.T., Dess, G.G.: Clarifying the entrepreneurial orientation construct and linking it to performance.
Acad. Manage. Rev. 21(1), 135–172 (1996)
Lundvall, B.Å.: National systems of innovation: Towards a Theory of Innovation and Interactive Learning.
Pinter, London (1992)
Lundvall, B.A.: National innovation systems—analytical concept and development tool. Ind. Innov. 14(1),
95–119 (2007)
Malerba, F.: Sectoral Systems of Innovation: Concepts, Issues and Analyses of Six Major Sectors in Europe.
Cambridge University Press, Cambridge (2004)
Marxt, C., Brunner, C.: Analyzing and improving the national innovation system of highly developed
countries—The case of Switzerland. Technol. Forecast. Soc. Change 80(6), 1035–1049 (2013)
Metcalfe, S.: The economic foundations of technology policy: equilibrium and evolutionary perspectives.
In: Stoneman, P. (ed.) Handbook of the Economics of Innovation and Technological Change. Blackwell
Publishers, Oxford (UK)/Cambridge (US) (1995)
Motohashi, K.: University-industry collaborations in Japan: the role of new technology-based firms in trans-
forming the National Innovation System. Res. Policy 34(5), 583–594 (2005)
Mowery, D.C.: Nanotechnology and the US national innovation system: continuity and change. J. Technol.
Transf. 36(6), 697–711 (2011)
Nasierowski, W., Arcelus, F.: On the efficiency of national innovation systems. Socio Econ. Plan. Sci. 37(3),
215–234 (2003)
Nelson, R.R.: National Systems of Innovation: A Comparative Analysis. Oxford University Press, Oxford
(1993)
Nelson, R.R., Nelson, K.: Technology, institutions, and innovation systems. Res. Policy 31(2), 265–272 (2002)
Niosi, J.: National systems of innovations are x-efficient (and x-effective): why some are slow learners. Res.
Policy 31(2), 291–302 (2002)
OECD.: The Measurement of Scientific and Technological Activities Using Patent Data as Science and Tech-
nology Indicators (Patent Manual). OECD, Paris (1994)
OECD.: The Knowledge-Based Economy. OECD, Paris (1996)
OECD.: Managing National Innovation Systems. OECD, Paris (1999)
OECD.: Innovation and Productivity in Services. OECD, Paris (2001)
O’Neill, L., Rauner, M., Heidenberger, K., Kraus, M.: A cross-national comparison and taxonomy of DEA-
based hospital efficiency studies. Socio Econ. Plan. Sci. 42(3), 158–189 (2008)
Pan, T.W., Hung, S.W., Lu, W.M.: DEA Performance measurement of the national innovation system in Asia
and Europe. Asia-Pacific Journal of operational research 27, 369–392 (2010)
Pandza, K., Wilkins, T.A., Alfoldi, E.A.: Collaborative diversity in a nanotechnology innovation system:
evidence from the EU framework programme. Technovation 31(9), 476–489 (2011)
Park, K.H., Lee, K.: Linking the Technological regime to the technological catch-up: analyzing Korea and
Taiwan using the US patent data. Ind. Corp. Change 15(4), 715–753 (2006)
Pavitt, K.: R&D patenting and innovative activities: a statistical exploration. Res. Policy 11, 33–51 (1982)
Pavitt, K.: Patent statistics as indicators of innovative activities: possibilities and problems. Scientometrics
7(1–2), 77–99 (1985)
Porter, M.E.: The competitive advantage of nations. The Free Press, NY (1998)
Schmookler, J.: Invention and Economic Growth. Harvard UP, Cambridge (1966)
Schumpeter, J.A.: Theory of economic development. Harvard University Press, Cambridge, MA (1934)
Schumpeter, J.A.: Capitalism, Socialism, and Democracy. Harper & Brothers, New York (1942)
Sharma, S.: Applied Multivariate Techniques. Wiley, New York, USA (1996)
Tone, K.: A slacks-based measure of efficiency in data envelopment analysis. Eur. J. Op. Res. 130(3), 498–509
(2001)
Tone, K., Tsutsui, M.: Network DEA: a slacks based measure approach. Eur. J. Op. Res. 197(1), 243–252
(2009)
Torgerson, W.S.: Multidimensional scaling: I theory and method. Psychometrika 17(4), 401–419 (1952)
Wang, E.C., Huang, W.: Relative efficiency of R&D activities: a cross-country study accounting for environ-
mental factors in the DEA approach. Res. Policy 36(2), 260–273 (2007)
Whelan, E., Teigland, R., Donnellan, B., Golden, W.: How internet technologies impact information flows in
R&D: reconsidering the technological gatekeeper. R&D Manage. 40, 400–413 (2010)
Zander, I.: Technological diversification in the multinational corporation-historical evolution and future
prospects. Res. Policy. 26, 209–227 (1997)

123