Scandinavian Journal of Hospitality and Tourism

ISSN: 1502-2250 (Print) 1502-2269 (Online) Journal homepage: https://www.tandfonline.com/loi/sjht20

Measuring innovation in tourism with Community

Innovation Survey: a first step towards a more
valid innovation instruments.

Anne Jørgensen Nordli

To cite this article: Anne Jørgensen Nordli (2017) Measuring innovation in tourism
with Community Innovation Survey: a first step towards a more valid innovation
instruments., Scandinavian Journal of Hospitality and Tourism, 17:4, 423-440, DOI:
10.1080/15022250.2016.1247382

To link to this article: https://doi.org/10.1080/15022250.2016.1247382

Published online: 26 Oct 2016.

Submit your article to this journal

Article views: 1104

View related articles

View Crossmark data

Citing articles: 9 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=sjht20
SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM, 2017
VOL. 17, NO. 4, 423–440
https://doi.org/10.1080/15022250.2016.1247382

Measuring innovation in tourism with Community Innovation

Survey: a first step towards a more valid innovation
instruments.
Anne Jørgensen Nordli
Faculty of Economic and Organization Science, Lillehammer University College, Lillehammer, Norway

ABSTRACT ARTICLE HISTORY

Researchers require more innovation surveys in tourism, but have Received 25 April 2016
pointed out the deficient quantitative instruments used to Accepted 9 October 2016
measure this innovation. They have questioned whether hidden
KEYWORDS
innovation may explain the low innovation rates in the tourism Innovation in tourism;
industry. Two tourism surveys have been conducted in Norway community innovation
recently. One uses the Community Innovation Survey (CIS) to survey (CIS); “hidden”
measure innovation and the other a modified CIS instrument. The innovation; innovation
surveys produce quite different results. The CIS survey shows low iceberg
innovation rates and the modified survey shows high innovation
rates, which highlights the need for more research on the use of
the CIS in tourism research. The purpose of this study is to explore
and identify how the CIS can be improved and be a more valid
instrument for measuring innovation in tourism. Senior managers
and department managers from different tourism businesses were
interviewed after having completed the CIS survey. The findings
indicate that the CIS is too concerned with R&D and technology,
and does not capture important innovation in relation to service
characteristics. In addition, the terminology and the categorization
of four innovation types might distract and confuse the
respondents. Furthermore, the survey lacks procedures for
capturing all innovations developed at the departmental level
within the companies. Thus, the findings indicate that parts of
significant innovation are hidden. The article contributes with
several suggestions regarding how the CIS can be improved in a
more integrative direction, and concludes that the CIS should not
be dismissed as an instrument for measuring innovation, but
rather should be improved.

Introduction
Tourism is one of the fastest-growing industries globally (Alsos, Eide, & Madsen, 2014).
Therefore, tourism firms’ competitiveness depends on their innovativeness in achieving
lower costs and/or higher quality outputs that meet the demands of potential customers,
and their ability to introduce new products (Sundbo, Orfila-Sintes, & Sørensen, 2007).
However, research-based knowledge on innovation in tourism is deficient, and future
research should focus on both empirical research and the development of theory

CONTACT Anne Jørgensen Nordli anne.nordli@hil.no Faculty of Economic and Organization Science, Lillehammer
University College, Lillehammer, Norway
© 2016 Informa UK Limited, trading as Taylor & Francis Group
424 A. J. NORDLI

(Camisón & Monfort-Mir, 2012; Hall & Allan, 2008; Hjalager, 2010). Hjalager (2010) argues
that the tourism industry should be represented in survey programmes such as the Euro-
pean Community Innovation Survey (CIS), to provide a comprehensive picture of inno-
vation activity in the tourism industry and allow comparisons with other industries. At
the same time, the literature points to measurement bias following the use of scales
such as the CIS to measure innovation in tourism (Camisón & Monfort-Mir, 2012). Biased
and inadequate measurements do not permit comparisons with other industries.
Camisón and Monfort-Mir (2012) argue that low innovation rates may be a result of a
large number of hidden innovations. Furthermore, literature in other low-technology
industries also finds hidden innovation processes (Serin & Hansen, 1997). In a national pol-
itical context, policy-makers need more knowledge about innovation in the tourism indus-
try to assess the need for new ways to support the industry. That may result in adverse
consequences if the results from surveys are used to confirm whether funding for inno-
vation in tourism has had an effect.
Statistics Norway included the tourism industry in the Norwegian part of the CIS for the
first time in 2010. The survey shows low innovation rates in the Norwegian tourism indus-
try (Wilhelmsen & Foyn, 2012). Some years earlier, another national innovation survey was
conducted in Norway, using a modified version of the CIS questionnaire (Rønningen,
2009). This survey finds rather high innovation rates in the tourism industry. Thus, some
studies on innovation in the tourism industry find that the industry is weakly or moderately
innovative (Hjalager, 2010; Mattsson, Sundbo, & Fussing-Jensen, 2005; Sundbo et al., 2007;
Wilhelmsen & Foyn, 2012), while other studies find that the industry is quite innovative
(Fuglsang, Højland, Sundbo, & Sørensen, 2008; Rønningen, 2009; Rønningen & Lien, 2014).
There is no doubt that this field of tourism innovation research, that is, quantitative
measurement, needs further investigation (Hall & Allan, 2008; Hjalager, 2010; Rønningen
& Nordli, in press).
The conceptualization and operational definition of innovation used in the CIS are
provided in the Oslo Manual (OECD, 2005). The CIS has been criticized for favouring
technological innovation and not taking service features into account. Such favouring
of technology innovation may turn out to be unfortunate for an industry such as
tourism, and may explain some of the measurement bias. The CIS also contains a
large amount of technical and professional language that may create problems in
tourism research. If the current approach to tourism innovation research is inadequate,
then the measurements and questionnaires used in quantitative studies may be quite
biased. Even the procedure used to obtain information from firms may be affected by
the firms’ understanding of what constitutes innovation, and the procedure may affect
the innovation rates measured by the questionnaire. These effects will be examined
later.
This paper focuses on whether the CIS manages to measure all innovations
implemented in the tourism industry, and examines whether the CIS provides a reliable
and valid picture of the innovations introduced in tourism businesses. The research is
an explorative study and a first step towards a more systematic methodological analysis
of the CIS and similar surveys. The focus of the study is primarily methodological, and it
identifies possible ways of improving the CIS. It identifies to what extent innovation is cap-
tured by the CIS and whether some elements of innovation are not captured.
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 425

Literature review
CIS and the Oslo Manual
The CIS-based innovation statistics are part of the EU’s science and technology statistics.
Surveys are carried out every two years by EU member states and other member countries
in the European Statistical System. The CIS is a survey of innovation activities within
enterprises.
The Oslo Manual provides guidelines about conducting the CIS. The first edition of the
Oslo Manual was published in 1992, originally with a focus on manufacturing and technol-
ogy. Following the first edition, the second and third editions were published (OECD, 1997,
2005). The second edition was the first to cover services as well as manufacturing;
however, it still focused primarily on manufacturing. The focus on manufacturing made
innovation closely related to technological development and R&D. However, the second
edition of the Oslo Manual suggests that innovation in service-oriented sectors differs sub-
stantially from innovation in many manufacturing industries; however, the third edition
still included the services sector (OECD, 2005). The Oslo Manual became the reference
for various large-scale surveys that examine the nature and the impact of innovation in
the business sector, such as the CIS.
During the development of the Oslo Manual from 1996 to 2005 and the CIS question-
naires used from 2000 to 2010, service features were in some sense taken into account,
which significant changes in the CIS questionnaires since 1996 reveal. Until 2000, the defi-
nition and operationalization of innovation only focused on product and process inno-
vation. During development, adjustments were made, so that organizational innovation
and marketing innovation were given the same attention as product innovation and
process innovation. All four innovation types are measured using several items of the
questionnaire.
In Norway, Statistics Norway (SN) is the organization that collects the CIS data every
second year. The survey conducted for the period 2008−2010 included both manufactur-
ing and services, and also tourism for the first time. This is the survey referred to in the
introduction, which indicated that the Norwegian tourism industry has remarkably low
innovation rates. SN addresses in their report whether the results would have been differ-
ent if enterprises were given more customized questions about innovation specifically for
the tourism industry (Wilhelmsen & Foyn, 2012). The other national survey in Norway on
tourism innovation (Rønningen 2009) and the survey of Fuglsang et al. (2008) are
examples of surveys using the CIS that find high innovation rates. Some studies also indi-
cate that some tourism businesses have had difficulty understanding the CIS questions
(Rønningen, 2009; Rønningen & Nordli, in press). Rønningen (2009) conducted a pilot
study prior to the survey to test CIS items and identifies possible improvements. The
pilot study showed that changes in language and terminology were appropriate; these
changes will be explained in more detail in the discussion section below. This issue justifies
the first research question for the study concerning how well respondents understand the
CIS questions: How well do key persons answering the CIS relate to and understand the ques-
tions and items?
In the existing literature, the requirements and constraints on the respondents who
answer the CIS questionnaire have been given inadequate attention. The questionnaire
426 A. J. NORDLI

was and still is sent to each company’s central management at the main office address.
There are no guidelines relating to how companies gather information about innovation.
It is likely that innovation initiated at the management level will be reported, but it is
unknown whether companies have procedures to capture innovation initiated at the
departmental level. Thus, the second research question is as follows: Do companies have
procedures to capture innovations that are not initiated by or rooted in the senior level man-
agement, especially at the departmental or sectional level?

Defining service innovation

Schumpeter (1934) was the first to define innovation. His general definition states that
innovation is new combinations of existing resources. Schumpeter suggested five types
of innovation: a new good, a new method of production, opening of new markets, new
sources of raw materials and new organization of an industry. The definition within the
Oslo Manual is founded on the Schumpeterian definition, but has been broadened from
goods, to goods and services:
An innovation is the implementation of a new or significantly improved product (good or
service), or process, a new marketing method, or a new organizational method in business
practices, workplace organization or external relationship. (OECD, 2005)

Many definitions of innovation have strong similarities. There must be something new,
at least for the firm, implementation must be central, it must differ from invention and the
innovation must be reproduced and implemented more than once.
Differences are much more prominent when it comes to operationalization of the
concept and specification of types, modes or processes of innovation. This means that
operationalization in regard to methodology and measurement of innovation becomes
very crucial in innovation research.
The Oslo Manual distinguishes among four types of innovation: product innovation,
process innovation, market innovation and organizational innovation. This operationaliza-
tion is criticized by demarcation scientists, who argue that these innovation types overlap
and that they are impossible to separate and are thus reported incorrectly. This is
explained further in the following section, which deals with various approaches to
service innovation research.

Assimilation approach to service innovation research – the CIS

Even though economies in general are being increasingly dominated by services, inno-
vation is still in some research fields associated primarily with manufacturing and techno-
logically advanced characteristics of products (Gallouj & Savona, 2010). As services are
characterized by the supplier–customer relationship, questions arise over whether or
not any innovation occurs within service industries, and if they do, whether the under-
standing of innovation derived from studies of manufacturing is appropriate to service
innovation research. Consequently, there are several competing understandings of how
to study innovation in services, in particular the assimilation, demarcation and integrative
approaches (Coombs & Miles, 2000; Gallouj & Weinstein, 1997; Tether, 2005). The assimila-
tion approach assumes that services and innovation in services are fundamentally similar
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 427

to manufacturing and innovation in manufacturing. It considers innovation in services as

strictly represented by technology-related changes in products (Gallouj & Savona, 2008).
From the assimilation approach, services and innovation in services can be studied by
using or adapting the concepts and tools developed for studying innovation in manufac-
turing (Tether, 2005). The Oslo Manual, the guideline for using the CIS in manufacturing
and services, is an example of this thinking. Because of the tangible nature of manufactur-
ing products, the boundaries among the four types of innovation operationalized in the
Oslo Manual are quite clear. The nature of service products is more complex, and their
characteristics are often more intangible in addition to being more interactive (Tether,
2005). This has led to the development of an alternative position to the assimilation
approach, the “demarcation approach” (Coombs & Miles, 2000; Tether, 2005).

Approaches critical to assimilation

From the demarcation perspective, it is argued that innovation in services and services
outputs has the intrinsic characteristics of being immaterial, interactive and co-productive.
In particular, the interactive and dynamic characteristics of services outputs make the tra-
ditional analytical categories of innovation – product, process and organizational inno-
vation – inadequate and also possibly reductive, as they are not easily separable in the
context of services (Gallouj & Savona, 2008). Because services outputs tend not to have
an independent physical existence, changes and service innovation can be invisible and
will therefore be difficult to address and record. This may explain why some parts of inno-
vation become hidden, as Camisón and Monfort-Mir (2012) argue. Service firms often con-
stantly adapt and reform their activities to provide solutions to changing and
differentiated customer requirements. Tether (2005) refers to this as a continuous-
change mode of innovation and also argues that it is almost impossible to distinguish
between different innovation types. Furthermore, the co-production of services, where
the provider and client work closely together to produce the outcome, complicates the
definition of innovation. The origin and attribution of any innovation may be difficult to
determine. Several studies have examined service innovation through demarcation
(Gadrey & Gallouj, 1998; Sundbo & Gallouj, 2000).
The development of the CIS conveyed in the Oslo Manual from version 1 to 2 and 3 has
resulted in four categories of innovation. However, as the demarcation literature suggests,
it is almost impossible to distinguish between innovation types in services. The tourism
literature also supports this argument. Eide and Mossberg (2013) show how innovation
can be complex, fuzzy and intertwined in relation to experiences. They examine
whether intertwined innovation types should be registered as one innovation or multiple
ones, or perhaps as a new type of innovation. Thus, the third research question in this
study is as follows: Are tourism managers capable of distinguishing between innovation
types in the CIS?
Turning to service logic thinking, the model of Grönroos and Voima (2013) might help
to illustrate the service-related features that demarcation researchers argue are underva-
lued in the CIS. Grönroos and Voima (2013) illustrate the value of the co-creation process in
service logic, that is, how service logic is a process that encompasses design, development,
manufacturing, delivery and usage. Because the CIS originally defined innovation as being
strictly technological (and connected to manufacturing), it is reasonable to assume that
428 A. J. NORDLI

the CIS instrument covers the manufacturing stage in Grônroos and Voimas model well.
CIS has, however, been developed to include services, but still this advancement is criti-
cized for not measuring all aspects of service innovation. Consequently, the CIS’ measure-
ment might not capture all the stages in the service logic model adequately. It is clear in
the model of Grönroos and Voima (2013) how the customer sphere and the provider
sphere overlap and represent the stage of usage, which underlines the interactive
nature of services. Camisón and Monfort-Mir (2012) refer, for example, to service or
sales infrastructure and customer delivery processes as not included in the non-techno-
logical innovations defined in the Oslo Manual. Services or sales infrastructure might be
related to what Grönroos and Voima call back-office attributes, whereas customer delivery
processes occur in the front office (delivery and usage). In the context of the service logic
thinking, the question is whether the CIS, with its roots in manufacturing and technology,
is fully developed to capture innovations related to all stages of the service logic process.
Figure 1 illustrates the demarcation critique of the CIS based on a service logic thinking.
The figure illustrates that the manufacturing stage is completely covered, whereas the
measuring of the other stages is not fully developed and still inspired by the manufactur-
ing instrument. The dotted lines illustrate that the measurement are still are deficient, thus,
addressing the fourth research question in this study: Does the CIS fail to capture innovation
related to services features?
Recently, a third approach to understanding innovation in services has arisen. The third
perspective is called the “integrative approach” (Gallouj & Savona, 2008) or the “synthesis
approach” (Coombs & Miles, 2000). This approach covers both goods and services inno-
vation (Gallouj & Weinstein, 1997), and makes it possible to compare innovations
among industries in a more powerful way. This approach remains in a very early stage
of development, but as, for example, Djellal and Gallouj (2000) explain, the debate that
took place prior to the revision of the Oslo Manual in 1996 opened up a number of
new and very fruitful avenues of research, both service-oriented and integrative in
scope. However, these new paths were not taken into account in the revision of the
Oslo Manual in 1996 (Djellal & Gallouj, 2000). This underlines the potential for further
development of the CIS. Drejer (2004) also points to the assimilation approach as being
too narrow for both manufacturing and services and highlights the need for more integra-
tive approaches.

Figure 1. Service Logic and the CIS, a model inspired of Grönroos and Voima (2013).
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 429

The tourism industry is a good example of manufactured products and services

being intertwined. Tourism is a service industry, but at the same time, it depends on
technological development in relation to, for example, booking systems and ticket
sales. Lift systems in alpine centres and rollercoasters in parks are also examples of tan-
gible and material service products. At the same time, service features cannot be
ignored, such as, for example, the role of employees who play a very crucial role in
their dealings with guests. This underlines the need for the CIS to be improved in
an integrative manner.

Method
The study examines the CIS questionnaire and explores how well managers understand
the questions and how they identify/register reliable information about any innovations
implemented. Qualitative interviews represent a type of exploratory approach, and
provide access to the experiences and information of others (Weiss, 1995). Explorative
qualitative data can be a useful tool for acquiring basic information aimed at developing
quantitative instruments such as questionnaires (Miles & Huberman, 1994; Steckler,
McLeroy, Goodman, Bird, & McCormick, 1992). Eight senior managers and department
managers from tourism businesses were interviewed after they had completed the CIS
questionnaire.

Sampling, cases and respondents

In the selection of respondents, the focus was on the senior management level because
CIS surveys in Norway (SN) are sent to this level of the organization and we assume
that they are answered by managers at this level. Variation in company size and services
offered were the criteria.
Three senior managers from three tourism companies were interviewed: two of the
companies were small hotels and the other was one of the largest alpine centres in
Norway. The alpine centre employs the equivalent of about 60 full-time staff and has
200 employees on the payroll, and the two hotels employ the equivalent of about
five person-years and have 15−20 employees on the payroll. Furthermore, the three
companies are involved in different areas of tourism. The alpine skiing company
offers adventure experiences, while the hotels offer accommodation and restaurant
experiences. The alpine centre consists of various departments: ski school, tickets and
information, finance and personnel, outside operations, and a summer/biking depart-
ment. In attempting to identify innovations that were not reported in the CIS by a
senior manager (and thus were hidden), it was necessary to also interview department
managers. All department managers have management responsibility for between 10
and 30 employees, and thus have similar roles to senior managers in other small
tourism businesses, and because of that, they are regarded as representative of them
as well. The respondents were asked to participate in the study, and before answering,
they were told briefly, what the interview involved and that the study was about inno-
vation in tourism.
430 A. J. NORDLI

The interviews
We developed a semi-structured interview guide based on the research questions. First,
the respondents were asked to answer the CIS questions about the four innovation
types (the questions/items that were used by Statistics Norway in the 2010 tourism
survey). The respondents were then asked about their interpretation of the CIS questions,
and each was allowed to speak in a comfortable and relaxed setting. This was to minimize
any impact the interviewer may have on the responses (Seidman, 2013). Then respondents
were asked to describe the type and characteristics of all of the innovations reported.
Department managers were asked whether senior management had been involved in
or knew about the innovations and senior managers were asked whether they thought
crucial innovations occurred that they did not know about. This was done to identify
hidden innovation. Any respondents who were uncertain about the questions were
asked to read the introductory questions from the questionnaire used in the study by
Rønningen (2009) in order to see how respondents experience the changes they made
in that questionnaire. The interview guide was used as a checklist to ensure that all
issues were discussed. In total, we conducted eight interviews. All of them lasted
between 45 minutes and one hour and they were all transcribed.

Systematizing the data

The interview guide consists of defined key words and key questions connected to the
research questions. After transcribing all the data, the key findings (based on key words
and key questions) were summarized, Key Quotes were underlined and systematized in
a table (based on the same key words and key questions). Due to insufficient data on
market innovation in the ski centre, it was decided to concentrate the study on
product, process and organizational innovation. In addition, the literature review shows
how those three types of innovation are important when it comes to intertwined inno-
vation types and service feature-related innovation.

Summary of findings and key quotes

The key findings from the interviews are summarized and systematized in Table 1. The first
row lists the key topics in the interview guide, while the first column describes the respon-
dents briefly. The first five respondents R1–R5 are the department managers of the alpine
centre, R6 the manager of the alpine centre, R7 the manager of the family hotel and R8 the
manager of the mountain hotel. Main findings connected to each of the respondents is
summarized in Table 1.
Next, we discuss the findings in more detail.

Respondents’ backgrounds and understanding of questions

The first column presents the education levels of all the respondents. None of the depart-
ment managers R1–R5 has a university education (bachelor’s or higher). R7 and R8 are the
only respondents who hold master’s degrees. Employees in the tourism industry generally
have lower formal qualifications than do those in other industries and it is a strength of this
study that both highly educated and lower educated are represented.
Table 1. Summary of main findings.
Comparing Preparation before
Rønningen (2009) and Distinguishing among answering the CIS Examples of innovations that are not
Education Understanding of questions in the CIS CIS innovation types survey captured with the CIS
I1 Ski school No bachelor’s or “Possible to understand, but (shows Yes, a little easier to Struggled. Did not report Treasure Hunting, a product/service
master’s uncertainty); it would have been understand. Asked the interviewer innovation in the ski school that was
degree easier if I worked in a Explanations are what type of systematically to the significantly changed, without
manufacturing company that simplified. innovation it was senior level of changing the name.
created, for example, screws!” (process or management. “The Snake” a new tool (new
“I think it is challenging to connect organization). technic) used when teaching
the survey questions to the ski children to ski
school business.” (Top management did not know
about theme)
I2 Information No bachelor’s or “It was very difficult and I was “Yes, easier to Struggled. Did not report
and ticket master’s completely confused!” understand, more “They are challenging innovation
sales degree Caught up in product innovation. reader friendly and to distinguish.” systematically to the

SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM

an easier senior level of
phraseology” management.
(not as scary).
I3 Personnel No bachelor’s or “Would not call it difficult to No, not easier to Struggled. Did not report Changes in responsibility and
and Finance master’s understand, but challenging to understand. “Yes, yes, it is a matter innovation reorganization of employees at
degree answer the questions.” of taste where to systematically to the the departmental level, with
“One gets the impression that it is place it” (process/ senior level of significant increased satisfaction
more directed towards goods than org.). management. among employees
services!”
Omission of guest-related
questions.
Commented that it might have
been an advantage to read the
whole questionnaire first.
I4 Biking No bachelor’s or “Questions were in some sense okay No comment. Struggled, especially Did not report Many small changes in handling the
(summer) master’s …“ between process and innovation guest in the bike rental business
degree “I felt that I could not answer the organization. systematically to the that resulted in a remarkably
questions in the beginning.” senior level of better experience for the guest.
“I felt that I understood the management.
organizational questions. However,
the products and processes were
beyond my understanding.”

431
(Continued )
 432
A. J. NORDLI
Table 1. Continued.
Comparing Preparation before
Rønningen (2009) and Distinguishing among answering the CIS Examples of innovations that are not
Education Understanding of questions in the CIS CIS innovation types survey captured with the CIS
I5 External No bachelor’s or Unaccustomed and felt like an exam. Cannot see any Struggled, especially Did not report Development of aids/tools to
operations master’s “It was difficult to answer.” difference. between product and innovation internal maintenance of lift
degree process. systematically to the systems.
senior level of
management.
I6 Senior University Understood very well, but Was not asked Struggled. Did not gather Refers to employees innovating all
manager: educated commented: because he Miss questions information the time without him knowing it
Alpine centre (master’s) “It seems as if the questions are managed the CIS so appealing to systematically about (As for example, the Ski school
directed towards production and well. something in innovation at the products and the aids developed
physical goods production.” between product/ departmental level. in the outside Operation
In addition, he missed a better process and department). He tries to stimulate
presentation of how the innovation organization. employees to innovate.
types are indexed.
I7 Senior University “Fine to answer and easy to Was not asked Struggled. Did not gather Two new dinner concepts.
manager: educated understand.” because he Viewed all types as information (Experience dinners. One a
Small hotel (master’s) managed the CIS so intertwined. systematically about “mountain farm” dinner from
well. innovation. older days and the other a
cooking-competition concept).
I8 Senior No formal Understood the questions well, but Was not asked Did not struggle (had Did not gather
manager: education, commented that they were because he no process information
Small only superficial, and he needed to think managed the CIS innovations to systematically about
mountain experience hard to make questions relevant to well. report). innovation.
hotel from a range his business! He asked how many
of jobs small innovations he was allowed to
report.
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 433

First, three respondents asked for more information before starting the CIS. Obviously,
they felt uncertain when confronted with questions about innovation. One respondent
(R6) suggested some kind of index that outlines the content of the survey.
Table 1 shows that three of the eight respondents (R1, R3 and R6) argued that the ques-
tions seemed to have been developed for goods-producing businesses (manufacturing).
See their comments in column 3 of Table 1.
Two respondents commented that they had difficulty in connecting the questions to their
work, and two respondents believed that the questionnaire did not adequately address
certain areas. One of them stated that there were too few guest-related questions. R3 stated:
There could have been more questions about the guest, in relation to the guest experience
and the interaction between the guest and the organization. We are trained to think about
the guest and innovation in relation to the guest.

R3 discussed the lift personnel as an example, and how they interacted with guests in
the lift and how they communicated with guests. Such issues were omitted from the ques-
tionnaire, she argued.
When asking the respondents whether they understood the questions and question
formulations, two of the eight stated that the questionnaire was very difficult to complete.
Additionally, four more respondents were uncertain about the questions, which the
quotes in Table 1 illustrate. They stated that they understood the questions, but they hesi-
tated when responding. During the interviews, they gave the impression that they were
unsure about their answers. At the end of the interviews, the five respondents who
demonstrated the most uncertainty (R1−R5) were asked the questions from the study
by Rønningen (2009), that is, the introductory questions before each innovation type to
test whether they understood those adjusted questions better. Two respondents could
not see any difference between the two sets of questions, one had no comment and
two respondents (R1 and R2) found them easier to understand and more reader-friendly
because the explanations were simplified.
The lower education level of the R1–R5 respondents may limit their ability to under-
stand. Further attention will be given to this issue in the discussion section.

Distinguishing among innovation types

During the interviews, seven of the eight respondents were clearly struggling to distinguish
among the innovation types. They were particularly struggling in relation to process and
organizational innovation, but also with respect to product and process innovation, which
they perceived as containing elements from both product and process innovation. For
example, one respondent could not see the difference between the methods of providing
services (process innovation) and business practices for organizing procedures (organization
innovation), because to her, it is exactly the same thing. See the quotes/findings in Table 1.
There is no doubt that distinguishing innovation types in a tourism business is challen-
ging. Only R8 had no difficulty distinguishing among innovation types.

Hidden innovation
None of the respondents referred to procedures of reporting innovation to senior man-
agers or senior managers gathering information about innovation at the departmental
434 A. J. NORDLI

level. Respondents R1–R5 do not report innovation to senior management, and R6−R8 do
not systematically gather information about innovation. This means that if innovation
occurs at the departmental level, senior managers may be unaware of such innovations.
Our findings clearly indicate that some innovations are not reported in the CIS (see
Table 1, last column). Senior managers I6 and I8 state that innovation occurs among
employees, that they are not informed about it, and that they have no system of reporting
innovations. The following quotes from department managers support this:
We continuously develop our service product; this is not like manufactured products that you
finish the production of. (R3)

We change continuously. If someone comes up with a good idea, then we test it. If it works
then we use it. (R4)

These findings reveal more innovations that are not reported in the CIS. Some examples
are Hafjell-Skatten, a product/service in a ski school that they changed/developed con-
siderably to create a new concept, but without changing the name (a product innovation).
Innovations commonly occur without the senior management knowing. For example,
many small changes in dealing with customers in the bike rental industry have occurred,
including the queuing system, order of equipment handover and submission and
payment. According to R4, many small changes typically occur, which in total result in a
significantly better experience for the guest and the employees (process innovation). Fur-
thermore, changes in responsibility and reorganization of employees at the departmental
level (organization innovation) occur regularly. Two new dinner concepts that R7 forgot to
report because he was too focused on bigger investments (product innovations). A few
more examples are listed in Table 1 (last column).

Discussion
Summarizing the findings in Table 1, the majority of the respondents show uncertainty in
regard to their understanding of the CIS. Six respondents were uncertain when interpret-
ing some of the questions. Seven out of eight respondents find it difficult to distinguish
between (some of) the innovation types, and find the questions/items of little relevance
to their business. Obviously, there is potential for improving the questionnaire to help
tourism managers report more valid information. Below are various suggestions to
improve the CIS. The last column in Table 1 reveals that some innovations are not captured
by the CIS, a problem that may be solved by improving the instrument.
In general, employees and managers in tourism have less formal education compared
with other service businesses (Hjalager, 2002). Thus, they are probably not trained in using
technical terminology and language. This means that tourism managers may have differ-
ent perspectives when answering a questionnaire such as the CIS that contains technical
questions. When it comes to the language and terminology, the five most uncertain
respondents were presented with information from the paper by Rønningen (2009),
which contains an adjusted version of the CIS. These respondents were also the ones
with the lowest education level. Two of them found the information from Rønningen
(2009) easier to interpret. Because a pilot test of the adjusted CIS had previously indicated
language challenges, there is scope for the survey to be rewritten so that the terminology
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 435

is easier to associate with a given work environment. An example of how the questions
were simplified is the introduction to process innovation provided in the adjusted CIS:
SN’s Norwegian version (translated to English):

Process innovation involves a new or significantly improved production technology/method

and new or significantly improved methods for delivery of goods and services. The innovation
must be new for the enterprise but the enterprise does not necessarily need to be the first to
introduce this process. It does not matter whether the innovation was developed by your
enterprise or by other enterprises. Pure organizational changes should not be included here.

Rønningen’s (2009) Norwegian version (translated to English):

Process innovation involves new or substantially improved methods of delivery of existing

products/services. The innovation must be new to your business, but need not be new to
the market.

Another example of an adjustment that was made is in relation to organizational inno-

vation where expressions such as supply chain, lean production/BPR (Business Process
Reengineering), quality assurance and optimization were left out of the questions. Also
important to mention is that Rønningen (2009) asked about the four innovation types
at the beginning of the questionnaire, whereas SN added three pages of other questions
(many R&D related) between the questions on product/process innovation and those on
organization and market innovation. Ten of 600 tourism companies reported R&D activity
(Wilhelmsen & Foyn, 2012); however, SN present research results that show how R&D ques-
tions in relation to innovation questions can affect the answers and result in less reported
innovation (Wilhelmsen, 2012).
When using the CIS in the tourism industry, such minor adjustments could be con-
sidered to account for senior managers with lower educational level. It would also be poss-
ible to add questions to the CIS about respondents’ education level, thus providing a
possibility to test for such biases.
A large part of the service innovation literature points to major challenges in dis-
tinguishing among innovation types (Tether, 2005). All respondents in the study (except
R8) state that they struggled to differentiate between innovation types and that inno-
vation categories or types are intertwined. Many of the examples mentioned in Table 1
are related to uncertainty about product and process innovations or process and organiz-
ation innovations. In addition, all respondents mention that much of the innovation work
is ongoing unstructured processes, making it difficult to report details of starts, com-
pletions and numbers. This underlines the finding of Eide and Mossberg (2013) that inno-
vation in tourism can be complex, fuzzy and intertwined. No wonder that the introduction
to each innovation type confuses the respondents. As the respondents pointed out, a
better explanation of the survey is required, particularly regarding the four innovation
types. Otherwise, rather than separating innovation into the four types, the viability of
only using a broad definition of innovation could be examined.
Considering the current operationalization of innovation in the CIS, three respondents
state that the questions relate to manufacturing but less to services. This could also be the
reason why respondents struggle in relating the questions to their businesses. Even
though the CIS is adjusted in the Oslo Manual, the improvements do not seem to be ade-
quate for tourism services. One respondent suggested a lack of guest-related questions, an
436 A. J. NORDLI

example of which could be: Have you established new or significantly improved routines/
methods for how you interact with or serve customers? This question relates more to ser-
vices and interaction/co-creation (the service delivery stage in the Figure 1). Manufactur-
ing companies may also have departments delivering services and handling customers,
thus supporting the view that the assimilation approach is too narrow for manufacturing
as well (Drejer, 2004). Furthermore, even though the questionnaire seems to be appropri-
ate for manufacturing, not all manufacturing companies have R&D departments. This is a
challenge that the literature has identified when studying innovation in smaller manufac-
turing companies (Kleinknecht, 1987; Sterlacchini, 1999). This means that an explorative
research design similar to this should also be applied to a manufacturing context. This
is important for keeping the instrument similar across industries, which is one of the objec-
tives of the CIS.
Comparing today’s instrument with the service logic model (Grönroos & Voima,
2013), a review of the literature illustrates that the CIS mainly relates to the manufactur-
ing stage. If items are missing in relation to the stages of design, development and
delivery of services, we should determine whether there are innovations that the CIS
did not capture. Our findings, summarized in Table 1, show that there are innovations
that the CIS has not managed to capture, supporting the findings of Camisón and
Monfort-Mir (2012). Those innovations are mostly developed and implemented at the
departmental level in the companies, but are still quite significant innovations. As our
findings suggest, these innovations are not just small adjustments that cannot be
defined as innovations. An example from the ski school is the concept of Hafjell-
Skatten that has been considerably changed, almost redeveloped. This innovation is
not captured in the CIS because it has not been renamed, and therefore has not
involved senior management. The ski school has the authority to develop products
on their own. None of the companies has routines for reporting information to senior
management. This means that their innovations remain hidden in a CIS survey.
Another example is the two relatively large product innovations that were forgotten
by R7. In this case, the respondent forgot because he was so focused on investments
in new apartments and kitchen facilities. Furthermore, there are various examples of
hidden innovation shown in Table 1. Because senior managers are unaware of these
innovations, they would not be reported in a CIS survey. Requiring companies to estab-
lish routines and gather information from department managers is one alternative to
help overcome certain types of hidden innovation.
Some of the hidden innovation examples involve interacting with guests or the delivery
of services. This means that those innovations will be captured by adding the question:
Have you established new or significantly improved routines/methods for how you inter-
act with or serve customers? By comparing the CIS instrument as it is today with the model
of Grönroos and Voima (2013), it might be possible to identify even more missing items,
particularly in relation to design and development, as well as to service delivery (which
Camisón and Monfort-Mir (2012) state are not captured by the Oslo Manual).
Finally, we can conclude that our findings support those of Camisón and Monfort-Mir
(2012) that some innovations in tourism are not captured by the CIS. The CIS only captures
the tip of the innovation iceberg. Hidden innovations range from all the small changes and
improvements, which may or may not be considered innovations, to larger and more sig-
nificant innovations such as the ones identified in this study. Figure 2 illustrates the
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 437

Figure 2. CIS, the tip of the innovation iceberg.

innovation iceberg, illustrating how hidden innovations lie under the surface waiting to be
reported.
Even though these results show a clear tendency of under-reporting, the potential issue
of over-reporting should also be mentioned. This study does not reveal any over-reporting.
However, R3 and R4 reported relatively small innovations, which have had significant
overall effects, and thus interpret them as hidden innovations. In addition, our results
suggest that the informants naturally think of innovation as major changes only. One infor-
mant (R8) asked how small can the reported innovations be, which underlines this think-
ing. The fact that R7 forgot to report two new experience–dinner concepts, because he
was so focused on apartment and kitchen investments, further highlights this issue.
According to these findings, under-reporting (hidden innovation) is the biggest challenge
and should thus be given the most attention.

Conclusion
The findings of this study represent a first step in making the CIS a more valid instrument.
The paper suggests various ways in which the CIS should be improved. First, there are indi-
cations that the technical language and formulation should be modified to allow the
respondents to better understand the measurement aims of the survey. Second, the find-
ings point to major challenges for the respondents because they fail to distinguish among
the four types of innovation in the CIS or are confused and therefore forget or overlook
innovations. One suggestion is to remove the headlines and introductions distinguishing
between innovation types, and present all items together. Furthermore, statistical tech-
niques such as factor analysis can be used to identify underlying innovation dimensions.
Then, it is possible to infer a typology based on the empirical finding and not on the main-
stream innovation literature. Another suggestion is to test the effectiveness of an index at
the start of the survey that better prepares the respondents for the survey. We also ident-
ified a potential to develop new items related to service logic and additional items reflect-
ing the educational level of the person completing the questionnaire.
438 A. J. NORDLI

Last but not least, the study finds empirical evidence of hidden innovations that are
illustrated in the iceberg metaphor. Several types of innovation are hidden, especially
innovations at the department level. In regard to identifying more hidden innovations,
the findings also reveal that none of the companies has routines for reporting information
about innovations developed and implemented at the departmental level. Such pro-
cedures may also help to reveal more of the hidden innovations. Another issue is
whether it would be appropriate to separate reported innovations into two types of inno-
vation: easily reported innovation and hidden innovation. If surveys that find high inno-
vation rates, such as Rønningen (2009), have managed to capture more hidden
innovations, this implies that the tourism companies might be weak innovators in
regard to innovations that the CIS captures easily. Consequently, it is important to
analyse both captured and hidden innovation to determine how best to stimulate both
types of innovation in the tourism industry. For example, could more significant/major
innovations that are easily captured be categorized as type 1 innovations and smaller
innovations/reported at the department level be categorized as type 2 innovations?
This means that future research needs to examine qualitatively the difference between
reported and hidden innovations in tourism.
Furthermore, in regard to over-reporting, it is important to do more qualitative research
on how to separate hidden innovation and continuous development, where the latter
should not be classified as innovation.
All suggested improvements to the CIS instrument should, of course, be implemented
and tested further in a pilot study before use, since this study only represent a first explora-
tive step. Because the paper focuses on the contents the CIS (survey) is addressing, it does
not evaluate the survey from a technical survey quality perspective. This means that the
general literature on survey methodology has not been addressed, though this issue
deserves further attention.
Furthermore, in regard to other service industries and manufacturing, a research design
similar to the one used in this paper would be useful. The present study has improved the
knowledge about the use of quantitative instruments such as the CIS to measure inno-
vation in tourism. This study contributes by examining ways to improve innovation
surveys/CIS to capture more service innovation, and more hidden innovation. It is
argued that the CIS should not be rejected, but rather be improved by implementing
more ideas from the demarcation approach, and thus moving the CIS in a more integrative
direction. The suggestions given in this article may be characterized as a moderate inte-
grative approach.

Acknowledgements
The author would like to thank Professor and Supervisor Martin Rønningen for careful feedback,
supervision and collaboration through the working process. The article is a part of the PhD thesis
“Measurement and Management of Innovation in Tourism.”

Disclosure statement
No potential conflict of interest was reported by the author.
 SCANDINAVIAN JOURNAL OF HOSPITALITY AND TOURISM 439

References
Alsos, G. A., Eide, D., & Madsen, E. L. (2014). Handbook of research on innovation in tourism industries.
Cheltenham: Edward Elgar.
Camisón, C., & Monfort-Mir, V. M. (2012). Measuring innovation in tourism from the Schumpeterian
and the dynamic-capabilities perspectives. Tourism Management, 33(4), 776–789.
Coombs, R., & Miles, I. (2000). Innovation, measurement and services: The new problematique. In J. S.
Metcalfe & I. Miles (Eds.), Innovation systems in the service economy (pp. 85–103). Boston, MA:
Kluwer Academic.
Djellal, F., & Gallouj, F. (2000). Innovation surveys for service industries: A review. Paper presented at the
Conference on Innovation and Enterprise Creation: Statistics and Indicators, Sophia Antipolis,
France.
Drejer, I. (2004). Identifying innovation in surveys of services: A Schumpeterian perspective. Research
Policy, 33(3), 551–562.
Eide, D., & Mossberg, L. (2013). Towards more intertwined innovation types: Innovation through
experience design focusing on customer interactions. In J. Sundbo & F. Sørensen (Eds.),
Handbook on the experience economy (pp. 248–268). Cheltenham: Edward Elgar.
Fuglsang, L., Højland, J., Sundbo, J., & Sørensen, F. (2008). Innovation i oplevelsesvirksomheder:
Resultater fra en survey. Roskilde Universitet. Retrieved from http://rudar.ruc.dk:8080/bitstream/
1800/3103/2/Innovation_i_oplevelsesvirksomheder.pdf.
Gadrey, J., & Gallouj, F. (1998). The provider-customer interface in business and professional services.
Service Industries Journal, 18(2), 1–15.
Gallouj, F., & Savona, M. (2008). Innovation in services: A review of the debate and a research agenda.
Journal of Evolutionary Economics, 19(2), 149–172. doi:10.1007/s00191-008-0126-4
Gallouj, F., & Savona, M. (2010). Towards a theory of innovation in services: A state of the art. In F.
Gallouj & F. Djellal (Eds.), The handbook of innovation and services: A multi-disciplinary perspective
(pp. 27–48). Cheltenham: Edward Elgar.
Gallouj, F., & Weinstein, O. (1997). Innovation in services. Research Policy, 26(4/5), 537–556.
Grönroos, C., & Voima, P. (2013). Critical service logic: Making sense of value creation and co-creation.
Journal of the Academy of Marketing Science, 41(2), 133–150.
Hall, M. C., & Allan, W. (2008). Tourism and innovation. London: Routledge.
Hjalager, A.-M. (2002). Repairing innovation defectiveness in tourism. Tourism Management, 23(5),
465–474.
Hjalager, A.-M. (2010). A review of innovation research in tourism. Tourism Management, 31(1), 1–12.
doi:10.1016/j.tourman.2009.08.012
Kleinknecht, A. (1987). Measuring R & D in small firms: How much are we missing? The Journal of
Industrial Economics, 36(2), 253–256.
Mattsson, J., Sundbo, J., & Fussing-Jensen, C. (2005). Innovation systems in tourism: The roles of
attractors and scene-takers. Industry & Innovation, 12(3), 357–381. doi:10.1080/
13662710500195967
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook. Thousand
Oaks, CA: Sage.
OECD. (1997). Guidelines for collecting and interpreting innovation data, Oslo Manual (2nd ed.). Paris:
Author.
OECD/Eurostat. (2005). Oslo manual: Guidelines for collecting and interpreting innovation data (3rd
ed.). Paris: OECD Publishing.
Rønningen, M. (2009). Innovation i reiselivsnæringen. In H. Teigen, M. Mehmetoglu, & T. Haraldsen
(Eds.), Innovasjon, opplevelser og reiseliv (pp. 11–39). Bergen: Fagbokforlaget.
Rønningen, M., & Lien, G. (2014). The importance of systemic features for innovation orientation in
tourism firms. In G. A. Alsos, D. Eide, & E. L. Madsen (Eds.), Handbook of research on innovation
in tourism industries (pp. 27–55). Cheltenham: Edward Elgar.
Rønningen, M., & Nordli, A. J. (in press). Er reiselivsnæringene innovative? In E. Merok & T. Arnesen
(Eds.), Innovasjon i fragmenterte næringer. Vallset: Oplandske Forlag.
440 A. J. NORDLI

Schumpeter, J. A. (1934). The theory of economic development: An inquiry into profits, capital, credit,
interest, and the business cycle. Cambridge, MA: Harvard University Press.
Seidman, I. (2013). Interviewing as qualitative research: A guide for researchers in education and the
social sciences. New York, NY: Teachers College Press.
Serin, G. F., & Hansen, P. A. (1997). Will low technology products disappear? Technological Forecasting
and Social Change, 55, 179–191.
Steckler, A., McLeroy, K. R., Goodman, R. M., Bird, S. T., & McCormick, L. (1992). Toward integrating
qualitative and quantitative methods: An introduction. Health Education Quarterly, 19(1), 1–8.
Sterlacchini, A. (1999). Do innovative activities matter to small firms in non-R&D-intensive industries?
An application to export performance. Research Policy, 28(8), 819–832.
Sundbo, J., & Gallouj, F. (2000). Innovation as a loosely coupled system in services. International
Journal of Services Technology and Management, 1(1), 15–36.
Sundbo, J., Orfila-Sintes, F., & Sørensen, F. (2007). The innovative behaviour of tourism firms –
Comparative studies of Denmark and Spain. Research Policy, 36(1), 88–106. doi:10.1016/j.respol.
2006.08.004
Tether, B. S. (2005). Do services innovate (differently)? Insights from the European Innobarometer
Survey. Industry & Innovation, 12(2), 153–184. doi:10.1080/13662710500087891
Weiss, R. S. (1995). Learning from strangers: The art and method of qualitative interview studies.
New York, NY: Free Press.
Wilhelmsen, L. (2012). A question of context: Assessing the impact of a separate innovation survey and
of response rate on the measurement of innovation activity in Norway. Statistics Norway,
Documents, 51.
Wilhelmsen, L., & Foyn, F. (2012). Innovasjon i norsk næringsliv 2008–2010 (Rapporter 46/2012). Oslo –
Kongsvinger: Statistics Norway.