Measuring Innovation in the Australian Public Sector

Consultation Paper 1.0

The Australian Public Sector Innovation Indicators (APSII) Project

June 2011
Contents

Foreword ........................................................................................................................3

Introduction....................................................................................................................4

The APSII Project ..........................................................................................................5

Objectives: .............................................................................................................5
Consultation Process:.............................................................................................5
SRG Meeting: ........................................................................................................5
Further information:...............................................................................................6
Proposed Methodology: .........................................................................................7
1. Proposed measurement framework................................................................7
2. Proposed indicators........................................................................................7
3. Proposed data collection ................................................................................8
Project Plan: ...........................................................................................................9

Background information ..............................................................................................10

Public Sector Innovation developments in Australia...........................................10
The APSII Project as part of the Innovation Agenda ..........................................12
Measuring Public Sector Innovation – Other Initiatives......................................13
Foreword
For the past eight months I have had the pleasure of chairing the APS 200 project on
Public Sector Innovation, which was established under the guidance of the Secretaries
Board and the APS 200 Senior Leadership Forum and is hosted by the Department of
Innovation, Industry, Science and Research.

The work done by this group is part of the wider APS Reform: Building the Future
Together, which implements the recommendations of the Ahead of the Game –
Blueprint for the Reform of Australian Government Administration. These
recommendations were adopted by the Australian Government in March 2010.

The project committee was charged with translating the theoretical foundation of the
Empowering Change: Fostering Innovation in the Australian Public Service Report
into more concrete action. The Australian Public Sector Innovation Indicators (APSII)
Project arose from this initiative. It will assist the public sector in building a
systematic and strategic approach to facilitating innovation by providing public sector
agencies with the tools to identify their strengths and weaknesses in their innovation
capacity and performance. Innovation is critical if we are to address social,
environmental and economic challenges.

There is much greater capacity for innovation in the APS than is currently being
realised. Your involvement in the APSII Project is essential if we are to develop
meaningful ways of measuring innovation and provide guidance to the APS on public
sector innovation.

Patricia Kelly
Chair – APS 200 Public Sector Innovation Project
Introduction
The Australian Public Sector Innovation Indicators (APSII) Project aims to develop
and implement a way of measuring and reporting on innovation capacity and
performance in the Australian Public Service and wider public sector. It is being
undertaken by the Department of Innovation, Industry, Science and Research (DIISR)
and reports to the APS Secretaries Board.

This initial consultation paper provides information about the project and invites
comment and input. Specifics about the consultation process are given, details of the
ASPII Project, its aims, methods, project plan, and anticipated outcomes are provided,
as well as a sense of where the Project sits relative to the broader innovation policy
context and international and domestic initiatives in this new and developing field.
Whilst it is initially being circulated to 19 federal government agencies, contributions
from the wider public sector will be invited in due course.

Innovation in the context of public sector innovation measurement, and for this
project, is defined as:

“the implementation of a significant change in the way an

entity operates or in the products it provides. Innovations
comprise new or significant changes to products, operational
processes, organisational methods, or the way an entity
communicates with users.” 1

Sound measurement of innovation is crucial for policy formulation and delivery of

public goods and services. In many countries governments are encouraging
innovation capabilities within their public services. The policy environment today is
complex and volatile. In grappling with complicated policy issues progress requires
new and increasingly sophisticated responses. Sometimes referred to as “wicked
problems”, policy challenges, such as climate change, can have no final definite
solution, but rather are a work in progress. 2 They cannot be adequately addressed
without innovative approaches.

The Australian Government has recognised that an innovative public sector is a

necessity. The APSII Project places Australia at the forefront of international efforts
and aims to equip agencies with the data and indicators3 needed to better understand
and encourage their innovation performance and capacity. This will be a tool to drive
decisions to achieve goals and better outcomes for Australia.

1
A significant change – significant does not relate so much to the size of change but to how new or different it is
(Empowering Change, 2010, p.89)
2
Cutler, T. (2009) Designing Solutions to Wicked problems: A Manifesto for Transdisciplinary Research and Design.
Proceedings from the symposium, Melbourne, 9-10 November 2009.
3
Indicator – the thing to be measured eg. the level of employee satisfaction
The APSII Project

Objectives:
1. To provide data and metrics on the innovation performance and capacities of
agencies in order to assist the management of innovation in the Australian
Public Service.
2. To provide internationally comparable data and metrics that allows
benchmarking of the Australian public sector against overseas counterparts.

Consultation Process:
We are seeking your input to help shape the project in its next phase – the
development of a measurement framework with information, indicators, and reporting
tools.

Your contributions will help guide the project, provide advice and ensure the delivery
of high quality innovation data and measurement tools that have utility across the
Australian public sector.

As a member of the Stakeholder Reference Group (SRG) your role is to provide input
on the utility of the proposed pilot survey, the pilot report on the state of innovation in
the APS, and the pilot analytical reports on innovation for individual agencies. The
SRG is made up of representatives from across the Australian Public Service at the
senior executive band 1 levels. Several experts in public sector policy from academia
and public policy organizations have also been invited to participate.

A second group, the Technical Reference Group (TRG), provides technical input on
all aspects of the project’s activities and deliverables. It is chaired by DIISR and is
comprised of members with relevant technical expertise from the Australian Public
Service Commission (APSC), Australian Bureau of Statistics (ABS), DIISR and
academia.

SRG Meeting:
The first SRG meeting will be on 15 June 2011.

This meeting provides an opportunity to discuss the APSII Project with DIISR staff,
experts in public sector policy, and other APS representatives on the SRG. The DIISR
will provide a project overview and explain the more technical aspects of the project.
In order to develop a measurement framework which addresses the needs of
Australian public sector agencies, the correct indicators must be chosen. Your
consideration of the issues below is suggested.

1. How does your agency report on innovation?

2. What sorts of indicators are useful to your agency and why?

 3. How would you use these indicators? 4

4. How would they assist you in enhancing innovation in your agency?

5. What are the key areas for measurement? 5

6. Is benchmarking against international and other Australian agencies useful?

Why/why not?

7. When the analysis of data has been done and is being reported to your
agency, what sort of format would best suit?

8. How much time would people in your agency be prepared to spend in

responding to a questionnaire?

Further information:

For more detailed information and background on the APSII Project read:
1. Working towards a measurement framework for public sector innovation in
Australia: A draft discussion paper for the Australian Public Sector
Innovation Indicators project (2011). APSII team, Department of Innovation,
Industry, Science, and Research, Canberra, Australia. 6
2. The Measurement of Public Sector Innovation: A literature review (2011).
APSII team, Department of Innovation, Industry, Science, and Research,
Canberra, Australia. 7 It forms the basis of this initial Consultation Paper and
provides the methodology and survey design for gathering innovation data.

Information on the APSII Project will be continually updated through blogs on the
Innovation website: http://innovation.govspace.gov.au/ and relevant documents
pertaining to the project will be hosted on the Govdex website:
http://www.govdex.gov.au/ run by the Department of Finance and Deregulation.

You are also welcome to contact the APSII team if you have any queries prior to the
meeting. The SRG coordinator is Dorothea Huber 02 6213 6118,
dorothea.huber@innovation.gov.au.

Any input you provide now and in the future will be treated as confidential. It will be
amalgamated with other responses from members of the SRG to provide guidance to
the ASPII team in the development of the project. Identifying data will be removed
during this process to ensure anonymity of individual responses.

It is envisaged that after the meeting your written feedback can be provided and
should be directed to: APSII@innovation.gov.au.

4
Examples include who developed the innovation, innovation strategies, expenditure, innovation cooperation
5
Examples include innovation procurement, implementation, barriers
6
http://www.govdex.gov.au/
7
\ http://www.govdex.gov.au/
Proposed Methodology:
This section briefly outlines the measurement framework and methodology developed
in the APSII Project Discussion Paper. 8

1. Proposed measurement framework

The measurement framework proposed by DIISR suggests public sector innovation be
measured by consideration of five main components:

• Inputs to innovation
• Innovation processes within the organisation
• Outputs of the innovation process
• General outcomes of innovation
• Environmental conditions (affecting innovation in public sector
organisations)

2. Proposed indicators
In order to quantify these five elements, measurement of more specific indicators
within each of the five components of the measurement framework is needed.

These indicators can be used to develop the survey questions directed to public sector
organisations and their employees to collect data on innovation. Where possible
existing data sources may be used. All data collected contributes to the formation of
the analytical indicators which measure the performance and capabilities of the
agency.

The APSII Project consultation process will determine which are the best indicators to
use in the Australian context. Consultation with stakeholders is essential to ensure the
indicators chosen reflect the needs of public sector agencies.

Table 1 below shows suggested indicators for the five framework themes.

8
Working towards a measurement framework for public sector innovation in Australia: A discussion paper for the
Australian Public Sector Innovation Indicators Project (2011). http://www.govdex.gov.au/
Table 1: Framework themes with Indicators for measuring public sector innovation
Themes Potential indicators for measuring public sector innovation
• Investment in intangible assets (such as investment in computerised information, innovative property,
and economic competencies)
• Innovation expenditures (R&D, training, consulting expenditures and other knowledge purchases etc.)
• Human resources devoted to innovation (number of staff involved in innovation)
Input • Staff skills for innovation
• Staff attitudes and attributes to innovation
• Sources of information (e.g. the Australian Government, management/senior staff versus
employees/frontline staff, users, suppliers)
• Technological infrastructure for innovation (including access to and use of ICT and Web 2.0.)
• Explicit innovation strategy and targets
• Systematic, internal measurement and evaluation of innovation
• Management practices for innovation (active involvement, support for new ideas, trials and
experimenting, support/commitment to innovation and implementation)
Process
• Incentive and reward structures
• Practices for learning and diffusing knowledge and innovations
• Innovation collaboration and alliances
• Perception of enablers and barriers to innovation
• Ongoing innovation projects
• Types of innovations (product, services, processes, delivery models, organisational design and
practices, etc.)

Output • Degree of novelty and scope of innovations (e.g. incremental versus radical innovation, autonomous
versus systemic innovation)
• Innovation intensity (proportion of employees reporting innovation in their work group)
• Related, intangible outputs (e.g. patents, copyright, trademarks)

• Quality and efficiency

• Productivity
• Employee satisfaction
Outcomes • User satisfaction
• Societal and environmental impacts
• Other intangible effects, (e.g. increased trust and legitimacy)
• Effects of innovations
• User innovation (e.g. user-initiated innovation; user as a sources of information
• Supplier innovation (e.g. Suppliers-initiated innovation; suppliers-developed innovation)

Environmental
• Wider public sector culture and leadership identified as drivers or barriers
conditions • External political and legislative factors identified as drivers or barriers (policy, regulation, budget and
organisational structure)
• Leadership and culture (e.g. supportive organisational culture, free flow of new ideas, challenging)
• Public tolerance of risk

Further detail is available in the APSII Draft Discussion Paper. 9

3. Proposed data collection

Feedback on these proposed indicators from the SRG is essential to ensure that the
indicators chosen reflect both the five themes and the needs of public sector agencies.

9
Working towards a measurement framework for public sector innovation in Australia: A draft discussion paper for
the Australian Public Sector Innovation Indicators Project (2011). http://www.govdex.gov.au/
It is proposed to utilise the existing innovation data collection of the Australian Public
Service Commission’s (APSC) State of the Service Survey in late 2011 to generate an
initial data set. A pilot set of innovation questions have been included in this year’s
APSC State of Service employee survey. There are three questions which ask for
information on innovation strategies and for the respondent’s perceptions of the
impacts and drivers for one innovation example which occurred in the past year.
Although this will not give the comprehensive data set required for complete analysis,
the results will inform future questionnaire development.

Whilst still undergoing development, it is envisaged that data collection will be

underpinned by an agency level survey distributed to APS agencies and an individual
level survey randomly targeting APS employees. Comparison with the agency survey
would then permit analysis of perceptions between individuals and agency
representatives, something which is unique internationally. Data mining from other
sources, such as agency annual reports, may also be helpful.

Ultimately the methodology outlined will deliver the innovation performance and
capability of each agency. It will detail the internal and external drivers and barriers to
innovation.

Project Plan:
A conceptual and methodological framework will be developed, as well as a survey
instrument for measuring public sector innovation. This will generate outputs at an
agency and whole-of-APS level. Although the methodology will be trialled at the
federal level, it is being developed with the intention that it be applicable to State,
Territory, and local government agencies.

The project consists of the following stages:

Stage 1 - The conceptualisation and development of the measurement framework and

stakeholder consultation (including the formation of the SRG) over its utility within
the APS by the end of 2011. Substantial research on international approaches to
measuring public sector innovation underpins this work. Utilisation of APSC State of
Service survey data analysis to direct further data collection.

Stage 2 - The gathering of innovation data through a pilot survey and other
appropriate sources and finalisation of the measurement framework. By the end of
2012 the APSII project aims to have conducted a pilot APS innovation survey,
delivered a measurement framework paper for public sector innovation in Australia,
produced a pilot report on the state of innovation in the APS, and prepared pilot
analytical reports on innovation for individual agencies.

Stage 3 – The possible extension of the measurement framework to the wider public
service at the State/ Territory and Local Government levels. This stage is subject to
the outcome of the previous stages and is unlikely to commence before the end of
2012.
 July 2010

Technical Draft Discussion Paper

Reference (Conceptual Framework & Survey
Group Methodology)
S SoS Surveys
T Employees (May 2011)
Agency (July 2011)
A
G
E

1
Final Preliminary Report
Stakeholder
June 2011 Consultation
Reference Discussion State of APS
Paper Innovation
Group Paper

Preliminary
Analytical Report

APS Agencies

Dec 2011

APS employees Pilot Survey Additional Data

& Agencies Employee Questionnaire Sources
Mid 2012 Agency Questionnaire (if required)

S
T
A
G
E

Dec 2012 Framework Pilot Report Pilot Analytical

Paper on Report
for State of APS for
Measuring APSII Innovation APS Agencies

Background information

Public Sector Innovation developments in Australia

In 2009 -10 the Australian public sector accounted for around one third of GDP. It
provides services in numerous areas including health, defence, education, public order
and safety and administration.
Creating and maintaining an environment in which innovative ideas can be generated,
tested, implemented, diffused and evaluated is now regarded as a priority by the
Australian government. A challenge for the public sector is that it has been
characterised by ‘one-off’ innovations which occur despite, rather than because of, the
working environment. 10 Typical scenarios involve crisis-driven innovation or an
innovation being championed by an individual or a small group of individuals.

Innovation in the Australian public sector currently reflects untapped potential in the
form of individuals who want to innovate but believe that the APS limits their
opportunities to do so 11 . Facilitating innovation across the APS and the wider public
sector will involve focussing on identifying the conditions for innovation, as well as
the specific actions that create and sustain those conditions.

Although innovation has been on the Australian Government’s policy agenda since
the 1980s, the change of government in December 2007 prompted a renewed focus on
innovation. To address the challenges of managing and supporting innovation,
agencies are being encouraged to incorporate innovation into their strategic plans, to
regularly review their innovation performance, and to develop strategies for
improvement. This is occurring through the Australian government’s innovation
agenda and a wider APS reform agenda.

The APS Innovation Action Plan endorsed by the Secretaries Board in 2011 provides
a platform and agenda for building an innovative culture in the APS. However the
importance of enhanced innovation in improving the productivity, efficiency and
agility of the public sector has been articulated and reinforced through a number of
reviews and guidance documents, including Venturous Australia: Building strength in
innovation (August 2008), Powering Ideas: An Innovation Agenda for the 21st
Century (May 2009), Ahead of the Game: Blueprint for the Reform of Australian
Government Administration (March 2010), and Empowering Change: Fostering
Innovation in the Australian Public Service (May 2010). 12 The APS 200 Project
Committee final report recommends agencies are more explicit in describing the
contribution of innovation to the achievement of their goals.

There have been some initiatives at the state and local government levels too. In
Victoria, the state Public Service has developed an Innovation Action Plan which
focuses on increasing collaboration, building capability, generating ideas and sharing
information and data. Their aim is to embed innovation across the public service. 13
Since 2008 Western Australia has also been active, running their first Innovation
Workshop for public sector managers in August 2010. 14

At the local government level a new online Local Government Innovation and
Knowledge Exchange Network will shortly be launched by the Australian Centre of
Excellence for Local Government (ACELG) 15 . It aims to provide a multimedia

10
Empowering Change, 2010, p.5
11
The APSC’s State of Service Report 2009-10 and similar sentiments expressed in Empowering Change
12
A specific recommendation in Empowering Change is annual reporting on innovation performance.
13
Department of Premier and Cabinet, Victoria, VPS Innovation Action Plan, Nov. 2009.Available at: www.dpc.vic.gov.au.
14
Examples of public sector innovation strategies and initiatives are outlined in the WA Public Sector Case Studies
Booklet.
15
University of Technology, Sydney.
interactive forum for practitioners and other professionals associated with local
government to exchange information and ideas, showcase and encourage innovation
and better practice, and provide links to resources, research opportunities,
collaboration and support.

The APSII Project as part of the Innovation Agenda

Progress against a significant number of recommendations in the Empowering
Change 16 report will be measured through the APSII Project.

Table 2: The degree to which the APSII project addresses the recommendations
of the Empowering Change Report. [very high (√√√) high (√√)]

No. Recommendations of the Report APSII project Application

1 Innovation should be part of an agency’s Measures the degree to which innovation √√√
strategic thinking and planning. is part of organizational strategic planning
and organizational culture.

2 The APS should adopt innovative practices Identifies and measures the degree to √√
and increased openness in the development which innovation barriers exist within an
of new policy proposals. agency.

4 Agencies should promote effective leadership Measures the degree to which √√√
which builds an innovative culture within the organizational leadership supports
APS. innovation.

5 Agencies should take a more proactive Provides an agency-specific analytical √√

approach to incorporating innovation into their report showing strengths and weaknesses
operations (such as the Toolkit) thereby incorporating innovation into
operations.

6 Agencies should identify and address Identifies and measures innovation barriers √√√
systemic innovation barriers across the APS and at an agency level.

10 Agencies should focus on outcomes rather Provides information about the extent to √√
than specifications in their procurement which procurement is associated with
procedures. innovation.

12 The APS and its agencies should institute Provides information about the extent to √√
mechanisms to recognise, celebrate and which innovation is recognised and shared
share innovation efforts. within agencies.

The remainder of the report’s recommendations (7, 8, 9 and 11) are also addressed by
the APSII Project but at a lower (non-specific) level.

The full text of the recommendations appears in Appendix 1 of the APSII Draft
Discussion Paper. The summary here is intended to give a sense of both the
recommendations and how the APSII Project will measure progress against them.

The key events which have been instrumental in developing a framework for
systematic innovation and a culture of innovation across the APS are outlined fully in
the APSII Draft Discussion Paper. 17

16
Empowering Change: Fostering innovation in the Australian Public Service (2010)

17
Working towards a measurement framework for public sector innovation in Australia: A discussion paper for the
Australian Public Sector Innovation Indicators Project (2011). http://www.govdex.gov.au/
The governance arrangements for the APSII Project are outlined below:
Secretaries Board
Chair: PM&C (Terry Moran)
Members: Secretaries (20 CW Agencies)

APS 200
Senior Leadership Forum for APS
Members: 200 Senior SES

Public Sector Innovation

Project
(August 2010 – April 2011)

Sponsor: Mark Paterson AO DIISR

Chair: Patricia Kelly

Other Deliverables:
APSII — PS Innovation Tool Kit
— APS Design Centre
Project — Awards & Events
— Innovation Showcase

Measuring Public Sector Innovation – Other Initiatives

The APSII Project aims to position itself at the forefront of developments in the
measurement of public sector innovation. This section briefly outlines the major
international and Australian developments.

International Developments
International efforts to collect data on, and develop indicators for, public sector
innovation have been increasing since the mid-2000s. There is no international
agreement, however, about how to define ‘innovation’ as it applies to the public
sector, or how public sector innovation can best be measured.

Research is relatively new and limited, and extensive discussions are taking place
about what types of information (or ‘indicators’) are the most meaningful. DIISR is
monitoring these developments, and actively participates in relevant discussions
through its membership of the OECD and through other channels. Australia aims to
contribute to the international debate and benefits from exposure to international
experience. The potential for international comparability in developing measurement
tools for the Australian public sector is one important consideration in determining the
direction of the APSII Project.

In this new and challenging field of innovation measurement, the APSII Project has
drawn on a number of recent international projects to progress its concepts and
methodologies, including:

• Nordic countries’ Measuring Public Innovation in Nordic Countries:Toward a

Common Statistical Approach (MEPIN) project, 2010 18

18
http://www.mepin.eu/documents/public/News/MEPIN_Final_Report.pdf
• OECD’s Creating Guidelines for the Measurement of Public Sector Innovation
led by the National Experts in Science and Technology Indicators (NESTI)
Taskforce on public sector innovation (on which Australia is represented) 19
• South Korea’s Government Innovation Index (GII) project 20
• the United Kingdom’s Public Sector Innovation Index project, led by the National
Endowment for Science, Technology and the Arts (NESTA) 21
• the European Commission’s European Public Sector Innovation Scoreboard
(EPSIS) project 22

Most international efforts have been designed to contribute to guidelines for

collecting internationally-comparable data. This will allow benchmarking of public
sectors internationally and assist in identifying conditions which encourage
innovation. The development of ‘innovation indicators’ and an ‘innovation index’ is a
common approach with information presented in the form of ‘scorecards’ or
‘scoreboards.’

The information provided by these international projects is assisting to define the

APSII Project’s approach to the development of innovation indicators and the survey
questionnaires. Our approach is unique in that it includes both agency and employee
information. Although a number of countries also conduct public sector employee
surveys which include questions about innovation, other major international public
sector measurement projects tend to focus on information provided by agencies rather
than by employees. Australia is therefore well positioned to contribute to international
efforts in this regard.

Australian Developments
Whilst efforts have been made at all government levels to incorporate innovation
strategies, little progress has been made in the measurement of innovation in the
Australian public sector.

The annual State of the Service surveys conducted by the Australian Public Service
Commission (APSC) already provide a number of measures of innovation, including
those relating to individuals’ preparedness to be innovative and their perception of
agency support for innovation. Its annual State of Service Report provides the results
of data analysis from the surveys.

DIISR, in collaboration with AGIMO 23 , and with input from external stakeholders, is
developing an on-line Public Sector Innovation Toolkit for launch around June 2011.
This is an online resource centre designed to provide easily accessed resources to
support innovation in the public sector and the broader public. It is envisaged that the
Toolkit will provide an avenue for individuals to input information on successful
innovative practices and act as a resource of tools and best practice to help individuals
imbed innovation in their work practices. The APSII Project will inform the Toolkit in
relation to measurement reporting and evaluation.

19
NESTI (2010) Creating guidelines for the measurement of public sector innovation. OECD Cote:
DSTI/EAS/STP/NESTI (2010) 19
20
http://unpan1.un.org/intradoc/groups/public/documents/un/unpan022718.pdf
21
http://www.nesta.org.uk/library/documents/Innovation_in_public_sector_organisations_v9.pdf
22
http://ec.europa.eu/research/innovation-union/pdf/iu-scoreboard-2010_en.pdf
23
The Australian Government Information Management Office
Despite these initiatives, what is lacking is a robust method of measuring innovation
in the Australian public sector. The APSII Project is a response to that need and offers
an opportunity to develop a reliable objective measurement framework.