Internal Metrics and

Benchmarking
New Era Measurement of TTO Performance
and Impact

Speakers
Charles Louis - charles.louis@ucr.edu
Melba Kurman - melbak@triplehelixinnovation.com
 Speakers
• Charles Louis, Vice Chancellor
for Research at the University
of California, Riverside

• Melba Kurman, principal with

Triple Helix Innovation

Webinar: Internal Metrics and Benchmarking 2

 Overview of today’s program
• Metrics: how can university technology
transfer offices best quantify their
performance, impact, and productivity?

– The approach taken by the University of

California
– Adding an additional tool: the technology
transfer health index concept
– Discussion

Webinar: Internal Metrics and Benchmarking 3

 The challenge of benchmarking
innovation intermediaries
TTO metrics should…
• Represent effort, activity and outcomes that
support a single, tightly-defined mission
• Capture outcomes the TTO can actually
affect
• Capture the entire big picture of TTO activity,
time spent, and results
• Provide clear insight into TTO activity &
performance
Webinar: Internal Metrics and Benchmarking 4
 University of California’s technology
transfer metrics

1. Define success criteria for the University's technology transfer

program, taking into consideration the documented objectives
of the program and the different stages of development of the
different campus technology transfer offices.

2. Identify and describe metrics for measuring these criteria.

3. Advise on the scope, format, and frequency of reports of the

university’s technology transfer program to both the public
and to internal stakeholders.

Webinar: Internal Metrics and Benchmarking

 University of California metrics committee

Within UC, at least four stakeholders were identified

by the directors of the campus technology programs:

1. The academic community of faculty and students.

2. State and local government.
3. Industry partners.
4. The public.

Webinar: Internal Metrics and Benchmarking

 Criteria for inclusion of a metric to
be used and reported
• Perceived value of the metric.

• Is the data for the metric currently captured in a

database.

• If the data is not currently captured in a database, what

is the expected level of difficulty in capturing the data.

• If the level of difficulty is deemed to be high, then the

perceived value also must be high if it is to be collected.

Webinar: Internal Metrics and Benchmarking 7

 Five identified goals for the UC
campus Tech Transfer programs

1. Create public benefit.

2. Provide service to the UC academic community.

3. Create, support and maintain research

partnerships with industry.

4. Support economic development.

5. Secure fair compensation for UC technologies.

Webinar: Internal Metrics and Benchmarking
 Goal #1. Create public benefit
• The public benefits when the innovations and
discoveries of the faculty and staff of the University of
California are made available to the public and
commercial sectors.

• Transferring university research outside of the

institution enables further research, the creation of new
companies, and the development of products and
services.

• By facilitating this transfer, Technology Transfer Offices

are instrumental in promoting innovation and public
benefit.
Webinar: Internal Metrics and Benchmarking
 Data collected to measure creating public
benefit

1. Number of tech transfer agreements.

2. Number of Start-Up Companies.
3. Number of products on market.

Webinar: Internal Metrics and Benchmarking 10

 Goal #2. Provide service to the UC
academic community
• Technology Transfer Offices serve the academic
community in all types of intellectual property related
activities such as the transfer of proprietary materials
into and out of the university.

• Provides policy guidance, mentoring and education

about protecting and commercializing intellectual
property.

• Supports faculty, postdocs and graduate students who

have the desire, but not necessarily the experience to
champion the development of promising early stage
technology and to start new ventures.
Webinar: Internal Metrics and Benchmarking
 Data collected to measure providing
service to the academic community
1. Number of disclosures.
2. Number of individuals disclosing for first time.
3. Number of individuals disclosing more than one
time.
4. Number of interactions.
5. Number of individuals receiving income from tech
transfer agreements.
6. Number of Material Transfer Agreements.

Webinar: Internal Metrics and Benchmarking 12

 Data collected to measure providing
service to the academic community (cont.)
7. Number of copyright licenses.
8. Number of visitor agreements with industry for
which the tech transfer office provided input.
9. Number of patents filed.
10. Number of issue patents.
11. Number of researcher-involved start-up
companies.
12. Results of customer satisfaction surveys.

Webinar: Internal Metrics and Benchmarking 13

 Goal #3. Create, support and maintain
research partnerships with industry
• Research is a primary mission of the University of
California.

• Partnerships with industry allow for the expertise of

university researchers to be focused on solving
problems/questions of interest to industry and also allow
for innovations resulting from these interactions to be
made available to industry.

• Technology Transfer Offices enhance research

partnerships by offering university intellectual property to
support and expand interactions with industry.
Webinar: Internal Metrics and Benchmarking
 Data collected to measure research
partnerships with industry
1. Number of tech transfer agreements resulting
from industry research.
2. Number and dollar value of research agreements
resulting from tech transfer agreements.
3. Number of agreements for which the tech
transfer office provided input.
4. Number of visitor agreements with industry for
which the tech transfer office provided input.

Webinar: Internal Metrics and Benchmarking 15

 Goal #4: Support economic development
• Technology-driven businesses create new jobs and
drive regional and national economic
development.

• Effective technology transfer from land grant

universities is an essential first step in creating
new companies employing highly skilled graduates
and contributing to the economy through
increases in the tax base and the purchase of
goods and services.

Webinar: Internal Metrics and Benchmarking

 Data collected to measure support of
economic development
1. Number of Start-up companies.
2. Number of jobs created by start-up companies.
3. Funding of start-up companies.
4. Sales revenue of start-up companies.
5. Total gross sales reported by licensees .
6. Number of products on market.
7. Number of individual companies with a tech
transfer agreement .

Webinar: Internal Metrics and Benchmarking 17

 Goal #5. Secure fair compensation for UC
technologies
• Technology Transfer Offices are the stewards of public
intellectual property assets.
• Accordingly, they seek to secure fair compensation for
their available technologies keeping in mind the need to
provide service to all of the stakeholders involved in the
tech transfer process; namely, industry, university
researchers and the public.
• Revenue generated from the technology transfer program
is invested in the university’s education and research
programs.
Webinar: Internal Metrics and Benchmarking 18
 Data collected to measure fair
compensation for UC technologies
1. Royalty and fee income.
2. Income from the sale of equity.
3. Number of tech transfer
agreements generating income.
4. Reimbursement of patent costs.

Webinar: Internal Metrics and Benchmarking 19

 UC metrics project conclusions
• This 2009 report has proven a valuable resource for enhancing the
reporting of technology transfer activity of the campuses.

• Some of the metrics may best serve as a management tool rather than
published metrics – especially of value for newer offices.

• In spite of this document, many senior administrators and faculty still

perceive that the primary goal their TTO is #5, generating revenue, not
recognizing the many other, non-monetary services that they provide to
the researchers on their campuses. Certainly a common view of faculty
who do not interact with their TTOs.

• A number of national organizations have a similar opinion where they

compare licensing income $/sponsored funding award $ as the metric for
evaluating TTO performance.

• This report provides a broader discussion of this agenda where TTOs are
seen as a service to support the research mission of the University of
California.
Webinar: Internal Metrics and Benchmarking 20
 Speaker Melba Kurman, principal at Triple Helix
Innovation

melbak@triplehelixinnovation.com

Webinar: Internal Metrics and Benchmarking 21

 The technology transfer health index

A technique & tool that can

• Capture the entire big picture of TTO

activity, time spent, and results
• Provide clear insight into TTO activity
& performance

Webinar: Internal Metrics and Benchmarking 22

 What is the tech transfer health
index?

The tech transfer health index quantifies the

impact and productivity of the entire long
tail curve of technologies in a university’s
IP portfolio.

Webinar: Internal Metrics and Benchmarking 23

 Three universities
Each university has 1000 patents and total
revenue of $10M.
•University A: 2 patents earn $5M each
•University B: 100 patents earn $100K each
•University C: 1000 patents earn $10K each

…Which university has the healthiest IP

portfolio?
Webinar: Internal Metrics and Benchmarking 24
 Three universities
Each university has 1000 patents and total
revenue of $10M.
•University A: 2 patents earn $5M each
– Too risky
•University B: 100 patents earn $100K each
– Well-balanced and diversified
•University C: 1000 patents earn $10K each
– Nothing is bearing fruit

Webinar: Internal Metrics and Benchmarking 25

 Simple tallies don’t tell the whole
story

Webinar: Internal Metrics and Benchmarking photo by Petr Jan Juračka 26

 Using the tech transfer health index
Quantifying the revenue earned per patent

Webinar: Internal Metrics and Benchmarking 27

 Three universities
Each university has 1000 patents and total
revenue of $10M.
•University A: 2 patents earn $5M each
– Health index = 2
•University B: 100 patents earn $100K each
– Health index = 100
•University C: 1000 patents earn $10K each
– Health index = 10

Webinar: Internal Metrics and Benchmarking 28

 Comparing small universities to large
ones
Simple tallies discriminate against universities
that have small or newer tech transfer
offices.

Simple tallies favor universities that have large

budgets and large research expenditures

The tech transfer health index corrects for size,

age, budget and other variables
Webinar: Internal Metrics and Benchmarking 29
 Large, medium and small universities
Three universities, each with 1000 patents and total
revenue of $10M.
•University X: health index = 200. Total research exp. =
$500M
•University Y: health index = 100. Total research exp. =
$200M
•University Z: health index = 20. Total research exp. =
$100M

Which university is doing better, given their resources?

Webinar: Internal Metrics and Benchmarking 30

 Answer
Three universities, each with 1000 patents and
total revenue of $10M.
•University X: health index = 200. Total research
exp. = $500M
– 200/500 = 0.4
•University Y: health index = 100. Total research
exp. = $200M
– 100/200 = 0.5
•University Z: health index = 20. Total research
exp. = $100M
– 20/100 = 0.2

Webinar: Internal Metrics and Benchmarking 31

 Other ways to apply the health index
• Licensing rate
• FTEs per startup

Other categories for the health index

• Per TTO staff member
• Per department or college
• Per regional cluster

Webinar: Internal Metrics and Benchmarking 32

 Analyze your own data
Download the tech
transfer health index
calculator at

http://triplehelixinnovation.com/
measuring-performance-with-the-
tech-transfer-health-index/677

Webinar: Internal Metrics and Benchmarking 33

 Show stakeholders the entire iceberg

photo by Ludovic Hirlimann on Flickr

Webinar: Internal Metrics and Benchmarking 34
 The tech
transfer health
index
overcomes the
limitations of
current metrics

Webinar: Internal Metrics and Benchmarking Photograph by Dennis Mojado. 35

 Pulling it all together
Quantifying the activity, performance and impact
of an innovation intermediary is tough work
Education of stakeholders is key
It’s possible and some TTOs such as U Cal system
are leading the way
Additional tools such as the tech transfer health
index introduce additional insight and enable
comparison across different universities

Webinar: Internal Metrics and Benchmarking 36

 Discussion & questions
Additional comments or questions?
Contact

Charles Louis - charles.louis@ucr.edu

or
Melba Kurman -
melbak@triplehelixinnovation.com

Webinar: Internal Metrics and Benchmarking 37