7E2-2
Concept of Economic Readiness Levels Assessment
Yuniaristanto
Industrial Engineering Department,
Sebelas Maret University
Surakarta, Indonesia
yuniaristanto@ft.uns.ac.id
Anugerah Widiyanto
Technology Incubation Center, Agency for the Assessment
and Application of Technology
Jakarta, Indonesia
anugerah.widiyanto@bppt.go.id
Abstract - This research aims to build a concept of Economic
Readiness Level (ERL) assessment for incubation center. ERL
concept is arranged by considering both market and business
aspects. Every aspect is divided into four phases and each of
them consists of some indicators. Analytic Hierarchy Process
(AHP) is used to develop the ERL in calculating the weight of
every single aspect and indicator. Interval scale between 0 and 4
is also applied in indicator assessment. In order to calculate
ERL, score in every indicator and the weight of both the aspect
and indicator are considered. ERL value is able to show in detail
the innovative product readiness level from economic sight,
market and business aspect. There are four levels in Economic
Readiness Level scheme which are investigation, feasibility,
planning and introduction.
Keywords - Economic Readiness Level, business aspect, market
aspect, AHP, interval scale
I. INTRODUCTION
University as research center is considerable in promoting
technology based economy growth [1]. Specifically,
university recently has a responsibility to commercialize
research output as a support in economy development [2].
Technology is a driving key to innovate and grow business as
a tremendous number of resources are utilized by research and
development unit either from private or government
institution.
Notwithstanding the great number of research output in
university, commercialization attempt in product
dissemination so that people can utilize the output and it
provides economics benefit is still minimum. Basically,
technology commercialization in university environment is
one of Three Pillars (Tri Dharma) of Higher Education, which
is community dedication. Technology commercialization
inevitably requires an appropriate mechanism in order to
avoid the research output from the valley of death [3].
Technology commercialization is one of format and
activity to transfer the technology innovation to a business
world. With the presence of technology commercialization
process, technology can provide benefit for the inventor by
978-1-5090-3803-9/16/$31.00 ©2016 IEEE
2016 Annual Conference on Industrial and System Engineering (ACISE)
Wahyudi Sutopo
Industrial Engineering and Techno-economics Research
Group, Sebelas Maret University
Surakarta, Indonesia
wahyudisutopo@gmail.com
Arinda Soraya Putri
Laboratory of Business and Logistic System,
IE Department, Sebelas Maret University
Surakarta, Indonesia
arindasoraya@gmail.com
applying it in either production or consumption activity [4]. In
accordance with technology commercialization activity, a
service unit called as technology transfer service unit is
required as support. The unit tends to prepare technology
innovation of a research output for commercialization in
business world.
Technology commercialization and readiness of the
technology as a research output to be acceptable and usable by
user are inseparably linked. Therefore, some tools are
available to assess the readiness level of a product from a
research. Some of them have been commonly used in
assessing the product readiness level. They are Technology
Readiness Levels, System Readiness Levels, Innovation
Readiness Levels and Demand Readiness Levels.
Technology Readiness Levels (TRL) reflects trustworthy
and feasibility of the technology to be diffused, but not
indicating the competency of concerned research and
development officer nevertheless. TRL concept is developed
as a tool to assist the technology development and monitoring
activity. In 1980 NASA created this concept in order to help
management making decision about technology development
and transition [5].
Limitation and drawback of TRL occur during the practice
[6]. Technology failure to integrate to the existing system may
either confine or postpone technology introduction and
implementation. The circumstance encourages a developed
concept of System Readiness Level (SRL) to emerge which is
proposed by Sauser et. al. [7]. SRL is a scale that combines
maturity level of prior component and operational ability
between components in a particular system.
Tao, et. al. [8] view that it is necessary to improve
managerial approach for innovation through Innovation
Readiness Levels (IRL) concept. IRL concept provides
monitoring and control improvement based on a framework
which illustrates an innovation development during the life
cycle. IRL is a process of identifying the five key aspects of
innovation-technology, market, organization, partnership and
K-7
7E2-2
risk-which influence the effectiveness of innovation
implementation during its life cycle.
Paun defines DRL as a new measurement to assess
demand maturity from potential innovation actor towards
conceptualization stage from appropriate market needs [9]. As
the result, the scientific research team is able to suggest a
solution from scientific research output by technology transfer
process or translating demand to a Research and Development
project. In order to provides essential decision-making
information and predicts technology implementation and
commercialization condition, Paun use two reference systems
which combine Push Technology and Market Pull system.
However, none of the four assessment tools has reviewed
the readiness of technology product from economy side. It is
necessary to know the economic value of the product, product
demand, and consumer respond regarding to the product.
Hence a study related to economic readiness level for
incubated product by incubator institution is necessary.
This study aims to construct Economic Readiness Level
(ERL) for a technology incubation center. The result is an
ERL assessment concept for research output product that will
be incubated by an incubator. This article starts with
introduction then continue with literature review about
readiness level assessment models that have been developed
by previous research in section 2. In section 3, research
methodology explains about how the research is conducted.
Next in section 4 results and discussion about ERL model
framework. Finally, conclusion is explained in final section.
II. LITERATURE REVIEW
This section elaborates several concepts used in this study.
A. The Product Life Cycle
Technology has an observed life cycle as growth, maturity
and declining phase with S-curve character [10]. Product life
cycle can be illustrated by S-curve as seen in Fig. 1.
Fig 1. Product Life Cycle [11]
On the perspective of selling against time, S-curve can be
classified into four stages:
1) Introduction: product market introduction begins by
quality and function test of the new product when it is
introduced to prospective customers. There is not any
product in market introduction stage which is fully-
accepted by consumer without any rejection.
2016 Annual Conference on Industrial and System Engineering (ACISE)
2) Growth: when a new product is being marketed, gradually
being accepted by consumer and gaining increasing profit.
As a product become more successful in market, more
companies start to develop similar product and technology.
On the other hand, the company will keep develop their
product to satisfy the dynamic customer needs in order to
escalate their competitiveness level.
3) Maturity: product quality and reliability reach the peak
during this period. Company will earn enormous amount
of profit, however the profit growth is slowing down. A
few number of product brands are to dominate the market.
4) Decline: new generation of the product is going to appear.
Most products will lose their competitiveness due to the
presence of those mainstream products. Price competition
will arise during this period.
B. Technology Readiness Levels
Technology Readiness Levels (TRL) is a systematic
system which support assessment of certain technology
maturity and consistent comparison of maturity between
various types of technology [12]. TRL concept is developed
as a tool in assisting technology monitoring and development.
This concept was formulated by NASA in 1980 to help
management in decision making policy regarding to
technology development and transition [5].
Technology Readiness Levels Summary [13]:
TRL 1 Basic principles observed and reported
TRL 2 Technology concept and/or application formulated
TRL 3 Analytical and experimental critical function and/or
characteristic proof-ofconcept
TRL 4 Component and/or breadboard validation in laboratory
environment
TRL 5 Component and/or breadboard validation in relevant
environment
TRL 6 System/subsystem model or prototype demonstration
in a relevant environment
TRL 7 System prototype demonstration in a operational
environment
TRL 8 Actual system completed and “flight qualified”
through test and demonstration
TRL 9 Actual system “flight proven” through successful
mission operations
C. System Readiness Levels
Just like the other management tools, there is limitation in
utilization of TRL [14] [15]. Technology failure to integrate to
the existing system may either confine or postpone technology
introduction and implementation. Baines [16] explains that
wrong technology or appropriate technology with bad
implementation can be a disaster.
Sauser, et. al. [7] propose to develop system called System
Readiness Levels (SRL) to respond these circumstances. SRL
combines maturity level of main components and operational
ability between components in a particular system. SRL
indexes are defined based on condition of system
development in accordance with Development Phase from
K-8
7E2-2 2016 Annual Conference on Industrial and System Engineering (ACISE)

Department of Defense about Life Cycle Management In order to provides essential decision-making information
Framework as shown in Table I [17]. and predicts technology application and commercialization
condition, Paun use two reference systems which combine
TABLE I. SYSTEM READINESS LEVELS “Push Technology” and “Market Pull” system. Table III
SRL Name Definition shows the scheme suggested by Paun [9].
5 Operations & Execute a support program that meets operational
Support support performance requirements and sustains TABLE III. DRL-TRL SCHEME
the system in the most cost-effective manor over
its total life cycle DRL Description of DRL Description of TRL TRL
4 Production & Achieve operational capability that satisfies 1 Occurrence of a Feeling “something is Market certification and 9
Development mission needs. missing” sales authorisation
3 System Develop a system or increment of capability; 2 Identification of a specific need Product Industrialisation 8
Development & reduce integration and manufacturing risk; ensure 3 Identification of the expected Industrial Prototype 7
Demonstration operational supportability; reduce logistics functionalities for the new
footprint; implement human systems integration; Product/Service
design for producibility; ensure affordability and 4 Quantification of the expected Field demonstration for 6
protection of critical program information; and functionalities the whole system
demonstrate system integration, interoperability, 5 Identification of the systemic Technology development 5
safety, and utility. capabilities (including the project
2 Technology Reduce technology risks and determine leadership)
Development appropriate set of technologies to integrate into a 6 Translation of the expected Laboratory 4
full system. functionalities into needed capabilities demonstration
1 Concept Refine initial concept. Develop system/ to build the response
Refinement technology development strategy 7 Definition of the necessary and Research to prove 3
sufficient competencies and resources feasibility
D. Innovation Readiness Levels 8 Identification of the Experts possessing Applied research 2
the competencies
Tao, et. al. [8] view that it is necessary to improve 9 Building the adapted answer to the Fundamental research 1
managerial approach for innovation through Innovation expressed need on the market
Readiness Levels (IRL) concept. IRL concept provides
monitoring and control improvement based on a framework None of the four assessment models have considered
which illustrates an innovation development during the life economy aspect. It is necessary to consider economy aspect
cycle. IRL is a process of identifying the five key aspects of especially both market and business aspects. The position of
innovation-technology, market, organization, partnership and the current study is described in Table IV.
risk-which influence the effectiveness of innovation
TABLE IV. RESEARCH POSITION
implementation during its life cycle.
Aspects
IRL framework [8] is six of ‘C’ model separating Assessment
Technology Market Business
Levels
comprehensive innovation life cycle to be six phases TRL  1-9
(readiness level), and examine innovation process SRL  1-5
management by considering five key aspects (see Table II). IRL   1-6
DRL   1-9
ERL   1-4
TABLE II. IRL FRAMEWORK
Technological development Market evolution III. RESEARCH METHODOLOGY
Phase IRL1 IRL2 IRL3 IRL4 IRL5 IRL6
Concept Components Completion Chasm Competition Changeover Economic Readiness Levels assessment concept is
Aspects /Closedown constructed in order to examine both market and business
Technology TRL, SRL
Market Diffusion of innovation theory aspects based on commercialization model by Goldsmith [18].
Market adoption model Research method in this study consists of four stages refers
Product life cycle of innovation
Organization
ERL model scheme formulation as shown in Fig. 2.
Partnership Key activities
Risk Indicators Building Indicator Testing

E. Demand Readiness Levels Define and describe phase • Build questionnaire I

and measurement indicator • Validity-reliability testing
Paun together with ONERA (Space Laboratory of France)
[9] suggest additional scale to complete TRL thus introduce
Demand Readiness Level (DRL) that related to maturity level
of a particular innovation market demand. Paun defines DRL Assessment AHP
as a new measurement to assess demand maturity from Determine scale and • Build hierarchy
potential innovation actor towards conceptualization stage assessment • Pairwise comparison
from market needs [9]. As the result, the scientific research • Consistency testing
• Weighting
team is able to suggest a solution from scientific research
output by technology transfer process. Fig 2. Steps of process in ERL Model Formulation

K-9
7E2-2 2016 Annual Conference on Industrial and System Engineering (ACISE)

First step is to define and describe phases and indicator TABLE V. MARKET ASPECT: PHASE AND INDICATOR
based on literature and stakeholder needs. Commercialization Phase Indicator Source
model of Goldsmith [18] is used as the reference in MA1 Market potency
determining the phases in market and business aspect. Those MA2 Product uniqueness
phases are explained into measurable indicators. Preliminary MA3 Sales growth
verification by focus group discussion is done after that. Market MA4 Competitive level [18] [19]
Assessment
In the second step, first questionnaire is built based on MA5 Competitor price level [20]
(MA)
indicators that have been determined before. The MA6 Product acceptance by consumer
questionnaire should be well-arranged to avoid bias and MA7 Facility of market access
ambiguous sentences. A matching understanding of MA8 Distribution channel availability
respondents towards the meaning of each question in MS1 Market segmentation
questionnaire is a compulsory. The questionnaire is MS2 Industrial and economical trend
distributed to respondents including incubation center, tenant, MS3 Market volume
and start-up entrepreneur. Returning questionnaires from Market Study MS4 [18] [21]
Sales level prediction
(MS) [22] [23]
respondent have validity and reliability testing later. The MS5 Price target determination
purpose is to ensure that chosen indicators is appropriate and MS6 Distribution channel selection
consistent against the concept that will be measured. MS7 Competitor characteristic
MP1 Marketing team selection
A hierarchy consisting aspects, phases and indicators is
constructed in the next step. The second questionnaire is MP2 Target market determination
Strategic Market MP3
arranged by using pairwise comparison for every existing Distribution channel determination [18] [24]
Plan (MP)
indicator. The same respondents with respondents in first MP4 Consumer preliminary evaluation
questionnaire are asked to fill in the second questionnaire after MP5 Production strategy determination
it is distributed. Consistency data from respondents is MV1 Distribution channel evaluation
examined by pairwise comparison matrices. Once all of the Market MV2 Initial product sales
matrices are consistent, weight of each indicator and global Validation MV3 Sales outcome evaluation [18]
weight for every aspect can be calculated. (MV) MV4 Buyer respond evaluation
MV5 Marketing strategy evaluation
Eventually, assessment by determining scale and range of
scale is condusted in final step. Score of each determined
TABLE VI. BUSINESS ASPECT: PHASE AND INDICATOR
indicator is measured then. Based on these weight and score,
ERL score for the observed product can be assessed. Phase Indicator Source
BA1 Business profit potency
IV. RESULT AND DISCUSSION BA2 Readiness of product concept
Venture commercialization
This section consists of four stages which every stage will Assessment BA3 Human resources requirement [18] [25]
be elaborated in more detail in these sub sections. (BA)
BA4 Business capital requirement
A. Indicator Building BA5 Intellectual property potency
BF1 Set of financial assumption
Indicators are built by sequential process starting from
Economic BF2 Capital resource
aspect determination, phase determination and indicator Feasibility [18] [26]
BF3 Business feasibility analysis
building. Goldsmith model [18] is used as a basis on aspect (BF)
and phase determination. That model actually views from BF4 Financial ratio evaluation
three aspects: technology, market and business. Due to the BP1 IPR finalization
goal of this study-construct economy readiness levels model-it BP2 Business format
only consider market and business aspects. Goldsmith model BP3 Business organization structure
Strategic BP4 Board of director selection
also consists of six sequential phases but we apply only four Business (BP) [18]
phases with a reason that incubation process just cover four BP5 Management team selection
phases including investigation, feasibility, development and BP6 Business budgeting
introduction phases without covering the commercial phase. BP7 Business planning formulation
BS1 Organization job function
Based on Goldsmith [18], market aspect is divided into BS2 Employee recruitment
four phases which are market assessment, market study,
BS3 Employment contract
strategic market plan and market validation. For each phase, Business Start- BS4 Initial capital preparation [18] [27]
indicators are built to assess an innovative product readiness Up (BS)
BS5 Policy and SOP formulation
from market side. There are eight indicators in market
BS6 Financial control mechanism
assessment phase and seven indicators in market study while
BS7 Target achievement evaluation
strategic market plan and market validation phase have five
indicators for each of them. The list of phase, indicator and
reference of market aspect can be seen in Table V.

K-10
7E2-2 2016 Annual Conference on Industrial and System Engineering (ACISE)

So do with business aspect, it can be divided into four Hierarchy shows the influence from highest level of
phases which are venture assessment, economic feasibility, purpose to lowest level. A hierarchy can also be used in
strategic business and start-up business. Then, indicators are decomposing a complex problem so that the problem will be
built to assess an innovative product readiness from business more structured and systematic. Hierarchy structure used in
side for every phase. Venture assessment phase consists of this study is illustrated in Fig. 3.
five indicators; economic feasibility has four while strategic
business and start-up business has seven indicators ERL
respectively. The list of them and the reference is shown in
Table VI. Market Business
B. Indicator Testing
This sub-section begins with questionnaire compilation MA MS MP MV BA BF BP BS
based on chosen indicator and will be distributed by email.
There are some advantages in using questionnaire as the data MA1 MS1 MP1 MV1 BA1 BF1 BP1 BS1
MA2 MS2 MP2 MV2 BA2 BF2 BP2 BS2
collection method such as quick, economic and simple to be MA3 MS3 MP3 MV3 BA3 BF3 BP3 BS3
analyzed [28]. Data collection can be done by various MA4 MS4 MP4 MV4 BA4 BF4 BP4 BS4
MA5 MS5 MP5 MV5 BA5 BP5 BS5
methods like interview either in face to face or telephone, MA6 MS6 BP6 BS6
questionnaire and combination of both methods [29]. MA7 MS7 BP7 BS7
MA8
The questionnaire is distributed to desire respondent with
sufficient sample size for systematic examination. In this case, Fig 3. ERL Hierarchy Model
purposive sampling method is selected to determine the
respondents. Purposive sampling is sample selection method 2) Pairwise Comparison
based on knowledge regarding to population, element and Pairwise comparison is used to measures relation between
research goal [30]. Sample is chosen non-randomly based on aspects, phases and indicators. The second questionnaire is
certain characteristics [31]. The obligatory characteristic of prepared by making pairwise comparison for each existing
respondent in this study is understanding technology indicator and distributed to be filled in by respondents
commercialization and incubation process in a business afterwards. For instances there is a hierarchy with criteria of
incubation center. The survey is done comprising throughout ‘C’ and as many as ‘n’ alternatives below: Ai to An.
Indonesia. In this study respondent consists of incubation Comparison between alternatives for that hierarchy can be as
center, tenant and start-up entrepreneur. Especially for start-up n × n matrices format as seen in Table VII.
entrepreneur, it is chosen for those who has been incubated in
TABLE VII. PAIRWISE COMPARISON MATRICES
a business incubation center
A1 A2 ... An
Validity and reliability testing for assessment tool is done
A1 a11 a12 ... a1n
afterwards. Validity testing determines whether the research
measures correctly what the researcher want or not [32]. Kirk A2 a21 a22 ... a2n
and Miller identify three kinds of reliability in quantitative ... ... ... ... ...
study which related to: (1) consistency of measurement result An an1 an2 ... ann
when it is given repeatedly (2) stability of measurement from
time to time and (3) similarity of measurement for a given Meanwhile, Table VII explains comparison scale used to
time period [33]. determine numeric value for all comparison.

Validity testing used in this study is convergent validity TABLE VIII. SCORING VALUE OF PAIRWISE COMPARISON [35]
with correlation analysis tool. Meanwhile, the reliability Intensity of Definition
testing uses inter-item consistency reliability with Cronbach Importance
alpha analysis tool. It aims to give conviction that assembled 1 Equal importance
indicator is appropriate and consistent toward concept that
2 Weak
will be assessed. 3 Moderate
C. Analytic Hierarchy Process importance
4 Moderate plus
Analytic Hierarchy Process (AHP) is measurement theory 5 Strong importance
by using pairwise comparison and depending on examination
from experts in order to get scale of priority [34]. Comparison 6 Strong plus
7 Very strong or
is made by using absolute assessment scale that represent how demonstrated
powerful an element can dominate another element related to importance
a particular attribute. Here are the stages used in AHP [35]: 8 Very, very strong
9 Extreme importance
1) Hierarchy Building
Explanation
Two activities contribute equally to the
objective
Experience and judgment slightly favor
one activity over another
Experience and judgment strongly favor
one activity over another
An activity is favored very strongly over
another; its dominance demonstrated in
practice
The evidence favoring one activity
over another is of the highest
possible order of affirmation

K-11
7E2-2 2016 Annual Conference on Industrial and System Engineering (ACISE)

3) Consistency Checking Sjk = score in phase j and aspect k

Consistency measurement of AHP is done by measuring ωijk = weight in indicator i, phase j, aspect k
every pairwise comparison matrices consistency. This ωk = weight of indicator in aspect k
measurement based on eigenvalue maximum (λmax). σijk = score in indicator i, phase j, aspect k
A.w = λmax.w (1) i = indicator 1, 2, ..., I
CI = (λmax-n) / (n-1) (2) j = phase 1, 2, ..., J
CR = CI/RI (3) k = aspect 1, 2
Annotation:
w : eigenvector ERL level will be in range between 0 and 4 which has own
λ : eigenvalue definition in each level as the Goldsmith [18]. They are: ERL-
A : pairwise comparison matrices 1: investigation; ERL-2: feasibility; ERL-3: planning and
λmax : eigenvalue maximum ERL-4: introduction.
n : matrices order V. CONCLUSION
CI : consistency index
CR : consistency ratio Research output is an assessment concept of Economic
RI : random index Readiness Levels for research product result that will be
incubated by incubator. Refer to Goldsmith model [18],
Importance level assessment by Saaty [12] is said to be market and business aspect take into consideration in ERL.
consistent if consistency ratio value (CR) < 0.1. CR ≤ 0.2 is Constructed ERL model consider the weight of every aspect,
tolerated, however it can not be more than 0.2. indicator and score for each indicator. Final result of ERL is
able to shows level of product readiness from economic side
TABLE IX. RANDOM INDEX (RI) LIST
in more detail, in this case they are market and business
N 1 2 3 4 5 6 7 8 9 10 aspect. Economic Readiness Level is divided into four levels:
RI 0 0 0.52 0.89 1.11 1.25 1.35 1.40 1.45 1.49 investigation, feasibility, planning and introduction.

4) Weighting Result
Consensus weight (priority weight) is obtained from the
assessment of all respondents once the geometric average
value and priority vector is calculated. Meanwhile, global
weight is resulted from multiplication between priority weight
from each aspect, phase and indicator.
Future research may determine the weight of each aspect
and indicator by conducting a survey research to incubation
center, tenant and start-up entrepreneur. The obtained weight
from that research can be reference of ERL assessment from a
certain innovative product that either will be or is incubated in
incubation center.

D. Assessment
W. Nolte together with his team in Air Force Research
Laboratory developed TRL calculator [36]. TRL calculator
both simplifies process in determining maturity level of
technology and makes the process is possible to be repeated
with the presence of a series of questions for every user [36].
In 2006, BPPT and Ministry of Research and Technology
of Indonesia created Techno-meter as a result of modification
of TRL calculator developed by AFRL and has been adapted
with condition in Indonesia [37]. On every TRL level, there
are a series of indicators that scored by six standards which
are: 0 = not fulfilled; 1 = 20% fulfilled; 2 = 40% fulfilled; 3 =
60% fulfilled; 4 = 80% fulfilled and 5 = 100% fulfilled.
Interval scale is used in this study as the measurement
scale. Interval scale is able to shows difference, order and
similarity of difference range from a particular variable [38].
Five standard scales are used to assess every indicator which
are 0 = not fulfilled; 1 = 25% fulfilled; 2 = 50% fulfilled; 3 =
75% fulfilled and 4 = 100% fulfilled.
Last step is construction of assessment concept by ERL as
these formulations below.
Sjk = (∑i ωijk . σijk)/J, each j,k (4)
ERL = ∑k (ωk . ∑j Sjk) (5)
Annotation:
ACKNOWLEDGMENT
This research is supported by Research Scheme of
Mandatory Research Grant with funding source from PNPB
Sebelas Maret University – Finance Year of 2016, Contract
No: 632/UN27.21/LT/2016, April 24th, 2016.
REFERENCES
[1] B. P. Clayman and J. A. Holbrook. The Survival of University Spinoffs
and Their Relevance to Regional Development. Canada: Centre for
Policy Research on Science and Technology (CPROST), Simon Fraser
University, 2003
[2] M. C. Leung and J. A. Mathews. Origins and Dynamics of University
Spin-Off Enterprises in Hong Kong. International Journal of Transitions
and Innovation Systems, I (2), pp. 175 – 201, 2011.
[3] K. Ismail, I. A. Majid, and W. Z. W. Omar. The Commercialisation of
University Patents: A Case Study. African Journal of Business
Management, V(17), pp. 7198-7208, 2012.
[4] R. A., Siegel, S., Hansen and L.H., Pellas. "Accelerating the
commercialization of technology: commercialization through
co‐operation", Industrial Management & Data Systems, Vol. 95 Iss: 1,
pp.18 – 26, 1995
[5] S. R. Sadin, F. P. Povinelli and R. Rosen. The NASA technology push
towards future space mission systems. Acta Astronautica, 20, 73-77,
1989
[6] B. Sauser, J. Ramirez-Marquez, R. Magnaye and T. Weiping. Approach
to Expanding the Technology Readiness Level within Defense
Acquisition. International Journal of Defense Acquisition Management.
Vol. 1, pp. 39-58, 2008

K-12
7E2-2
[7] B. Sauser, D. Verma, J. Ramirez-Marquez, and R. Gove. From TRL to
SRL: The concept of systems readiness levels. In Proceedings of the
Conference on Systems Engineering Research. Los Angeles, CA:
CSER, 2006
[8] L. Tao, D. Probert and R. Phaal. Towards an integrated framework for
managing the process of innovation. R&D Management. Vol.40. Issue
1. p. 19–30, 2010.
[9] F. Paun. Demand Readiness Level as equilibrium tool for the
hybridization between Technology Push and Market Pull Approaches.
ANR-ERANET Workshop. 8th February 2011, Paris, 2011
[10] C. M. Christensen. Exploring The Limits of The Technology Curve.
Part I: Component Technologies, Strategic Management of Technology
and Innovation. 4th ed. New York: Mc Graw Hill, 2004
[11] M. C. Lee, T. Chang and W. T. C. Chien. An Approach for Developing
Concept of Innovation Readiness Levels, International Journal of
Managing Information Technology, Vol.3, No.2, 2011
[12] J. C. Mankins. Technology Readiness Levels, A White Paper, Advanced
Concepts Office, Office of Space Access and Technology, NASA, 1995
[13] DoD. Levels of Information systems interoperability. Washington. DC,
1998
[14] R. Gove, Development of an integration ontology for systems
operational effectiveness (Master’s Thesis). Hoboken, NJ: Stevens
Institute of Technology, 2007
[15] J. Mandelbaum. How the S&T community can best support the
technology readiness assessment (TRA) process do’s and don’ts. In
Proceedings of the Technology Maturity Conference. Virginia Beach,
VA: AFRL, 2007
[16] T. Baines. An integrated process for forming manufacturing technology
decisions. International Journal of Operations & Production
Management, 24(5/6), 447-467, 2004
[17] DoD. Chapter 4: Systems Engineering (DoD Directive 5000.2).
Washington. DC, 2005
[18] R. Goldsmith. “Model of commercialisation”, http://asbdc.ualr.edu/
technology/commercialization/the_model.asp, 2003
[19] FAO. Food Security Information for Action. Markets Assessment and
Analysis, 2008
[20] A.O., Miehlbradt. Guide to Market Assessment for BDS Program
Design, Geneva: International Labour Organization, 2001.
[21] X. Luo and C. B. Bhattacharya. Corporate Social Responsibility,
Customer Satisfaction, and Market Value, Journal of Marketing, Vol.
70, No. 4, pp. 1-18, 2006
[22] A. H. Kirca, S. Jayachandran, and W. O. Bearden. Market Orientation:
A Meta-Analytic Review and Assessment of Its Antecedents and Impact
on Performance. Journal of Marketing Vol. 69, 24–41, 2005
2016 Annual Conference on Industrial and System Engineering (ACISE)
[23] D. Mitra and P. N. Golder. Whose Culture Matters? Near-Market
Knowledge and Its Impact on Foreign Market Entry Timing. Journal of
Marketing Research Vol. XXXIX (August 2002), 350–365.
[24] C. Gilligan and R. M. S. Wilson. Strategic Marketing Planning.
Butterworth-Heinemann, 2009
[25] D. A. Shepherd, R. Ettenson and A. Crouch. New venture strategy and
profitability: A venture capitalist's assessment, Journal of Business
Venturing, Vol. 15, 449–467, 2000
[26] M. Jamil, S. Kirmani, and M. Rizwan. Techno-Economic Feasibility
Analysis of Solar Photovoltaic Power Generation: A Review. Smart
Grid and Renewable Energy, Vol. 3, 266-274, 2012
[27] T. Åstebro and I. Bernhardt. Start-up financing, owner characteristics,
and survival. Journal of Economics and Business. 55, 303–319, 2003
[28] A. Bowling, Research Methods in Health, 2nd ed., Buckingham: Open
University Press, 1997.
[29] M. T. Siniscalco and N. Auriat, "Questionnaire Design," in Quantitative
Research Methods in Educational Planning, Paris, UNESCO
International Institute for Educational Planning, 2005.
[30] E. Babbie, Survey Research Methods, Belmont: Wadsworth Publishing,
1990
[31] L. R. Frey, C. H. Botan and G. L. Kreps, Investigating Communication:
An Introduction to Research Methods, 2nd ed., Boston: Allyn and
Bacon, 2000
[32] M. Joppe. The Research Process, 2000 http://www.ryerson.ca/
~mjoppe/rp.htm
[33] J. Kirk and M. L. Miller. Reliability and validity in qualitative research.
Beverly Hills: Sage Publications, 1986
[34] T. L. Saaty, Decision making with the analytic hierarchy process, Int. J.
Services Sciences, Vol. 1, No. 1, 2008
[35] T. L. Saaty and L. G. Vargas. Models, Methods, Concepts &
Applications of the Analytic Hierarchy Process, Second Edition,
Springer Science+Business Media New York, 2012
[36] W. Nolte, B. C. Kennedy, and R. J. Dziegiel. Technology readiness
calculator, NDIA system engineering conference, 2003.
http://www.dtic.mil/ndia/2003systems/nolte2.pdf
[37] BPPT. Guidance of Technology Readiness Levels Assessment: Techno-
meter, Gerbang Indah Nusantara, Jakarta, 2012 (in Indonesia)
[38] U. Sekaran and R. Bougie. Research Methods for Business: A Skill
Building Approach, John Wiley & Sons, 2009
K-13