Scenario-Based Software Operational Profile
Qi Ao, Jun Ai, Minyan Lu, Fangling Zhong
School of Reliability and System Engineering,
Beihang University,
Beijing, China
doublenumber@dse.buaa.edu.cn
Abstract—As the diversity of software status and the
uncertainty of software operation, the traditional methods to
construct software reliability operational profile are difficult
to be applied to reliability testing. This paper introduces an
approach to generate operational profile based on scenario. It
partitions the usage of software into various paths of software
running according to the users and missions. With the
proposal of scenario, a relation between user's activities and
software's execution is established. Through the correlation
which allows software inputs to be obtained from system
usage, the generation of reliability test data could be
implemented. The effect of this approach is illustrated with
an example of missile control demonstration software
(MCDS).
Keywords—Software reliability; operational profile;
scenario
I. INTRODUCTION
With the development of computer technology, the
system is more and more complex. System reliability is a
critical measure of complicated system during the
operational phase. In order to improve software reliability,
software reliability test techniques have been developed,
and gradually applied to the actual products. A major way
to generate the software reliability test cases is operational
profile, which reflects the actual usage information of
product. Operational profile is a set of software operations
with their probability. The input of software is from the
complex random factors, operational profile is a description
of the entire process of use.
For the complex systems, especially real-time
embedded systems, not all the input of software is derived
from the user’s operation (e.g., the input data of some
control software are from the temperature and pressure of
system operating environment). Even from the same
operation, the input data of software also have correlation.
Traditional Musa operational profile does not consider the
impact of non-users’ factors on the software input [1]. The
definition of operation is limited. In addition, the
interdependence between the input variables is ignored. In
Markov operational profile, the software’s state is divided
on the basis of the operation, which has the same limitation.
Furthermore, only from the software point of view, it is
difficult to determine the state transition probability [2].
There are some other methods of conducting operational
978-1-61284-666-8/11$26.00 2011 IEEE
profile in [3] and [4]. The profile is only applied to a
particular type of software and difficult to describe the
software input with the distribution of usage in complex
systems. In [5], the software reliability model does not
consider the links and constraints between the input data of
software. In [6], the approach does not consider the effect
on the software when the system is used.
Because of diversity of the source of software input,
correlation between input data and difficulty in determining
the probability of software’s usage, the common
operational profile methods are difficultly applied to actual
project. In this paper, the new approach is illustrated with
an example of missile control demonstration software
(MCDS). It can be concluded that the introduction of
scenario can be more effective on the reliability testing of
complicated system.
The rest of the paper is organized as follows: Section 2
presents the definitions related to scenario and operational
profile. Section 3 describes the procedure of building the
new operational profile in more detail. Section 4 gives an
example to illustrate our approach. Section 5 concludes the
paper.
II. DEFINITION
There is no standard definition of the scenario [7]. It can
be defined from the system-level and software-level.
Definition 1: The system-level scenario is defined as a
set of the interactions between multiple systems, denoted
by Ssym = {Si | i = 1, 2,..., n} . The software-level scenario is
defined as a set of the interactions between the components
within the system, denoted by Ssw = {si | i = 1, 2,..., m} .
Scenario has the following characteristics:
• The system-level scenario is a direct reflection of
usage of product. The software-level scenario is a
description of the execution of software.
• The scenario reflects the completion of specific
task of system and software. The end of scenario
means that the termination of usage.
Definition 2: The scenario object is defined as the
systems and components which interact with each other on
700
the scenario. On the system-level scenario the objects is
denoted by Osym = {Oi | i = 1, 2,..., n} . On the software-level
scenario the objects is denoted by Osw = {oi | i = 1, 2,..., m} .
Definition 3: The sub-scenario is defined as a part of
execution of system and software, divided on the basis of
'
different phases of mission. Let Ssym = {Si' | i = 1, 2,..., n}
denotes the system-level sub-scenarios. Let
'
Ssw = {si' | i = 1, 2,..., m} denotes the software-level sub-
scenarios.
To describe the source of software input better, there is
an improvement of definition of operation which is defined
in Musa operational profile [1].
Definition 4: The object’s operation is defined as the
external motivations which lead object to corresponding
functional activity. Let
OOsymi = {OOij | j = 1, 2,..., n} denotes the operation j of
object i on the system-level scenario. Let
OOswi = {ooij | j = 1, 2,..., m} denotes the operation j of
object i on the software-level scenario.
Definition 5: The scenario-based software operational
profile (SBSOP) is defined as (1).
SBSOP = ( S , OO, P )
S is a set of all the scenarios, OO is a set of all the
object’s operations, and P is a set of all the probability on
the scenario.
III. DEVELOPMENT PROCESSES
Scenarios describe the behavior of software running
from a user perspective and reflect the expected way of
software running. By establishing the different scenarios of
software, SBSOP reveals the execution of the software. It is
an implicit operational profile.
The method is depicted in Figure 1Figure 1. ., as five
major steps.
Figure 1. Development processes of SBSOP
A. Scenario Object
All the objects that interact with the tested system and
software are considered as scenario objects. The definitions
of object in the different levels of scene are not the same.
On the system-level scenarios the tested object is the
system on which the tested software resides. The other
systems which impact on the tested system can be
considered as the other objects. The tested system has the
following characteristics.
• Users can directly control the system with the
operations, which reflect the users’ intent.
• The relation between the tested system and other
systems is obviously determined. Therefore, the
interacting information is easily obtained.
• The system should include the environment which
the software running.
On the software-level scenarios the objects are the
hardware and software which complete the tested
software’s work through the transmission of data.
B. User-Mode and Scenario-Mode
The scenarios of system and software can be obtained
(1) by analyzing the user-mode and scenario-mode.
There are differences in the way of use among different
users. Users are divided into several groups in which
customers use the system in the same way. A set of usage
of them is the user-mode, which can be obtained by
collecting and analyzing the information of various
customers.
In the same user-mode, the mission of product is
different according to the distinct purpose. The scenario-
mode is defined as a set of missions with the same intention.
For this reason, it can be acquired by investigating why
customers use the system.
For describing the various usages of system and
software, the scenario-mode is proposed. In this sense, it is
a group of scenarios to distinguish the execution of system
and software under different conditions. In other words,
scenario is a detailed example of scenario-mode, which
illustrates the implementation of specific task.
C. System-Level Scenario
Establishment of the system-level scenarios depends on
the tested software. In the same system, as long as
the tested software is different, the system-level scenario
will change. Because the system’s activities are completed
by working together of the internal hardware and software,
the system must have its corresponding task activities
for specific software.
701
 There are some general processes of constructing the
system-level scenarios.
• According to the function of the tested software,
the corresponding stage of the system tasks should
be found out.
• These stages are partitioned and sorted so as to list
the sub-scenarios whose sequence is determined on
the basis of location in the tasks.
• Through analyzing the operations in these sub-
scenarios, the relation between them can be found.
And the input and output of the tested object can be
obtained.
D. Software-Level Scenario
For a specific system-level sub-scenario, the
implementation of software functions needs to be analyzed.
In a complex system, operation of the software depends on
the operation of the system. Therefore, the input of tested
software can be generated by building the connection
between them.
There are some general processes of constructing the
software-level scenarios.
• Identify the objects, operations and states on the
software-level scenario.
• The influence of system operation on the software
operation needs to be analyzed. Then the relation
between them can be found.
• The software-level scenarios can be obtained
through organizing the software-level sub-scenarios
according to the structure of system-level sub
scenarios.
operational profile methods are difficult to be applied. In
order to explain the process clearly, some of the features of
MCDS has been simplified.
MCDS is the tested software, so the
associated equipment (e.g. seeker, rudder, inertial
navigation system (INS), launcher and fuse) are considered
as the objects on the software-level scenarios. On the
system-level scenarios, the missile on which MCDS resides
is the tested system, and the other objects are target and
aerial carrier.
From the use of missile, it can be found that the usage
of pilot is a user-mode. According to the task types of the
missile, air-to-air combat is considered as the scenario-
mode.
On the basis of the function of software, the scenario-
mode is decomposed into different system task units as the
system-level sub-scenarios. The factors affecting
the system operation need to be divided. TABLE I. shows
the various targets attacked by missile and their probability.
These system-level scenarios can be composed of these
units.

E. Probability in the SBSOP.

To complete the SBSOP, the probability of user-modes,
scenario-modes and operations on the scenarios need be
determined. The probability of user-modes is considered as
the proportion of different groups of users. The probability
of scenario-modes is identified with the frequency of
occurrence of different types of missions. The probability
of operations can be obtained from the data of field Figure 2. The relation with the missile sub-scenarios
experiments, the historical statistics of the previous version
or the usage of similar system. In case of the lack of data,
depending on the application background, the TABLE I. TARGET TYPE AND PROBABILITY
estimates given by experts need be adopted. If there is no
Target type Probability
way to obtain, the probability can even be distributed
equivalently. Fighter 0.7
Bomber 0.2
Helicopter 0.1
IV. EXAMPLE
In this section, SBSOP is illustrated with the example of In Figure 2. , there are the missile sub-scenarios: missile
MCDS. After the missile launches, because the input of prepares ( S1' ), missile launches ( S2' ), missile approaches
MCDS has nothing to do with the pilot, and there is a
strong correlation between the input data, the common target ( S3' ), missile locks target ( S4' ) and missile detonates

702
target ( S5' ). According to the paths from starting point to
terminal point, the scenarios of missile can be found in
TABLE II. .

TABLE II. MISSILE SCENARIOS AND SUB-SCENARIOS

Missile scenarios Missile sub-scenarios

S1 S1'

S2 S1' , S2'
S3 S1' , S2' , S3'
S4 S1' , S2' , S3' , S4' , S5'
S5 S1' , S2' , S3' , S4' , S3' , S4' , S5'
S6 S1' , S2' , S3' , S4' , S3'

Next, the establishment of software-level sub-

scenarios can be implemented: initialization ( s1' ),
Figure 3. The relation with the MCDS sub-scenarios
launching control ( s2' ), middle homing ( s3' ), terminal
homing ( s4' ) and detonating fuse ( s5' ). They are shown in To generate the testing data, the operations need to be
TABLE III. . analyzed to find the software’s input and output. Figure 4.
shows the transference of data on the software-level
TABLE III. CORRESPONDENCE BETWEEN SUB-SCENARIOS scenarios: s1' → s2' → s3' → s4' → s5' .

System-level sub-scenarios Software-level sub-scenarios

Missile prepares ( S1' ) Initialization ( s1' )

Missile launches ( S2' )
Missile approaches target ( S3' )
Missile locks target ( S4' )
Missile detonates target ( S )
Launching control ( s2' )
Middle homing ( s3' )
Terminal homing ( s4' )
'
5 Detonating fuse ( s5' )

To identify the input of software, the link of input

between system and software should be generated. For
example, TABLE IV. shows the relation between the flag
of intercepting target (FIT) and the input of missile.

TABLE IV. THE VALUE AND PROBABILITY OF FIT

Software System
Conditions Value Probability
input input
Distance T is maneuvering 1 0.9
between avoidance
0 0.1
the target
and T is decoy 1 0.8
Sİ projectile 0 0.2
missile
FIT 20km T is maneuvering 1 0.9
(S);
Target’s avoidance and
0 0.1
way to decoy projectile
escape (T) 1 0.2
Sı20km
0 0.8

From Figure 2. and TABLE III. , the relation with the

software-level sub-scenarios can be obtained as shown in
Figure 3. . Through the historical statistics and estimates Figure 4. The description of a software-level scenario
given by experts, the probability is identified.

703
 After building the SBSOP, test data can be generated on REFERENCES
the basis of the probability for software reliability test. [1] J.D. Musa, “Operational Profiles in Software-Reliability
TABLE V. is a simplified example of reliability test data Engineering,” IEEE Software, vol. 10, pp. 14-22, March 1993.
of MCDS. [2] Z.H. Chen and F. Wang, “Operational profiles and software
reliability,” Computer Science, vol. 33, pp. 86-89, August 2006.
TABLE V. A EXAMPLE OF TEST DATA OF MCDS
[3] F.Y. Han, Z. Tan, and X. Wang, “Generating technique for software
Software-level sub- Input reliability test data based on test profile,” Journal of Xi’an Jiao
Input Value
scenario parameter Tong University, vol. 40, pp. 1073-1077, October 2006.
Initialization  [4] X.S. Chen, M.Y. Lu, and L. Ruan, “Study on software reliability
Launching control  test data generation method,” Acta Aeronautica et Astronautica
Target type Fighter Sinica, vol. 22, pp. 509-512, November 2001.
Target Target height 11km [5] D.M. Woit, “Operational profile specification, test case generation,
information Target speed 223km/h and reliability estimation for modules,” CRL Report 281, Faculty of
Middle homing
  Engineering, McMaster University, Hamilton, Ontario, Canada,
Navigation February 1994.
parameters
 
[6] Y.F. Luo, K.R. Ben, and J. Pu, ”Scenario-based early software
Terminal homing  reliability models,” Computer Engineering and Science, vol. 30, pp.
Detonating fuse  98-101, November 2008.
[7] J. Ryser and M. Glinz, “A scenario-based approach to validating
TABLE V. shows the whole work process of MCDS. and testing software systems using statecharts,” in CNAM, editor,
The source of software input is actual use of missile. In Proc. 12th International Conference on Software and Systems
Engineering and their Applications, December 1999.
addition, because the probability is determined from the
[8] S. Kuball, G. Hughes, and B.I. Gilchrist, “Scenario-based unit
frequency of usage, the value of input is authentic. testing for reliability,” In Proceedings of the Annual Reliability and
Maintainability Symposium, February 2002.
V. CONCLUSION [9] S. Uchitel, J. Kramer, and J. Magee, “Synthesis of behavioral
models from scenarios,” IEEE Transactions on software engineering,
In this paper, on the basis of traditional methods we vol. 29, pp. 99-105, February 2003.
have presented a scenario-based software operational [10] S. Uchitel, G. Brunet, and M. Chechik, “Behaviour model synthesis
profile for software reliability testing. Compared with the from properties and scenarios,” International Conference on
operational profile methods already presented, expansion of Software Engineering, pp. 34-43, 2007.
the definition of operation can describe the non-user factors
(in TABLE IV. S and T are these factors) better that impact
the software input. Layered scenarios can establish a
relation between system and software and benefit
determining the relationship between the input data (e.g.,
TABLE IV. ). The proposal of user-mode and scenario-
mode can determine the probability of product usage more
easily. SBSOP improves the deficiencies of description of
traditional approaches. It can be more effectively applied
to complex systems.

704