A Systematic Review of Use Cases Based Software Testing Techniques

International Journal of Software Engineering and Its Applications
Vol. 10, No. 11 (2016), pp. 337-360

http://dx.doi.org/10.14257/ijseia.2016.10.11.28
A Systematic Review of Use Cases based Software Testing

Techniques
Asad Masood Qazi1*, Adnan Rauf2 and Nasir Mehmood Minhas3

1, 2, 3
University Institute of Information Technology – PMAS – Arid Agriculture
University Rawalpindi, Pakistan
1
asadmasood1@hotmail.com, 2mindz.xpert@gmail.com,
3
nasirminhas@uaar.edu.pk
Abstract
Use case based Software testing concerned with testing an entire system’s
functionality and its constraints. User’s requirements can be drawn in form of use cases
to show the internal and external behaviour of a system. The use case has been used to
test software system for different levels as well as through many ways according to
nature of software. Rather than proposing a new technique, it is necessary to synthesize
the existing techniques of Use case based software testing which is available in the
literature. A Systematic Literature Review has been performed to investigate all the
current approaches of Use case based software testing techniques. A question-wise
analysis has also been presented for the researchers to investigate the effectiveness of
these techniques. In results, we have discussed all the existing approaches of Use case
based software testing along with its strengths & weaknesses and measure the
characteristics of these approaches based on some key parameters. We have concluded
the current state of the art of all the existing approaches of Use case based testing.
Results are based on both types of analysis; Qualitative as well as Quantitative.
Keywords: Software testing, systematic literature review, use cases based testing, test
cases
1. Introduction
Software testing refers to executing a program with the intent of finding errors [1].
Producing testability in software is a complex task. Testing is also a time consuming and
costly procedure. Software testing is mainly categorized into two types; white box and
black box. White box testing refers to structural testing and black box testing refers to
functional testing of any Software under Test (SUT) [2] [3]. Some researchers added grey
box testing [3] in these types, which is an integration of white box and the black box or
mixing of the specification and implementation or design testing. [4]
In this survey, our main focus is on use case based software testing. Use case models
can test the system’s specification, design, architecture and behavior etc. In Literature,
many researchers applied use case based testing to integrate with different UML
diagrams to support the model based testing. A Model-based testing (MBT) is a type of
testing that relies on explicit behavior models that encode the intended behaviors of an
SUT and/or the behavior of its environment. Test cases are generated from one of these
models or their combination and then executed on the SUT [5].
Use case and other modelling languages like UML are used to visualizing through
graphical components of a system which helps the software engineer to develop software
in an easy way. It reduces the complexities of requirement engineering process as well.
Use case are used to transform the requirements specifications of any software to show its
*
Corresponding Author
ISSN: 1738-9984 IJSEIA

Copyright ⓒ 2016 SERSC
Vol. 10, No. 11 (2016)
functionality and constraints. Through these Use case models, a specification based
testing is also possible. Use case diagrams are used to show that how users will perform
any task with their system or website. Use cases are used to show an internal and external
behavior of a system. Also, it describes the relation between components as well as with
sub-components by using "extend", "include" and "generalization" property. Use cases
can be written in descriptive form as well as showed in graphical and visual form by
showing actors and their association with a specific component of a system. Use case also
helps to integrate product line or reuse features in a software development by improving
Meta concepts of Use case. [7]
A major contribution of existing Use case based software testing approaches [6] - [11]
deals with Object Oriented and component-based software domain. These techniques can
be applied in System Testing [8], Black box testing [9], Requirements based testing [10],
integration Testing [11], system testing GUI and testing of web based systems.
The rest of the paper is organized as follows: in Section 2, we present a Review of
related surveys in which Use case, model based testing or other related work has been
performed. Section 3 illustrates a complete Systematic Literature Review to gather
thoroughly all the existing approaches of Use case based software testing techniques.
Section 4 is the analysis of existing approaches according to criteria set in SLR. Finally,
Section 5 concludes the research work and discuss some future work.
2. Related Work
The quality of software systems needs to improve, and for this purpose, testing must
be performed. Different tools, mathematical techniques/formulas and models are
available in the field of testing to ensure the overall quality of software product. Model-
based or specification based testing mostly works on Use case and other UML (Unified
Modelling Languages) diagrams like activity, collaboration, sequence etc.
Authors in [12] presented a survey related to model-based testing up to the current
year 2015. This survey includes the current challenges and research gaps related to
model-based testing in the field of computer science, software engineering, electrical
engineering and other fields. This paper also focuses on the validation techniques of
functional and non-functional testing techniques. Requirements gathering and
specifications are important because testing depends upon the requirements. If
requirements are gathered and specified correctly, testing will be easy, and gives correct
results. This paper also covers the elements which are required for using a specific testing
technique.
Another survey was conducted by Dr. Arvinder Kaur and Vidhi Vig given in [13]. The
scope of that survey was based on testing through UML diagrams. Different authors' uses
a specific diagram like Use case, class diagram, collaboration diagram; sequence diagram
etc. to generate auto test cases for software model based testing. In this survey the
existing work is merged, and its research questions also answers the most popular
technique (in terms of its usage in research field or in the industry) of model-based
testing. Later on, it also discusses the type of testing used in each study.
Javier J. Gutireez et. al., (2006) [14] presented existing techniques of Use case-based
approaches and then proposed an approach to generate test cases from Use case for web
applications. Because existing approaches are not delivering the feasible results; they
expressed that Use case are more appropriate to start the generation process of test cases
for web systems. This research followed case study approach for web applications and
check that how a new link is added and stored into the system. The generation of test
cases will invoke three different elements i.e., the test values, the interactions with the
system under test and the expected results. The aim of his research is to validate that all
the Use case have been successfully implemented in the web system under test.
338 Copyright ⓒ 2016 SERSC

Vol. 10, No. 11 (2016)
Noraida Ismail et al., (2007) [15] in their study, highlighted some of the issues of
model-based testing because these approaches were not fully automated. To tackle such
issues author(s) has developed a tool for automatically generating test cases according to
the system’s requirement. The researcher classified his process into two steps. First, the
system requirements were transformed into Use case diagram and secondly the test cases
will be generated according to the Use case respectively. These test cases are important to
be used in analyzing and validating the requirements of the system as software testing is a
key part of Software Development Life Cycle (SDLC) and delaying this process will
definitely increase the number of errors and also it will be more tough and complex to fix
these errors at the end. The researcher explained that once the use case diagram has been
finalized from system requirements, it can be saved for further use and modification.
Clementine Nebut et. al., (2006) [16] emphasized on the barriers in model-based
testing approaches using Use case and proposed an approach for generating concrete test
cases by building a transition system from which the researcher create the test cases. This
approach eliminated ambiguities encountered at the time of writing the requirements in a
natural language simulation model of the use case. The simulation model is also used to
explicitly build a model of all the valid sequences of use case, and from it to extract
relevant paths using coverage criteria. These paths are called test objectives. The test
objectives generation from the use case constitutes the first phase of their approach. The
goal of this approach is to relieve of the automation work in test generation and to shift
the effort to the specification activity. The aim of their research is to generate test cases
for efficiently detecting faults in embedded software. The researcher performed an
empirical study to validate his approach by using several case studies and deploy their
approach for validating the results.
According to Anjali Sharma and Maninder Singh [17], the generation of test cases is
an important activity in testing. For this purpose, authors defined an algorithm which
takes UML diagrams as an input and generates test cases. The procedure of the proposed
framework can work in any given UML diagram includes use case, class diagram,
sequence diagram, data flow diagram etc. and convert them into a control flow graph
(CFG). Then they apply test case generator and hit cost generator algorithm on it which
will give an output in the form of test cases and coverage file. These test cases and
coverage file is then use as an input to report generator tool and it gives a complete report
to user or tester.
Shafique et. al., (2010) [18] presented a systematic literature review of tools supported
by model-based testing technique. The scope of this systematic literature review is
limited to identification of only tools and their features. Some of the tools from the
findings of this literature review are Test Designer, Qtronic, MOTES, GOTCHA-TC
Beans and others. The comparison criteria of these above-mentioned tools are States,
Transitions, Transition Pair, Sneak Paths, Parallel Transitions and All scenarios.
A review of some of the model based testing approaches are given in [19] and [20],
but the review is not systematic, so the comparative analysis is not been done systematic
way. Secondly, there is no proper analysis approach of these MBT tools. Also, the study
contains the review of approaches up to 2008, so to tackle the advancements later from
2008, there is a need to conduct a survey or systematic literature review.
In [21], the authors summarize the generation of test cases techniques for object
oriented software in their review study. The study focuses on all three aspects related to
generation of test cases which includes the generation, execution and evaluations of test
cases. The scope of this study is gathering all possible techniques either UML based or
not. This study also gives us the motivation to be a specific and in-depth analysis of
doing any model based testing analysis related to a specific diagrammatic model.
It can also be observed from the study given in [22] a model is used to describe the
behavior of the system, and to model the software systems, UML is one of the modeling
languages. Test cases are generated through these UML based techniques, but how to
Copyright ⓒ 2016 SERSC 339

Vol. 10, No. 11 (2016)
make them effective? This issue has been discussed in this review study. In this study, the
scope of the test cases is very broad and focuses on the complete software development
life cycle. This study found various types of testing through UML based approaches,
includes goal oriented techniques [23], specification based techniques [24], sketch
diagram based techniques [25] and source based techniques.
In [26], a recent work of literature review related to a model based testing by using
UML state chart diagram is presented. A survey table presented in this study is well
organized and presents complete details of the findings. The Table included the fields:
Author / Technique Name, Input Model, Method, Intermediate model and coverage
criteria. Most of the techniques used DFS algorithm. According to authors, the purpose of
generating test cases through UML is to verify the relation between behavior, state,
action and event. The correctness of test cases is totally based upon the correctness of the
model. If the model is wrong and unreliable, the output will also be unreliable. Another
survey related to software testing techniques was also presented in [27]. According to
authors, software testing is not an easy task; it takes time as well as a proper sequence of
steps. Which includes: planning, analysis, design, construction, testing cycle, final testing
cycle and post implementation? Any software application which is under test can be
tested through activity diagrams and other UML diagrams which help to reuse the
components of a system. Model-based testing approach takes system under test as an
input, generate system model, defining test coverage criteria, analyses using CPM
(Category Partition Method) [28] and then generate test cases. In this survey paper, a
review of work related to this particular approach is presented, and also complete UML
diagrams are not covered.
In [29] a comparative evaluation of test cases generation through UML diagrams was
presented. The comparative analysis showed that for unit level testing, state diagrams
give better results and for integration level testing, sequence diagrams show better results
in the generation of test cases. An experiment to evaluate the results followed their
sequence of steps: Analyze the specific UML diagram of any particular system,
Implements in Java, Generation of test cases, Design faults and run each test case of a
system. It can be observed from another survey on automatic generation of test cases
given in [30] that there were many techniques to generate the test cases automatically.
Model-based testing is one of the techniques; and in model-based techniques, UML
modeling is one of the popularly used approaches. These can be categorized in
specification based test case generation [31], model based test case generation [32], path
oriented test case generation [33] and intelligent techniques test case generation [34].
There is no main focus on any specific UML modeling diagram.
The above mentioned related works are presented to find the existing reviews and
surveys related to use case based software testing. Some of the related works are not
properly systematic, and some of them are too old to be considered. By presenting the
related work, we set an objective of this study is to tackle all the approaches of use case
based software testing in a systematic way.
3. Systematic Literature Review of Use cases based Testing

For the purpose of synthesizing the approaches described for Use case based software
testing, the guidelines have been tailored from the systematic literature review proposed
in [55]. In this regard, review protocol has been developed. The purpose of our study will
be described in this protocol which may include the need of review protocol for our
study, the definition of research questions, search process, inclusion / exclusion criteria,
and primary study selection and finally analyzing the approaches used for Use case based
software testing on the basis of different parameters. A detailed systematic literature
review process of this paper is presented below in graphically in Figure 1.

Vol. 10, No. 11 (2016)
Figure 1. Literature Selection and Analysis Strategy for SLR
3.1. Research Questions

The research questions to be addressed in this study are:
RQ1: What are the published Use case based software testing techniques used for test
case generation?
RQ2: Which parameters are used to measure the characteristics of Use case based
software testing techniques?
3.2. Search Process

In order to determine the similar work already performed in this area, we have to
generate a search string. There are two ways to perform search strategy; one is manual
search and the other is automatic search. Our search process is an automatic search of
specific conference proceedings and journal papers. We had seen lots of papers published
in different conferences, the nominated journals and conferences are as follows: Science
Direct, Springer Link, IEEE Explorer, ACM and others.

Vol. 10, No. 11 (2016)
In order to create the search string, the Boolean Operator “OR” was used to join
alternative terms and synonym; and the Boolean Operator “AND” was used to join all the
main terms.
We have identified some search terms and verify the keywords from major search
terms of our Research Questions and by using the Boolean search expressions, we have
constructed the following search string for our research questions:
(("test case generation from Use case diagrams" OR "approaches to generate test
cases from Use case diagrams" OR "testing techniques to generate test cases from Use
case diagrams" OR "testing approaches for test case generation using Use case
diagrams") AND ("characteristics of Use case-based approaches" OR "characteristics of
Use case-based techniques”) AND ("systematic review" OR "systematic literature
review" OR "systematic map" OR "systematic mapping" OR "mapping study" OR
"evidence-based study" OR "mapping study")).
A trial search was conducted using this search string on Science Direct Digital Library
and IEEE Transactions. The results of search terms proposed here were found sufficient
after checking the above mentioned two electronic databases. So we concluded that
Systematic Review is the best way to finding that what kind of similar work had been
already performed in this area.
3.3. Results of Search String

The list of Journals / conference proceedings, keyword searched and number of papers
found on different above mentioned sources. Approximately 384 papers are extracted
from different sources when search string is triggered. Detail of the results of search
string from different data sources is given in the following table, Table 1.
Table 1. Results Founded against Keywords
Results of Search Strings

# of
Data
Keywords Searched Papers
Sources
Found
"test case generation from use case diagrams" OR 25
"approaches to generate test cases from use case
diagrams"
Science Direct
"testing techniques to generate test cases from use 42

case diagrams" OR "MBT approaches for test case
generation using use case diagrams"
"characteristics of use case-based approaches" OR 14
"characteristics of use case-based techniques"
Total - Science Direct 81
Springer Link
diagrams"

Vol. 10, No. 11 (2016)
Total - Springer Link 74

diagrams"
CiteSeerX

Total - CiteSeerX Library 58
diagrams"
IEEE Explorer

Total - IEEE Explorer 115
diagrams"
ACM

Total - ACM 56
Grand Total 384
3.4. Study Selection Criteria

Initially, 384 papers were collected from results of our search string found in above
mentioned electronic databases and some from other journals as well. These papers are
analyzed on the basis of following inclusion-exclusion criteria:
3.4.1. Inclusion Criteria: Inclusion criteria for selection of studies is given below:
• Articles/papers which belong to use case-based software testing techniques.
• Articles/papers which belong to use cases based test cases generation.
• All published papers that have the potential of answering at least one research
question.

Vol. 10, No. 11 (2016)
3.4.2. Exclusion Criteria: Exclusion criteria for findings is:

• Topic who’s abstract and title not matching to our area.
• Multiple papers on the same topic will be excluded.
• Papers that do not have any link with our research questions.
• Duplicate papers (only the most complete, recent and improved one is included).
The rest are excluded.
3.4.3. Selected Primary Study: The appropriately selected studies are enlisted by a
single researcher. The rejected studies are checked by another researcher. We maintain a
list of papers that were rejected with reasons for the rejection. Further, we executed the
process of title and abstract screening of 384 searched papers in study selection process
and discovered 45 relevant papers as a primary study.
4. Analysis on Findings
The purpose of this section is to analyses the literature found from our search process
which will guide us to examine the approaches described for software testing in terms of
UML Use case. The data collected from this stage will help us to answer our research
questions. The summary of techniques found from our selected studies is given below:
4.1. Analysis on RQ 1
In this section, the analysis on existing techniques has been presented. To answer the
RQ1, we have gone through a systematic review of Literature and identify the existing
approaches of Use case based software testing and we enlist them in tabular form along
with their strengths and limitations below in Table 2.
Table 2. Use Case based Software Testing Techniques along with their
Strengths and Weaknesses
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
In this study, RA
model is proposed
which includes
following steps for No test
model-based testing: coverage
Shinpeiogata  Interaction Activity Requirements criteria is
Analysis based testing, properly
and Saeko
1.  Concrete I/O Data to meet the defined. Also [37]
Matsuura, analysis user’s the approach
(2010)  UML Based requirement has no
scenario analysis
 Scenario based supporting
prototype generation tool available.
 Definition process
of test case for
Integration test stage
MouradBadri The proposed Complete Complex
2. et.al, approach is based procedure, and Procedure [36]
(2004) upon Usecase. A make the because of

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
specification of process more involving
system is defined strength via multiple
using Usecase and including diagrams.
CDL and implements sequence
java code. Through diagrams as
Usecase, sequence well
diagrams are
generated, and
through execution and
verification process, a
fully instrument code
is taken as an output.
The proposed
approach is based
upon black box or Reliability
specification based rather than Lengthy
testing. The proposed fault tolerance. process of
approach consists 4 And proper defining,
Matthias et.al, steps, which includes:
3. verification refining and [35]
(2003)  Domain
understanding, process finalizing the
 Model it in UML, through XML- Usecase
 Convert it into based model models
Usage model and transformation
finally
 Converted into Test
model
The proposed
approach has
following steps:
 UML design
analysis & search of
relevant Usecase
 Analysis of
Sequence & class Fast Analysis
diagrams approach Multiple
Basanieri  Test Units Success Diagrams
4. et.al,  Research of setting experiment in analysis
[38]
(2000) & Interactions
categories Industry level involved
 Test specific projects
construction
 Search of message
sequence & Test
case
 Usecase test suite &
incremental
construction of Test

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
frame
Authors proposes
TOTEM model,
which includes
following steps:
 Analyse model to
check completeness
& correctness Successfully Multiple UML
L Briand and
 Derive Usecase, applied in diagrams are
5. Y Labiche class diagram & [39]
Case study / involved to
(2002) sequence diagrams Experiment analysis
 Derive variant
sequences
 Derive requirements
for system testing
 Derive test cases for
system testing
 Derive test oracles
Authors proposes
following sequence of
steps to generate test
cases from Usecase: No proper
JJ Gutiérrez  Generation of Simple &
instances of Usecase objective is
6. et.al, applied in a [14]
(2006)  Generation of test case study
presented in
values study
 Generation of test
cases
 Generation of
expected results
This model is based
upon two steps: Abstract level
Convert simple technique. The
Li, Liping, Usecase model to
hierarchical UCTMs approach
and Huaikou Only focuses
7. (Usecase transition needs simple [41]
Miao. on Usecase
model) and complete
(2008) Generate test cases Usecase
by using CSC models.
(Constraint Scenario
Coverage)
Following steps to
generate test cases are Simple
defined by Rational Approach and
Small projects
J Heumann. Rose Team: generate test
8. validation [42]
(2001)  Generate Scenario cases by using
only
 Identify Test cases only Usecase
 Identify Data values models
to Test

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
 Putting all together
This paper presents an
automatic tool for
generating test cases
using Usecase.
No
R Ibrahim Sequence diagrams
Automated Operational
9. et.al, and flow of events [43]
Tool Support Description
(2007) used for checking
consistency and
validity of the
generated by test
cases
Authors propose IFA
(Interaction Finite
Automation) state
based model to
perform Usecase
based testing. The
steps are:
 Transformation of
Usecase in to state
models
 Interaction Finite Experiment /
L Chen et.al, automation and Complex
10. Case study [44]
(2010) transformation of Algorithm
algorithms validation
 Test cases generation
The rules are:
 Each state in IFA
should traverse at
least once
 Each transition in
IFA should traverse
at least once
 Each path in IFA is
traversed at least
once
A Global
methodology of Involve
requirement based Validation
multiple tools
testing is applied in through Case
Complex
Nebut et.al, Thales Airbone study
11. scenario and [16]
(2006) System. Which Only use
suite for
generate test cases Usecase
complex and
using following models
large projects
sequence of steps:
 Requirement

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
Gathering
 Model the Usecase
as Usecase scenarios
 Convert Usecase into
simulation model
 Identify Test
objectives
 Generate test
scenarios with the
help of Usecase
scenarios and test
objectives
 Finally, Set of test
scenarios.
The proposed
approach works on
three stages of
software development
life cycle:
Requirements, Design
and Implementation &
Testing. And the Validation
procedure of test through
cases generation is as Experiment /
follows: Case study of
Bill Hasling  Convert the User’s Medical
Extra Training
12. et.al, requirements into is required for [45]
Usecase model system.
(2008) testers
 Store model into Improved
Test database Requirements
 Generate reports and and test cases
metrics and store
them also in test
database
 Generate test scripts
and stores them into
test database
 Execution of test
cases and stores
them also in test
database.
Authors proposes a Good for large Too much rely
simple approach to scale projects on tools, so
Serguei
generate test cases but to perform extra training
Roubtsov and
13. by using multiple acceptance of tools may [46]
Petra Heck.
tools involvement testing be required for
(2006)
through multiple Small number testers.
tools. Ideally of levels, to Heavily

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
approach is based generate test documentation
upon: cases procedure as
 Test plan Validation well.
 Test design through real
specification
 Test case projects
specification
 Test Procedure
Authors include
multiple UML models
to generate the test
cases. It is a complex
approach which
includes the Complex
transformation of procedure and
UML diagrams into involve
Santosh different intermediary Implemented
multiple UML
14. Kumar et.al, models includes CFG, in [47]
diagrams to
(2010) UDG, CCFG and ComTestTool
generate test
others. The steps cases for one
includes: case study.
 Deriving Usecase
dependency
sequences
 Deriving test
requirements from
sequence diagrams
 Deriving test cases.
The Proposed
approach is based on
the integration some
existing tools which Automated Merger of
supports the UML Tools UML
based testing CDT supported diagrams and
Bertolino and Cow_Suite. The approach, tools, so
15. et.al, [51]
proposed approach which is easy additional
(2003) have following to apply in training may
phases: different be required to
 Test of the single scenarios testing team
virtual component
 Test of a group of
integrated virtual
components
M. Sarma and Authors focuses on Proposed Only focus on
16. R. Mall the generation of test approach is an test [52]
(2007) specifications to automated specifications.

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
achieve test transition approach of
and named the generation of
approach as STRAP test cases from
(System Testing based UML
on transition Path diagrams. Also
coverage) and work supports tools
on following two for verification
phases: and generation
 Synthesize of SSG of test cases,
(System State and proposed
Graph)
 Generate system test approach
specification follows proper
automatically by algorithms.
traversing SSG.
Proposed approach is
designed for
generation of test
cases for web based
applications. Authors
have proposed
following steps to
generate test cases: A standard
 Modelling of web methodology,
application using which can be
page flow diagram applied in any Proposed
 Definition of logical
components and web approach has
partitioning of web application. It no tool
application. is also a support and
Chen et.al,
17. comprehensive not even [53]
(2007) And to generate the approach of validated
test cases, authors generation of through any
proposes following test cases case study or
steps: through experiment
 Composition of Usecase and
logical components other UML
using intermediary
diagrams like diagrams.
control flow graph
etc.
 Reuse of logical
component
interactions and
supporting of
concurrent access
and interaction.
18. Vieria et al, In this paper, authors Proposed Discussion [54]

Vol. 10, No. 11 (2016)
Sr. No
Publication
and Year of
Author(s)
Name of
Approach
Strengths
Limitations
Reference
(2006) presented the methodology based
importance of model- is a simple evaluation of
based testing methodology proposed
techniques and GUI and can be methodology
based system testing easily is presented in
is performed by using implemented this study.
Usecase models of the in various GUI Also the
system. The proposed based approach
methodologies applications. includes
includes following multiple
phases to generate test modelling
cases: languages for
 Model the system a single
using Usecase, program.
activity diagram and
class diagrams.
 Convert model in to
activity graph
 Setup of Data
coverage
 Generate of test
cases for sequence
of Usecase
 Analysis of results
4.2. Discussion on RQ 1
Use case has been involved in different levels of software testing like system testing,
requirements testing, integration testing and others. In this systematic review, a
comparison of all findings is discussed and categorized the techniques according to their
strengths and weaknesses. The strength of any technique depends upon its proposed
steps, use of tool, an algorithm applied and also the level of testing.
Similarly the weaknesses of any Use case based software testing techniques refers to
its complexity in terms of number of diagrams or models involved in each technique, like
many techniques do not use directly Use case, also they includes other UML diagrams as
well, so such techniques are complex in terms of time, cost as well as the training and
accuracy required in the modelling techniques.
Model driven techniques are automated testing techniques, whereas model based
techniques are not fully automated and the generation of test cases from Use case or other
models are manual. Use case supported both types of testing model based as well as
model driven testing. Other model based or model driven testing usually focuses on
control flow graph, algorithms etc. but our findings show that there are many Use case
based testing techniques which are simple, which generate test cases directly from the
Use case without involving intermediary modeling language.
To analyses the RQ1 or the Use case based software testing, our results also focus on
the acceptance level of any specific technique. Because the overall goal of any software
testing technique is the acceptance of users, so in the strength of all the techniques, we try
to find the level of acceptance of the results after testing the system through the use case.

Vol. 10, No. 11 (2016)
4.3. Analysis on RQ 2
In this section, analysis on RQ2 has been presented. To answer the RQ2, we have gone
through different literature repositories to identify the parameters, from which the
existing use case based techniques has been characterized. We have also analyzed some
of the most commonly used parameters which are presented and used by different
literatures. The commonly used parameters are: Level of Testing, Algorithm used, use of
tools or supported tools, Level of automation, software domain, complexity of techniques
and others. We have presented analysis according to different parameters separately in
tabular form as well as combined. A combined analysis is tabulated below in Table 3.
Next sections will provide a complete Qualitative analysis of RQ2 by using graphical and
tabular form to support the readers and to analyze the techniques in proper form.
Table 3. Analysis of Use Case based Software Testing Techniques w.r.t

Different Parameters
Algorithm
Technique
Validation
Technique
Reference
Coverage
Supporte
Software
Through
Name of
Criteria
Level of
Domain
Testing
Author
Sr. No
Tool
(s)
d
/
Shinpei Black
ogata and Not Not Experiment Business Box/ [37
1 Saeko Yes Defined Define / Domain Specificati ]
Matsuura, d Case Study on Based
(2010)
Mourad Not Experiment Object System [36
2 Badri et.al, Yes Defined Yes / Oriented Testing ]
(2004) Case Study
Matthias Not Experiment Object System [35
3 et.al, (2003) Define Yes Yes / Oriented Testing ]
d Case Study
Integratio
Basanieri Not Not Experiment Object n& [38
4 et.al, Define Defined Yes / Oriented Architectu ]
(2000) d Case Study re
Testing
L Briand Not Experiment System &
5 and Y Yes Yes
Define / Object Regressio [39
Labiche d Case Study Oriented n ]
(2002) Testing
JJ Gutiérrez Not Not Experiment Object Requirem [40
6 et.al, Define Defined Yes / Oriented ent ]
(2006) d Case Study Testing
Li, Liping,
and Not Experiment Object System [41
7 Huaikou Yes Defined Yes / Oriented Testing ]
Miao. Case Study
(2008)
Experiment Requirem
8 J(2001)
Heumann. Yes Yes Yes / Object
Oriented ent [42
]
Case Study Testing
R Ibrahim Not Automated Object Requirem [43
9 et.al, Define Yes Yes Tool Oriented ent ]
(2007) d Testing
L Chen
10 et.al, Yes Yes Yes Experiment Object System [44
/ Oriented Testing ]

Vol. 10, No. 11 (2016)
Algorithm
Technique
Validation
Technique
Reference
Coverage
Supporte
Software
Through
Name of
Criteria
Level of
Domain
Testing
Author
Sr. No
Tool
(s)
d
/
(2010) Case Study

Object
Experiment Oriented System &
11 Nebut
(2006)
et.al, Yes Yes Yes / /
Embedde
Requirem
ents
[16
]
Case Study d Testing
System
Bill Hasling Experiment Object System [45
12 et.al, Yes Yes Yes / Oriented Testing ]
(2008) Case Study
Serguei Experiment System &
13 Roubtsov &
Petra Heck. Yes Yes Yes / Object
Oriented
Users
Acceptanc
[46
]
(2006) Case Study e Testing
Santosh Experiment Integratio
14 Kumar et.al, Yes Yes Yes / Object n& [47
(2010) Case Study Oriented System ]
Testing
Bertolino Not Not Experiment Compone
nt Integratio [51
15 et.al, Define Yes Define / Based n ]
(2003) d d Case Study Software Testing
M. Sarma Automated Business Black
16 and R. Mall Yes Yes Yes Tool / Box/ [52
(2007) Case Study Domain System
Testing
]
Requirem
17 Chen et.al,
(2007) Yes Not
Defined Yes Not
Defined
Business
Domain ents [53
]
Testing
Graphical
Not Case Study User
18 Vieria
(2006)
et al, Not
Define Defined Yes / Interface System
Testing
[54
]
d Discussion (GUI)
Domain
4.3.1. Level of Testing: Level of testing [48] refers to the behavior of system model,
which is to be examining under the desired testing technique. An example of levels is
Integration, Requirements and Specification etc. We have analyzed each technique in
depth to analyze the discussed level of testing in all the techniques and if there is any tool
supported by that particular technique we have enlisted them as well. The details of
findings are presented in a diagram mentioned below named as Figure 2 and also enlisted
in tabular form below in Table 4. The analysis on the basic of number of approaches
found in each level as well as the number of approaches supported by a tool also
presented below in Figure 3 and Figure 4 respectively.

Vol. 10, No. 11 (2016)
Figure 2. Analyzing the Findings on basis of Level of Testing Performed by

Use Cases Based Techniques
Table 4. Analyzing the Findings on Basis of Level of Testing

Supporting
Sr. Level of Testing No. of Tool
No Approaches Use Not
Defined
1. System Testing 09 06 03
2. Integration Testing 03 02 01
3. Specification Based
Testing 01 01 -
4. Regression Testing 01 01 -
5. Requirements based
Testing 05 03 02

Vol. 10, No. 11 (2016)
Figure 3. Analysis on the Basis of Level of Testing of Total Number of Use

Case Based Testing Approaches Found
Figure 4. Analysis on the Basis of Use of Tools of Total Number of Use

Case Based Testing Approaches Found

Vol. 10, No. 11 (2016)
4.3.2. Algorithm Usage: Algorithms are used by each technique to traverse the graphs
and trees if used by any technique. Most of the cases DFS [49] and BFS [50] algorithms
are used by model – based testing techniques.
4.3.3. Tools Support: The support of tool is used to automate the testing process. Most
of the model – based testing approaches uses model checking tools and algorithms. Some
of the techniques are founded from literature, in which researcher proposes a new tool
[43] and some of the authors / researchers uses existing tools [47] to generate test cases
from use case models.
4.4. Discussion on RQ2

As discussed earlier, the findings of the use case based software testing techniques
involve many levels of testing i.e., system testing, integration testing, regression testing,
requirements based testing, acceptance testing and specification based software testing.
The main objective of this study is to find a specific technique which can be applied to
test the multiple levels of testing, which is not a complex issue and may reduce the cost
and other resources usage. The researchers implemented these techniques successfully
and evaluated in some case studies as well as experiments on a high level and complex
software development solutions.
To evaluate these issues, our analysis approach is based on maximum possible
parameters which can be set by any software tester, researchers and other software
engineers from academics. Use of tool and implementation of Algorithm are the main
attributes to make the technique simple or complex. Complex algorithms decrease the
level of automation of software testing technique, and also requires more support or
training to its implementation.
5. Conclusion and Future Work

In this article, we have performed a systematic literature review to find the existing
approaches to model-based testing through UML Use case diagrams. In our study, we
have found different approaches to performing some specific type of testing, which can
be performed in any specific or generic phase of software development life cycle.
UML Use case can be used to perform requirements based testing, which ensures that
user's requirements are completely fulfilled. And it is performed in a Requirement
analysis phase. Similarly, in High-level design or software architecture phase of SDLC,
system testing is performed through a use case. And in low-level design, integration
testing is performed through Use case. These all techniques and types of test cases full fill
the requirements of black box testing in some way. Regression testing, which can be
performed in any phase or stage, can also be done through the Use case diagrams. To
validate all the requirements and acceptance of software product, acceptance testing is
performed. If all the results from previously mentioned testing types are being corrected,
then acceptance testing can be performed. For this purpose, Use case can also be helpful
for the software engineers and practitioners.
The purpose of this study is to identify and synthesize the current state of the art of
existing approaches of Use case based testing techniques. In future, we will work on
improvements on different testing techniques performed through requirements
specifications.
References
[1] R. S. Pressman, “Software Engineering - A Practitioner’s Approach” Fifth edition. (2001).
[2] B. Beizer, “Software Testing Techniques. Van Nostrand Reinhold, New York, (1990).
[3] M. Shafique, “A systematic review of state-based test tools”, International Journal on Software Tools
for Technology Transfer, vol. 17, Issue 1, (2013), pp. 59-76.

Vol. 10, No. 11 (2016)
[4] R. Patton, “Software Testing”, SAMS, Indianapolis, (2005).

[5] M. Utting, A. Pretschner and B. Legeard, “A Taxonomy of Model-Based Testing Approaches”,
Software Testing, Verification and Reliability, vol. 22, (2012), pp. 297-312.
[6] A. C. Dias Neto, R. Subramanyan, M. Vieira and G. H. Travassos, “A survey on model-based testing
approaches: a systematic review”, Proceedings of the 1st ACM international workshop on Empirical
assessment of software engineering languages and technologies: held in conjunction with the 22nd
IEEE/ACM International Conference on Automated Software Engineering (ASE) November 05-05,
2007, Atlanta, Georgia, (2007), pp. 31-36.
[7] M. Eriksson, J. Börstler and K. Borg, “The PLUSS Approach - Domain Modeling with Features,
Usecase and Usecase Realizations”, Proceedings of the 9th International Conference on Software
Product Lines, LNCS, vol. 3714, Springer-Verlag, (2005), pp. 33-44.
[8] Q. Zhou, B. Liu, Z. Yu and X. Xing, “The process of requirement analysis about military software
system testing”, In Reliability, Maintainability and Safety (ICRMS), 2011 9th International Conference
on, IEEE, (2011), pp. 645-650.
[9] S. Rosiello, A. Choudhary, A. Roy and R. Ganesan, “Combining Black Box Testing with White Box
Code Analysis: A Heterogeneous Approach for Testing Enterprise SaaS Applications”, In Software
Reliability Engineering Workshops (ISSREW), 2014 IEEE International Symposium on, IEEE, (2014),
pp. 359-364.
[10] T. Sarwar, W. Habib and F. Arif, “Requirements based testing of software”, In Informatics and
Applications (ICIA), 2013 Second International Conference on, IEEE, (2013), pp. 347-352.
[11] A. Elsafi, D. NA Jawawi and A. Abdelmaboud, “Inferring approximated models for integration testing
of component-based software”, In Software Engineering Conference (MySEC), 2014 8th Malaysian,
IEEE, (2014), pp. 67-71.
[12] S. Rӧsch, S. Ulewicz, J. Provost and B. Vogel-Heuser, “Review of Model-Based Testing Approaches in
Production Automation and Adjacent Domains—Current Challenges and Research Gaps”, Journal of
Software Engineering and Applications, vol. 8, (2015), pp. 499-519.
[13] A. Kaur and V. Vig, “Systematic review of automatic test case generation by UML diagrams”, In
International Journal of Engineering Research and Technology, vol. 1, no. 6 (August-2012). ESRSA
Publications, (2012).
[14] J. J. Gutiérrez, M. J. Escalona, M. Mejías and J. Torres, “An approach to generate test cases from
Usecase”, In Proceedings of the 6th international conference on Web engineering, ACM, (2006), pp.
113-114.
[15] N. Ismail and R. Ibrahim, “Automatic generation of test cases from use-case diagram”, In Proceedings
of the International Conference on Electrical Engineering and informatics Institut Teknologi Bandung.
Institut Teknologi bandung, (2007).
[16] C. Nebut, F. Fleurey, Y. L. Traon and J.-M. Jezequel, “Automatic test generation: A Usecase driven
approach”, Software Engineering, IEEE Transactions on 32, no. 3, (2006), pp. 140-155.
[17] A. Sharma and M. Singh, “Generation of automated test cases using UML modelling”, International
Journal of Engineering Research and Technology (IJERT), vol. 2 Issue 4, (2013).
[18] M. Shafique and Y. Labiche, “A systematic review of model based testing tool support”, Carleton
University, Canada, Tech. Rep. Technical Report SCE-10-04 (2010), pp. 01-21.
[19] J. Boberg, “Early fault detection with model-based testing”, Proceedings of the 7th ACM SIGPLAN
workshop on ERLANG. ACM, (2008).
[20] A. Saifan and J. Dingel, “Model-based testing of distributed systems”, School of Computing Queen's
University Canada, (2008).
[21] R. Singh, “Test case generation for object-oriented systems: A review”, In Communication Systems and
Network Technologies (CSNT), 2014 Fourth International Conference on, IEEE, (2014), pp. 981-989.
[22] N. Pahwa and K. Solanki, “UML based test case generation methods: A review”, International Journal
of Computer Applications 95, no. 20, (2014).
[23] D. C. Nguyen, A. Perini and P. Tonella, “A goal-oriented software testing methodology”, Agent-
Oriented Software Engineering VIII. Springer Berlin Heidelberg, (2008), pp. 58-72.
[24] P. Stocks and D. Carrington, “A framework for specification-based testing”, Software Engineering,
IEEE Transactions on 22, no. 11, (1996), pp. 777-793.
[25] M. Utting and B. Legeard, “Practical model-based testing: a tools approach”, Morgan Kaufmann,
(2010).
[26] Y. D. Salman and N. L. Hashim, “Automatic Test Case Generation from UML State Chart Diagram: A
Survey”, Advanced Computer and Communication Engineering Technology, Springer International
Publishing, (2016), pp. 123-134.
[27] Karambir and K. Kaur, “Survey of Software Test case Generation Techniques”, International Journal of
Advanced Research in Computer Science and Software Engineering, vol. 3, no. 6, (2013), pp. 937-942.
[28] T. J. Ostrand and M. J. Balcer, “The category-partition method for specifying and generating fuctional
tests”, Communications of the ACM, vol. 31, no. 6, (1988), pp. 676-686.
[29] S. Kansomkeat, J. Offutt, A. Abdurazik and A. Baldini, “A comparative evaluation of tests generated
from different UML diagrams”, In Software Engineering, Artificial Intelligence, Networking, and

Vol. 10, No. 11 (2016)
Parallel/Distributed Computing, 2008. SNPD'08. Ninth ACIS International Conference on IEEE, (2008),
pp. 867-872.
[30] M. Prasanna, S. Sivanandam, R. Venkatesan and R. Sundarrajan, “A survey on automatic test case
generation”, Academic Open Internet Journal 15, no. part 6, (2005).
[31] R. Hierons, “Specification based testing”, Software Focus 1.2, (2000), pp. 25-26.
[32] J. Dick and A. Faivre, “Automating the generation and sequencing of test cases from model-based
specifications”, In International Symposium of Formal Methods Europe, Springer Berlin Heidelberg,
(1993), pp. 268-284.
[33] J. Zhang, X. Chen and X. Wang, “Path-oriented test data generation using symbolic execution and
constraint solving techniques”, Software Engineering and Formal Methods, 2004. SEFM 2004.
Proceedings of the Second International Conference on. IEEE, (2004), pp. 242-250.
[34] D. M. Cohen, “The AETG system: An approach to testing based on combinatorial design”, Software
Engineering, IEEE Transactions on 23.7, (1997), pp. 437-444.
[35] M. Riebisch, I. Philippow and M. Götze, “UML-based statistical test case generation”, Objects,
Components, Architectures, Services, and Applications for a Networked World. Springer Berlin
Heidelberg, (2003), pp. 394-411.
[36] M. Badri, L. Badri and M. Naha, “A Usecase driven testing process: Towards a formal approach based
on UML collaboration diagrams”, Formal Approaches to Software Testing. Springer Berlin Heidelberg,
(2004), pp. 223-235.
[37] S. Ogata and S. Matsuura, “A method of automatic integration test case generation from UML-based
scenario”, WSEAS Trans. Information Science and Applications, vol. 4, (2010), pp. 598-607.
[38] F. Basanieri and A. Bertolino, “A practical approach to UML-based derivation of integration tests”, 4th
International Quality Week Europe, (2000).
[39] L. Briand and Y. Labiche, “A UML-based approach to system testing”, Software and Systems
Modeling 1.1, (2002), pp. 10-42.
[40] J. J. Gutiérrez, M. J. Escalona, M. Mejías and J. Torres, “An approach to generate test cases from use
cases”, In Proceedings of the 6th international conference on Web engineering, ACM, (2006), pp. 113-
114.
[41] L. Li and H. Miao, “An approach to modeling and testing web applications based on use cases”, In
2008 International symposium on information science and engineering, vol. 1, IEEE, (2008), pp. 506-
510.
[42] J. Heumann, “Generating test cases from Usecase”, The rational edge 6.01, (2001).
[43] R. Ibrahim, M. Z. Saringat, N. Ibrahim and N. Ismail, “An automatic tool for generating test cases from
the system's requirements”, In Computer and Information Technology, 2007. CIT 2007. 7th IEEE
International Conference on, IEEE, (2007), pp. 861-866.
[44] L. Chen and Q. Li, “Automated test case generation from use case: A model based approach”, In
Computer Science and Information Technology (ICCSIT), 2010 3rd IEEE International Conference on,
IEEE, vol. 1, (2010), pp. 372-377.
[45] B. Hasling, H. Goetz and K. Beetz, “Model based testing of system requirements using UML use case
models”, In 2008 1st International Conference on Software Testing, Verification, and Validation. IEEE,
(2008), pp. 367-376.
[46] S. Roubtsov and P. Heck, “Use case-based acceptance testing of a large industrial system: Approach
and experience report”, In Testing: Academic & Industrial Conference-Practice and Research
Techniques (TAIC PART'06), IEEE, (2006), pp. 211-220.
[47] S. K. Swain, D. P. Mohapatra and R. Mall, “Test case generation based on Usecase and sequence
diagram”, International Journal of Software Engineering, vol. 3, no. 2, (2010), pp. 21-52.
[48] G. J. Myers, C. Sandler and T. Badgett, “The art of software testing”, John Wiley & Sons, (2011).
[49] R. E. Korf, “Depth-first iterative-deepening: An optimal admissible tree search”, Artificial intelligence
27.1, (1985), pp. 97-109.
[50] R. Zhou and E. A. Hansen, “Breadth-first heuristic search”, Artificial Intelligence 170.4, (2006), pp.
385-408.
[51] A. Bertolino, E. Marchetti and A. Polini, “Integration of “Components” to test software components”,
Electronic Notes in Theoretical Computer Science 82.6, (2003), pp. 44-54.
[52] M. Sarma and R. Mall, “Automatic generation of test specifications for coverage of system state
transitions”, Information and Software Technology, 51.2, (2009), pp. 418-432.
[53] S. Chen, H. Miao and Z. Qian, “Automatic generating test cases for testing web applications”, In
Computational Intelligence and Security Workshops, 2007. CISW 2007. International Conference on,
IEEE, (2007), pp. 881-885.
[54] M. Vieira, J. Leduc, B. Hasling, R. Subramanyan and J. Kazmeier, “Automation of GUI testing using a
model-driven approach”, In Proceedings of the 2006 international workshop on Automation of software
test, ACM, (2006), pp. 9-14.
[55] B. Kitchenham, R. Pretorius, D. Budgen, O. P. Brereton, M. Turner, M. Niazi and S. Linkman,
“Systematic literature reviews in software engineering–a tertiary study”, Information and Software
Technology 52, no. 8, (2010), pp. 792-805.

Vol. 10, No. 11 (2016)
Authors
Asad Masood Qazi, he is a young IT Professional with an interest
in software design and development, database management systems,
OLAP systems, data warehousing, software engineering, software
process improvement and software quality assurance. Mr. Asad
Masood is also a research student at University Institute of
Information Technology – PMAS – Arid Agriculture University
Rawalpindi, Pakistan, and also participated in different research
oriented projects of different areas related to software engineering.
Some of the research publications of Mr. Asad Masood could be
found in different International Journals.
Adnan Rauf, he is a Research student at University Institute of

Information Technology – PMAS – Arid Agriculture University
Rawalpindi, Pakistan, and also participated in different research
oriented projects of different areas related to software engineering.
The area of interest of Mr. Adnan Rauf in research is Software
Testing, Ontologies, Knowledge Management and GSD related
issues in software engineering.
Nasir Mehmood Minhas, he is working as Assistant Professor at

University Institute of Information Technology – PMAS – Arid
Agriculture University Rawalpindi, Pakistan. The major area of
research is Software Requirement Engineering, Software Process /
Software Process Improvement, Global / Distributed Software
Development and Ontologies in software development practices.

Vol. 10, No. 11 (2016)

A Systematic Review of Use Cases Based Software Testing Techniques

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Systematic Review of Use Cases Based Software Testing Techniques

Uploaded by

Copyright:

Available Formats

International Journal of Software Engineering and Its Applications

Vol. 10, No. 11 (2016), pp. 337-360

A Systematic Review of Use Cases based Software Testing

Asad Masood Qazi1*, Adnan Rauf2 and Nasir Mehmood Minhas3

ISSN: 1738-9984 IJSEIA

338 Copyright ⓒ 2016 SERSC

Copyright ⓒ 2016 SERSC 339

3. Systematic Literature Review of Use cases based Testing

340 Copyright ⓒ 2016 SERSC

Figure 1. Literature Selection and Analysis Strategy for SLR

3.1. Research Questions

3.2. Search Process

Copyright ⓒ 2016 SERSC 341

3.3. Results of Search String

Table 1. Results Founded against Keywords

Results of Search Strings

"testing techniques to generate test cases from use 42

342 Copyright ⓒ 2016 SERSC

"characteristics of use case-based techniques"

Total - Springer Link 74

"testing techniques to generate test cases from use 27

"testing techniques to generate test cases from use 58

case diagrams" OR "MBT approaches for test case

3.4. Study Selection Criteria

Copyright ⓒ 2016 SERSC 343

3.4.2. Exclusion Criteria: Exclusion criteria for findings is:

344 Copyright ⓒ 2016 SERSC

Copyright ⓒ 2016 SERSC 345

346 Copyright ⓒ 2016 SERSC

Copyright ⓒ 2016 SERSC 347

348 Copyright ⓒ 2016 SERSC

Copyright ⓒ 2016 SERSC 349

350 Copyright ⓒ 2016 SERSC

Copyright ⓒ 2016 SERSC 351

Table 3. Analysis of Use Case based Software Testing Techniques w.r.t

352 Copyright ⓒ 2016 SERSC

(2010) Case Study

Copyright ⓒ 2016 SERSC 353

Figure 2. Analyzing the Findings on basis of Level of Testing Performed by

Table 4. Analyzing the Findings on Basis of Level of Testing

354 Copyright ⓒ 2016 SERSC

Figure 3. Analysis on the Basis of Level of Testing of Total Number of Use

Figure 4. Analysis on the Basis of Use of Tools of Total Number of Use

Copyright ⓒ 2016 SERSC 355

4.4. Discussion on RQ2

5. Conclusion and Future Work

356 Copyright ⓒ 2016 SERSC

[4] R. Patton, “Software Testing”, SAMS, Indianapolis, (2005).

Copyright ⓒ 2016 SERSC 357

358 Copyright ⓒ 2016 SERSC

Adnan Rauf, he is a Research student at University Institute of

Nasir Mehmood Minhas, he is working as Assistant Professor at

Copyright ⓒ 2016 SERSC 359

360 Copyright ⓒ 2016 SERSC

You might also like