Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more ➡
Download
Standard view
Full view
of .
Add note
Save to My Library
Sync to mobile
Look up keyword
Like this
5Activity
×
0 of .
Results for:
No results containing your search query
P. 1
A Black-Box Test Case Generation Method

A Black-Box Test Case Generation Method

Ratings: (0)|Views: 1,476|Likes:
Published by ijcsis
Abstract—Test case generation techniques have been researched over a long period of time. Unfortunately, while many researchers have found methods of minimizing test cases, there are still a number of important related issues that need to be researched. The primarily outstanding research issue is a large single test suite containing a huge number of test cases. Our study shows that this can lead to other two problems: unable to identify suitable test cases for execution and those test cases are lack of ability to cover domain specific requirement. Therefore, we proposed an additional requirement prioritization process during a test case generation process and an automated method to generate multiple test suites while minimizing a number of test cases from UML Use Case diagram 2.0. Our evaluation result shows that the proposed method is the most recommendation method to minimize size of test cases while maximizing ability to cover critical domain specific requirements.

Keywords-component; Test generation, testing and quality, test case generation, test generation technique and generate tests
Abstract—Test case generation techniques have been researched over a long period of time. Unfortunately, while many researchers have found methods of minimizing test cases, there are still a number of important related issues that need to be researched. The primarily outstanding research issue is a large single test suite containing a huge number of test cases. Our study shows that this can lead to other two problems: unable to identify suitable test cases for execution and those test cases are lack of ability to cover domain specific requirement. Therefore, we proposed an additional requirement prioritization process during a test case generation process and an automated method to generate multiple test suites while minimizing a number of test cases from UML Use Case diagram 2.0. Our evaluation result shows that the proposed method is the most recommendation method to minimize size of test cases while maximizing ability to cover critical domain specific requirements.

Keywords-component; Test generation, testing and quality, test case generation, test generation technique and generate tests

More info:

Published by: ijcsis on Oct 10, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See More
See less

07/20/2011

pdf

text

original

 
 
A Black-Box Test Case Generation Method
Nicha Kosindrdecha
 
Autonomous System Research LaboratoryFaculty of Science and Technology, Assumption UniversityBangkok, ThailandP4919742@au.edu
Jirapun Daengdej
 
Autonomous System Research LaboratoryFaculty of Science and Technology, Assumption UniversityBangkok, Thailand jirapun@scitech.au.edu 
 Abstract
—Test case generation techniques have been researchedover a long period of time. Unfortunately, while manyresearchers have found methods of minimizing test cases, thereare still a number of important related issues that need to beresearched. The primarily outstanding research issue is a largesingle test suite containing a huge number of test cases. Our studyshows that this can lead to other two problems: unable to identifysuitable test cases for execution and those test cases are lack of ability to cover domain specific requirement. Therefore, weproposed an additional requirement prioritization process duringa test case generation process and an automated method togenerate multiple test suites while minimizing a number of testcases from UML Use Case diagram 2.0. Our evaluation resultshows that the proposed method is the most recommendationmethod to minimize size of test cases while maximizing ability tocover critical domain specific requirements.
 Keywords-component; Test generation, testing and quality, test case generation, test generation technique and generate tests
I.
 
I
NTRODUCTION
 Software testing is known as a key critical phase in thesoftware development life cycle, which account for a large partof the development effort. A way of reducing testing effort,while ensuring its effectiveness, is to generate a minimizenumber of test cases automatically from artifacts used in theearly phases of software development. Many test casegeneration techniques have been proposed [2], [4], [10], [11],[12], [15], [21], [22], [42], [47], [50], mainly random, path-oriented, goal-oriented and model-based approaches. Randomtechniques determine a set of test cases based on assumptionsconcerning fault distribution. Path-oriented techniquesgenerally use control flow graph to identify paths to be coveredand generate the appropriate test cases for those paths. Goal-oriented techniques identify test cases covering a selected goalsuch as a statement or branch, irrespective of the path taken.There are many researchers and practitioners who have beenworking in generating a set of test cases based on thespecifications. Modeling languages are used to get thespecification and generate test cases. Since Unified ModelingLanguage (UML) 2.0 is the most widely used language, manyresearchers are using UML diagrams such as UML Use Casediagram, UML Activity diagram and UML Statechart diagramto generate test cases and this has led to model-based test casegeneration techniques. The study shows that model-based testgeneration methods (or also known as black-box testgeneration) are widely-used for generating test cases in thecommercial industry.Moreover, the study [2], [4], [10], [11], [12], [15], [21],[22] shows that the primary research issue is that existingblack-box test case generation methods generate a huge singletest suite with a number of possible tests. The number of possible black-box tests for any non-trivial software applicationis extremely large. Consequently, it is unable to identifysuitable test cases for execution.Also, the study shows that the secondary research issue isthat the existing black-box test case generation methods ignorecritical domain specific requirements [5] during a test casegeneration process. These requirements are one of the mostimportant requirements that should be addressed during testactivities.Therefore, we propose a new black-box test casegeneration, with requirement prioritization approach, fromrequirements captured as use cases, 2.0, [23], [24], [33]. A usecase is the specification of interconnected sequences of actionsthat a system can perform, interacting with actors of thesystem. Use cases have become one of the favorite approachesfor requirements capture. Our automated black-box approachaims to generate a minimize number of suitable test cases whilereserving critical domain specific requirements. Additionally,we introduce an automated test generation method derivedfrom UML Use Case diagram, 2.0. Our approach is developedto automatically generate many test suites based on notionsannounced in the latest version of UML.The rest of the paper is organized as follow. Section 2discusses an overview of test case generation techniques.Section 3 describes motivated research issues. Section 4introduces a new test generation process with requirementprioritization step. Also, section 4 proposes a new black-boxtest generation method. Section 5 describes an experiment,measurement metrics and results. Section 6 provides theconclusion and research directions in the test case generationfield. The last section represents all source references used inthis paper.II.
 
LITERATURE
 
REVIEWThe literature review is structured into two sections. Thefirst section gives an overview of previous studies. The secondsection provides the related works
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201022http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
 
 A.
 
 An Overview of Recent Researches
Model-based techniques are popular and most researchershave proposed several techniques. One of the reasons whythose model-based techniques are popular is that wronginterpretations of complex software from non-formalspecification can result in incorrect implementations leading totesting them for conformance to its specification standard [43].A major advantage of model-based V&V is that it can be easilyautomated, saving time and resources. Other advantages areshifting the testing activities to an earlier part of the softwaredevelopment process and generating test cases that areindependent of any particular implementation of the design [7].The model-based techniques are method to generate testcases from model diagrams like UML Use Case diagram [23],[24], [33], UML Sequence diagram [7] and UML State diagram[5], [43], [22], [2], [21], [15], [32], [4]. There are manyresearchers who investigated in generating test cases fromthose diagrams. The following paragraphs show examples of model-based test generation techniques that have beenproposed for a long time.Heumann [23] presented how using use cases, derived fromUML Use Case diagram 1.0, to generate test cases can helplaunch the testing process early in the development lifecycleand also help with testing methodology. In a softwaredevelopment project, use cases define system softwarerequirements. Use case development begins early on, so realuse cases for key product functionality are available in earlyiterations. According to the Rational Unified Process (RUP), ause case is used to describe fully a sequence of actionsperformed by a system to provide an observable result of valueto a person or another system using the product underdevelopment." Use cases tell the customer what to expect, thedeveloper what to code, the technical writer what to document,and the tester what to test. He proposed three-step process togenerate test cases from a fully detailed use case: (a) for eachuse case, generate a full set of use-case scenarios (b) for eachscenario, identify at least one test case and the conditions thatwill make it execute and (c) for each test case, identify the datavalues with which to test.Ryser [24] raised the practical problems in software testingas follows: (1) Lack in planning/time and cost pressure, (2)Lacking test documentation, (3) Lacking tool support, (4)Formal language/specific testing languages required, (5)Lacking measures, measurements and data to quantify testingand evaluate test quality and (6) Insufficient test quality. Theyproposed their approach to resolve the above problems. Theirapproach is to derive test case from scenario / UML Use Casediagram 1.0 and state diagram 1.0. In his work, the generationof test cases is done in three processes: (a) preliminary test casedefinition and test preparation during scenario creation (b) testcase generation from Statechart and from dependency chartsand (c) test set refinement by application dependent strategies.
 B.
 
 Related Works
This section provides the related works used in this paper,prioritize requirement methods. Donald Firesmith [16]addressed the purpose of requirement prioritization as follows:(a) Determine the relative necessity of the requirements.Whereas all requirements are mandatory, some are morecritical than others. For example, failure to implement certainrequirements may have grave business ramifications that wouldmake the system a failure, while others although contractuallybinding would have far less serious business consequences if they were not implemented or not implemented correctly (b)Help programs through negotiation and consensus building toeliminate unnecessary potential “requirements” (i.e., goals,desires, and “nice-to-haves” that do not merit the mandatorynature of true requirements) and (c) schedule theimplementation of requirements (i.e., help determine whatcapabilities are implemented in what increment). Additionally,these researches in 1980-2008 [8], [27], [28], [29], [30], [38]reveal that there are many requirement prioritization methodssuch as Binary Search Tree (BST), 100-point method andAnalytic Hierarchy Process (AHP)III.
 
RESEARCH
 
PROBLEMThis section discusses the details of research issues relatedto test case generation techniques and research problems,which are motivated this study. Every test case generationtechnique has weak and strong points, as addressed in theliterature survey. In general, referring to the literature review,the following lists major outstanding research challenges.The first research problem is that existing test casegeneration methods are lack of ability to identify domainspecific requirements. The study [5] shows that domainspecific requirements are some of the most criticalrequirements required to be captured for implementation andtesting, such as constraints requirements and database specificrequirements. Existing approaches ignore an ability to addressdomain specific requirements. Consequently, software testingengineers may ignore the critical functionality related to thecritical domain specific requirements. Thus, this paperintroduces an approach to priority those specific requirementsand generates an effective test case.The second problem is that existing black-box test casegeneration techniques aim to generate a large single test suitewith all possible test cases which maximize cover for eachscenario. Basically, they generate a huge number of test caseswhich are impossible to execute given limited time andresources. As a result, those unexecuted test cases are uselessand it is unable to identify suitable test cases for execution.IV.
 
PROPOSED
 
METHOD
 A.
 
Test Case Generation Process
This section presents a new high-level process to generate aset of test cases introduced by using the above comprehensiveliterature review and previous works [43].
Autonomous System Research Laboratory, Faculty of Science andTechnology, Assumption University.
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201023http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
 
Figure 1. A Proposed Process to Generate Test Cases
From the above figure, there are two test case generationprocesses: existing and proposed process. The left-hand sideshows an existing process to generate test cases directly fromdiagrams. Meanwhile, the right-hand side proposes to add anadditional requirement prioritization process before generatingtest cases. The requirement prioritization process aims to beable to effectively handle with a large number of requirements.The objective of this process is to prioritize and organizerequirements in an appropriate way in order to effectivelydesign and prepare test cases [16], [25], [37]. There are twosub-processes: (a) classify requirements and (b) prioritizerequirements.Our study [51], [52], [53], [54] shows that a marketingperspective concentrates on two factors: customer’s needs andcustomer satisfaction. We apply that perspective to therequirement prioritization and propose the following:
Figure 2. Classify Requirement on Marketing’s Perspective
From the above figure, the horizontal axis presents acustomer’s need while the vertical axis represents a customersatisfaction. There are four groups of requirements based onthose two factors: delight, attractive, indifferent and basic.First, the delight requirement is known as ‘nice-to-have’requirement. If this requirement is well fulfilled, it will increasethe customer satisfaction. Otherwise, it will not decrease thesatisfaction. Second, the attractive requirement is called as‘surprise’ or ‘know your customer’ requirement. Thisrequirement can directly increase the customer satisfaction if itis fulfilled. Marketers and sales [53] believe that if we candeliver this kind of requirement, it will impress customers andsignificantly improve the customer satisfaction. Third, theindifferent requirement is a requirement that customer does notconcentrate and it will not impress customers at all. In thecompetitive industry, this requirement may be fulfilled, butthere are no any impacts to the customer satisfaction. Last, thebasic requirement is a mandatory requirement that customersbasically expect. Therefore, if this requirement is welldelivered, it will not increase the customer satisfaction.Furthermore, our study reveals that the requirement can besimply divided into two types: functional and non-functionalrequirement. Our study also presents that functionalrequirements can be categorized into two groups: domainspecific requirement [5] (or known as constraints requirement)and behavior requirement. The following shows therequirement classification used in this paper:
Figure 3. Classify Requirement on Software Engineer
From the above figure, functional requirement is arequirement that customers directly are able to provide. Thenon-functional requirement is a requirement that is givenindirectly. The domain specific or constraints requirement is arequirement relative to any constraints and business rules in thesoftware development. Meanwhile, the behavior requirement isa requirement that describes a behavior of system. Once therequirement is classified based on previous two perspectives,the next process is to prioritize requirements based on return oninvestment (or ROI) [51], [52], [53]. From businessperspective, ROI is the most important factor to assess theimportant of each requirement. The following presents aranking tree by combining those two perspectives.
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201024http://sites.google.com/site/ijcsis/ISSN 1947-5500

Activity (5)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
Rosen Rusev liked this
cowga liked this
Shamina Nazar liked this

You're Reading a Free Preview

Download
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->