Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
2Activity
0 of .
Results for:
No results containing your search query
P. 1
White-Box Test Reduction Using Case-Based Maintenance

White-Box Test Reduction Using Case-Based Maintenance

Ratings: (0)|Views: 228|Likes:
Published by ijcsis
Abstract— Software testing has been proven that it takes around 50-70% of the costs associated with the large development of commercial software systems. Many reduction techniques have been proposed to reduce costs. Unfortunately, the cost is usually over budget and those techniques are failed to reasonably control costs. The primarily outstanding research issues, motivated this study, are a large number of redundancy test cases and a decrease of ability to detect faults. To resolve these issues, this paper proposes new deletion algorithms to minimize a number of white-box test cases, while maximizing an ability to reveal faults, by using a concept of case-based maintenance. Our evaluations have shown that the proposed techniques can significantly reduce a number of unnecessary test cases while preserving the capability of fault detection.

Keywords-component; Test reduction, test case reduction, deletion method, case based maintenance and test case deletion
Abstract— Software testing has been proven that it takes around 50-70% of the costs associated with the large development of commercial software systems. Many reduction techniques have been proposed to reduce costs. Unfortunately, the cost is usually over budget and those techniques are failed to reasonably control costs. The primarily outstanding research issues, motivated this study, are a large number of redundancy test cases and a decrease of ability to detect faults. To resolve these issues, this paper proposes new deletion algorithms to minimize a number of white-box test cases, while maximizing an ability to reveal faults, by using a concept of case-based maintenance. Our evaluations have shown that the proposed techniques can significantly reduce a number of unnecessary test cases while preserving the capability of fault detection.

Keywords-component; Test reduction, test case reduction, deletion method, case based maintenance and test case deletion

More info:

Published by: ijcsis on Oct 10, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

06/29/2011

pdf

text

original

 
 
White-Box Test Reduction Using Case-BasedMaintenance
Siripong Roongruangsuwan
 
Autonomous System Research LaboratoryFaculty of Science and Technology, Assumption UniversityBangkok, ThailandP4919742@au.edu
Jirapun Daengdej
Autonomous System Research LaboratoryFaculty of Science and Technology, Assumption UniversityBangkok, Thailand jirapun@scitech.au.edu
 Abstract
— Software testing has been proven that it takes around50-70% of the costs associated with the large development of commercial software systems. Many reduction techniques havebeen proposed to reduce costs. Unfortunately, the cost is usuallyover budget and those techniques are failed to reasonably controlcosts. The primarily outstanding research issues, motivated thisstudy, are a large number of redundancy test cases and adecrease of ability to detect faults. To resolve these issues, thispaper proposes new deletion algorithms to minimize a number of white-box test cases, while maximizing an ability to reveal faults,by using a concept of case-based maintenance. Our evaluationshave shown that the proposed techniques can significantly reducea number of unnecessary test cases while preserving thecapability of fault detection.
 Keywords-component; Test reduction, test case reduction, deletion method, case based maintenance and test case deletion
I.
 
I
NTRODUCTION
 Software Testing is an empirical investigation conducted toprovide stakeholders with information about the quality of theproduct or service under test [13], with respect to the context inwhich it is intended to operate. Software Testing also providesan objective, independent view of the software to allow thebusiness to appreciate and understand the risks of implementation of the software. The software testingtechniques include the process of executing a program orapplication with the intent of finding software bugs. It can alsobe stated as the process of validating and verifying thatsoftware meets the business and technical requirements thatguided its design and development, so that it works asexpected. Software Testing can be implemented at any time inthe development process; however, the most test effort isemployed after the requirements have been defined and codingprocess has been completed.Many researchers [6], [7], [8], [9], [10], [19], [24], [25],[26], [28], [30], [36], [37], [39] have proven that these test casereduction methods can reserve the ability to reveal faults.However, there are many outstanding research issues in thisarea. The motivated research issues are: a large number of testcases particularly redundancy test cases and a decrease of ability to detect faults. The study shows that test case reductionmethods have been researched over a long period of time, suchas test case prioritization, random approach and coverage-basedtest case reduction techniques. Also, the study reveals thatcoverage-based approaches are wildly used and researched.Therefore, we concentrate on an approach to reduce test casesbased on the coverage factor. Many coverage factors have beenproposed over a long period of time. Unfortunately, existingfactors and test case reduction methods ignore the complexityand impact of test cases. Thus, we propose to reduce a numberof test cases by considering both of test case complexity andimpact.Our study [5] shows that one of effective approaches thatsignificantly reduce a number of redundancy test cases is toapply the concept of artificial intelligent. There are manyartificial intelligent concepts, such as neutral network, fuzzylogic, learning algorithms and case-based reasoning (CBR).CBR is one of the most popular and actively researched areasin the past. The researches [4], [5], [26], [35] show that CBRhas identical problems as same as software testing topic, suchas a huge number of redundancy cases and a decrease of system’s ability to resolve problems.Fundamentally, there are four steps in the CBR system,which are: retrieve, reuse, revise and retain. These steps canlead to a serious problem of uncontrollably growing cases inthe system. However, the study shows that there are manyproposed techniques in order to control a number of cases inthe CBR system, such as add algorithms, deletion algorithmsand maintenance approaches. CBR have been investigated byCBR researchers in order to ensure that only small amounts of efficient cases are stored in the case base.The previous work [27] shows that deletion algorithms arethe most popular and effective approaches to maintain a size of the CBR system. There are many researchers have proposedseveral deletion algorithms [4], [20], [35], such as randommethod, utility approach and footprint algorithm. Thesealgorithms aim to: (a) remove all redundancy or unnecessarycases (b) minimize a number of cases and (c) maintain theability of solving problems. Nevertheless, each technique hasstrength and weakness. Some methods are suitable forremoving cases. Some methods are perfectly suitable forreducing time. Some may be used for reserving the problemsolving capability. Eventually, the previous work [27]discovered several effective methods (e.g. confidential casefiltering method, coverage value algorithm and confidentialcoverage approach) to remove those cases, minimize size of 
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201032http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
 
CBR and reduce amount of time, while preserving the ability of CBR system’s problem solving skill. Therefore, this paperapplies those effective deletion techniques to resolve theproblems of software testing.In the light of software testing, the proposed techniquesfocus on how to maintain the test case while maintaining thecapability of fault detection. It is appear that test cases in thispaper are treated as cases in the CBR system. Also, there is anassumption that a given set of test cases are generated by apath-oriented test case generation technique. The path-orientedtechnique is widely used for a white-box testing that derivestest cases from available source code.Section 2 discusses an overview of test case reductiontechniques and approach to maintain CBR. Section 3 providesa definition of terminologies used in this paper. Section 4 liststhe outstanding research issues motivated this study. Section 5proposes deletion algorithms using the concept of CBR.Section 6 describes an evaluation method and discusses anevaluation result. The last section represents all sourcereferences used in this paper.II.
 
L
ITERATURE
R
EVIEW
 This section describes an overview of test case reductiontechniques and the concept of case based maintenance. Thefollowing describes those two areas in details.
 A.
 
Test Case Reduction Techniques
This section discusses and organizes test case reduction (orTCR) techniques researched in 1995-2006. This study showsthat there are many researchers who proposed a method toreduce unnecessary test cases (also known as redundancy testcases), like Offutt [2], Rothermel [8], McMaster [25] andSampth [31]. These techniques aim to remove and minimize asize of test cases while maintaining the ability to detect faults.The literature review [6], [7], [8], [9], [10], [11], [19], [24],[25], [36], [37], [39] shows that there are two types of reduction techniques, which are: (a) pre-process and (b) post-process. First, the pre-process is a process that immediatelyreduces a size of test cases after generating. Typically, it isoccurred before regression testing phase. Second, the post-process is a process that maintains and removes unnecessarytest cases, after running the first regression testing activities.Although these techniques can reduce the size of test cases, butthe ability to reveal faults seems slightly to be dropped.However, Jefferson Offutt [5] and Rothermel [6], [7], [8], [9],[10], [19], [20], [21], [32] has proven that these test casereduction techniques have many benefits, particularly duringthe regression testing phase, and most of reduction techniquescan maintain an acceptable rate of fault detection. Theadvantages of these techniques are: (a) to spend less time inexecuting test cases, particularly during the regression testingphase (b) to significantly reduce time and cost of manuallycomparing test results and (c) to effectively manage the testdata associated with test cases. This study proposes a new “2C”classification of test case reduction techniques, classified basedon their characteristics, as follows: (a) coverage-basedtechniques and (b) concept analysis-based techniques.
 B.
 
Case-Based Maintenance (CBM)
Due to the CBR’s life cycle [16], the case base size growsrapidly. That is caused a serious problem directly, for instance,duplicate data, inconsistency data, incorrect data, and anexpense of searching for an appropriate case in a large casebase size. CBR can be classified as one of the ArtificialIntelligence algorithms. CBR solves new problem by retrievingthe similar case from the existing case base and then adapts theexisting case according to the target problem. Over the time,CBR is growing. When the uncontrollable case-based growth isoccurred, the performance of CBR is decreasing. Therefore, themaintenance process is required in order to preserve or improvethe performance of the system. The process of maintainingCBR is called CBM. David C. Wilson [8] presented the overallconcepts of CBR and case based maintenance. This paperfocused on the case based maintenance (CBM) approach interm of the framework. In other words, this paper described thetype of data collection and how the case based maintenanceworks. There were so many policies for CBM, for example,addition, deletion, and retain.
“CBM was defined as the process of refining a CBR system’scase-base to improve the system’s performance. It implements policies for revising the organization or contents(representation, domain content, accounting information, or implementation) of the case-base in order to facilitate futurereasoning for a particular set of performance objectives.”
These studies [2], [3], [4], [16], [17], [27], [35] reveal thatseveral deletion algorithms have been proposed. For example, arandom approach (RD), utility deletion algorithm (UD),footprint deletion algorithm (FD), footprint utility deletionalgorithm (FUD) and iterative case filtering algorithm (ICF).RD is the simplest approach, which removes the caserandomly. UD deletes the case that has minimum utility value.Footprint algorithm uses the competence model and removesthe auxiliary case from the system. FUD is a hybrid approachbetween Utility algorithm and Footprint algorithm, and isconcerned with the competence model and the utility value.Finally, ICF focuses on the case, which the reachability set isgreater than the coverage set [16], [27].III.
 
DEFINITION
 This section describes a definition of CBR terminologiesused in the software testing area.
TABLE I. D
EFINITIONS OF
CBR
FOR
S
OFTWARE
T
ESTING
Element CBR Software TestingCoveragesetCoverage Set is theset of target problems,which it can be usedto solve successfully[4].Coverage set means aset of stages / elements, which theycan be used to testsuccessfully andreveal faults.ReachabilitysetReachability Set is theset of case bases thatcan be used to solveReachability set meansa set of test cases thatcan be used to reveal
Autonomous System Research Laboratory, Faculty of Science andTechnology, Assumption University.
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201033http://sites.google.com/site/ijcsis/ISSN 1947-5500
 
the target problem [4]. faults.CompetencesetCompetence is therange of the targetproblem that can besolved successfully[4].Competence is therange of ability toreveal faults that canbe used to testsuccessfully.AuxiliarysetAuxiliary Case is acase that does nothave a direct effect onthe competence of asystem when it isdeleted [4].Auxiliary case is a testcase that does not havea direct effect on thecompetence of asystem when it isremoved.Pivot set Pivotal Case is thecase that does have adirect effect on thecompetence of asystem if it is deleted[1], [29].Pivotal case is a testcase that does have adirect effect on theability to reveal faultsif it is deleted.
IV.
 
RESEARCH PROBLEM
 This section discusses the details of research issuesmotivated this study. The literature review reveals that [13],[23], [24], [25], [31], [38] those research issues are: (a) a largenumber of redundancy test cases and (b) a decrease of theability to reveal faults. These research issues can be elaboratedin details as follows:First, the literature review shows that redundancy test casesare test cases tested by multiple test cases. Many test cases thatare designed to test the same things (e.g. same functions, sameline of code or same requirements) are duplicated. Thoseduplicated tests are typically occurred during testing activities,particularly during regression testing activities [13], [23], [24],[25], [31], [38]. Those duplicated tests can be eventuallyremoved in order to minimize time and cost to execute tests.The following shows an example of redundancy test cases.
Figure 1. An Example of Control Flow Graph
From the above figure, there are two test cases, with theduplicated sequence and expected result, designed to test a“withdraw” feature in ATM machine. The sequence of the firsttest case is: (a) insert ATM card (b) insert PIN (c) select“withdraw” (d) select account type (e) identify an amount (f)click “OK” button (g) receive money and (h) receive card. Thesequence of the second test case is similar to the first one.However, the additional step in the second case is to inquiry abalance amount before withdrawing the money. Therefore, it isappear that the first test case is a part of the second test case.We call the first test case as a redundancy test case.The study shows that there are many proposed methods todelete those duplicated test cases such as McMaster's work [24][25], Jeff's method [13] and Khan's approach [23]. Also, thestudy shows that one of the most interesting research issues isto minimize those duplicated tests and reduce cost of executingtests. Although there are many proposed methods to resolvethat issue, that issue is still remaining. Thus, it is a challengefor researchers to continuously improve the ability to removeduplicated tests.Last, test cases are designed to reveal faults during softwaretesting phase. The empirical studies [8], [10], [19], [20], [21],[30], [32], [39] describe that reducing test cases may impact tothe ability of detect faults. Many reduction methods decrease acapability of testing and reveal those faults. Therefore, one of outstanding research challenges for researchers is to removetests while preserving the ability to defect faults.V.
 
PROPOSED METHOD
 For evolving software, test cases are growing dramatically.The more test cases software test engineers have, the more timeand cost software test engineers consume. The literature reviewshows that regression testing activities consume a significantamount of time and cost. Although, a comprehensive set of regression selection techniques [8], [9], [10], [19] has beenproposed to minimize time and cost, there is an available roomto minimize size of tests and clean up all unnecessary testcases. Thus, removing all redundancy test cases is desirable.There are many approaches to reduce redundancy test casesand applying an artificial intelligent concept in the test casereduction process is an innovated approach. The literaturereview [5], [27] shows that there are many areas of artificialintelligent concept, such as artificial neutral network, fuzzylogic, learning algorithms and CBR concept. Also, it revealsthat CBR has a same research issue as software testing has. Theissue is that cases in the CBR system will be consistencygrowing bigger and larger all the time. There are four steps inCBR that can uncontrollably grow a size of the system:retrieve, reuse, revise and retain. Therefore, many CBR papersaim to reduce
all redundancy cases, known as “deletionalgorithms”. The smaller size of CBR system is better anddesirable. Due to the fact that CBR has the same problem assoftware testing and this paper focuses on reduction methods,therefore, this paper proposes to apply CBR deletionalgorithms to the test case reduction techniques.This paper introduces three reduction methods that applyCBR deletion algorithms: TCCF, TCIF and PCF methods.
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 6, September 201034http://sites.google.com/site/ijcsis/ISSN 1947-5500

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->