Comparative Study of the Effectiveness of Ad Hoc
,
Checklist- and
Perspective-based Software Inspection Reading TechniquesOlalekan S. Akinola Ipeayeda Funmilola Wumi
Solom202@yahoo.co.uk funmipy12@yahoo.com
Department of Computer Science, Department of Physical Sciences,University of Ibadan, Ibadan, Nigeria Ajayi Crowther University, Oyo Town, Nigeria
ABSTRACT
Software inspection is said to be inevitable in order to ensure software quality assurance. Nevertheless,there have been controversies on which defect detection techniques should be applied in softwaredocument inspection. This work comparatively study the effectiveness of three software inspectiontechniques: Ad
Hoc,
Perspective-based and Checklist-based defect detection techniques. Paper-basedinspections of software artifact were carried out on an industrial code artifact seeded with forty bugs. Anexperimental 3 x 3 x 4 factorial design with three defect detection techniques (checklist-based, Adhocand perspective-based) as independent variables, three dependent variables (inspection effectiveness,effort and false positives) and four teams for each defect detection methods was used for the experiment.The data obtained were subjected to tests of hypotheses using One-way ANOVA, Post-Hoc tests andMean coefficients. Results from the study indicate that there were significant differences in the defectdetection effectiveness and effort in terms of time taken in minutes reported by the reviewers usingperspective-based, ad hoc and checklist-based based reading techniques in the industrial settings.Key words: Software inspection, Ad Hoc reading technique, Checklist reading technique, Perspectivereading technique
1. INTRODUCTION
The process of improving software quality hasbeen a growing discussion in the few decades.Software quality can be defined to be software thatsatisfies the needs of the users and theprogrammers involved in it or as the customer’sperception of how the system work. Softwareinspection is a fundamental component of thesoftware quality assurance process. It is a processwhereby a group of software competent peoplecritically checks a piece of software milestone fordetecting defects [1]. Inspection improves thequality of software products, such as understand-ability, portability, maintainability, testability, etc.Its success has always been demonstrated in manypublished articles.Software Inspections are a formalized, structuredform of
peer reviews
. They are an extremely cost-effective quality assurance technique that can beapplied to any type of software project deliverable,such as Requirements documents, Designdocuments, Code, and other items such as testplans and user documents. For most softwareorganizations, Software Inspections are
the most important single process improvement
. Accordingto Capers [4],
“… formal design and codeinspections rank as the most effectivemethods of defect removal yet discovered … (defect removal) can top 85%, about twice those of any form of testing.”
Since the year Fagan developed theinspection process in the early 70s at IBM;there have been many variations of theprocess put forth by others. The aim is touncover faults in the products, rather thanto correct them. The goal of inspectionmeetings is to collect the faults discoveredand bring synergy (process gains) to thesoftware inspection. It is believed that thecombination of different viewpoints, skillsand knowledge from many reviewerscreates this synergy [10].
Ad Hoc
, checklist-based and perspective-based reading techniques are the threecommonly used inspection artifactsreading techniques. To the best of ourknowledge, experiments comparing theeffectiveness of all three together arescarce. This research is thereforeconducted to find if there are any
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 11, November 2011163http://sites.google.com/site/ijcsis/ISSN 1947-5500
significant differences in the effectiveness of reviewers using Perspective-based, Checklist-based and Ad hoc code reading techniques in anindustrial code setting. Thirty volunteeredreviewers from ten software houses in Nigeriawere used to carry out code inspection on thevisual-basic large-sized code artifact.
1.1 Research Hypotheses
Three hypotheses were stated for this experimentas follows.Ho1: There is no significant difference among theeffectiveness of reviewers usingPerspective-based, Ad hoc and Checklistreading techniques in distributed codeinspection.Ho2: There is no significant difference among theeffort taken by reviewers usingPerspective-based, Ad hoc and Checklisttechniques in distributed code inspection.Ho3: There is no significant difference among thefalse positives reported by reviewers usingPerspective-based, Ad hoc and Checklisttechniques in distributed code inspection.
2. SOFTWARE INSPECTIONREADING TECHNIQUES
Software inspection encompasses a set of methodsin which the purpose is to identify and locatefaults in software. Software inspection is a peerreview process led by software developers who aretrained in inspection techniques [7]. MichaelFagan [10] originally developed the softwareinspection process ‘out of sheer frustration’ [13].Since Fagan developed the inspection process inthe early 1970s at IBM, there have been manyvariations of the process put forth by others.Overall, the aim in any review process is to applyinspection to the working product as early aspossible so that major faults are caught before theproduct is released.A reading technique can be defined as a series of steps or procedures whose purpose is to guide aninspector in acquiring a deep understanding of theinspected software product [8]. Thecomprehension of inspected software products is aprerequisite for detecting subtle and/ or complexdefects, those often causing the most problems if detected in later life-cycle phases. In a sense, areading technique can be regarded as a mechanismor strategy for individual inspector todetect defects in the inspected product.There are many reading techniques thatfocus on finding as many detects aspossible but three among the readingtechniques were used in carrying out thiswork; ad hoc, checklist-based andperspective-based reading techniques.According to Porter and Votta [11], defectdetection techniques range in prescriptionfrom intuitive, non-systematic proceduressuch as ad hoc or checklist techniques, toexplicit and highly systematic proceduressuch as Perspective technique.Ad-hoc reading, by nature, offers verylittle reading support at all since a softwareproduct is simply given to inspectorswithout any direction or guidelines on howto proceed through it and what to look for.However, ad-hoc does not mean thatinspection participants do not scrutinizethe inspected product systematically. Theword ad-hoc only refers to the fact that notechnical support is given to them for theproblem of how to detect defects in asoftware artifact. In this case, defectdetection fully depends on the skill, theknowledge, and the experience of aninspector. Training sessions in programcomprehension as presented in Rifkin andDeimel [12], may help subjects developsome of these capabilities to alleviate thelack of reading support.Perspective-based reading technique givesreviewers a set of procedures to inspectsoftware products for defects. Theperspective-based reading techniqueinstructs the reviewer to perform an activereview by assigning different perspectivesto each reviewer. Common perspectivesare user, tester, and designer.Checklist-based reading techniquereviewers use a checklist which guidesthem regarding what kind of faults to look for. The reviewers read the documentusing the checklist to guide the focus of their review. Checklist-based offersstronger, boilerplate support in the form of questions inspectors are to answer whilereading the document
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 11, November 2011164http://sites.google.com/site/ijcsis/ISSN 1947-5500
3. RESEARCH METHODOLOGY
3.1 Subjects
The subjects used for this research were theSoftware professionals drawn from ten softwarehouses in Nigeria. Software professionals werechosen as subjects for this research because resultsobtained with professionals would make us topredict what may likely happen at industry level.
3.2 Experimental artifact andInstrumentation
The artifact inspected was a large sized Visualbasic 6.0 language industrial code. It calculatesstaff allowances such as domestic, responsibility,hazard, housing, leave, medical, transport, utilityand so on in order to compute staff salarymonthly, annual salary and arrears as well as somedeductions(such as tax, cooperative, officialhousing accommodation rent and so on) to bemade on staff salary. The artifact which was 500lines of code was tested okay before it was seededwith 40 bugs which are syntax, semantics andlogical in nature.The designed instruments for this experiment werethe individual preparation forms, the experimentalartifact (code) and the collection meeting forms.The experimental artifact and individualpreparation forms were given to each reviewer.The individual preparation form was filled duringpreparation by each reviewer. During individualpreparation, each reviewer recorded the start andthe end times for the review of the artifact. Theline number of the suspected defect and thedescription of defects suspected were alsorecorded in the forms.The meeting form was filled in at the collectionmeeting. The start and end times of the teamcollection meetings held was filled on thecollection meeting forms. The line numbers of thedefect and the defect description was recorded onthe collection meeting forms. Most importantly,the teams’ identification numbers were filled onthe collection meeting form in order to identifyeach team.
3.3 Conducting the Experiment
The experiment was monitored and conducted bythe researchers. The software professionals wereused for the experiment without specifying anyparticular year of experience. There was no specialtraining given to them on Visual Basicprogramming because they are softwareprofessionals who are conversant with thelanguage used for the artifact.Nevertheless, the reviewers were givensome initial code inspection trainingsbefore the real experiments were carriedout.During individual preparation, reviewersexamined the artifact in order to identifythe bugs seeded in them. In perspective-based technique, reviewers were assignedwith a particular role to play (a designer,tester, reader and user) and in checklist-based, a Visual basic checklist questiondesigned by the researchers was given toreviewers in order to guide them in fishingout bugs. There was no particular timegiven to the reviewers for their individualpreparations. All the suspected defectswere recorded on the individualpreparation form.Moreover, before the commencement of the team collection meeting, the individualpreparation forms were collected by theresearcher in order that the reviewers donot add to their preparation forms anydefects that were not found during teamdefects collection meeting. During defectscollection meetings, there was no specificduration given to the reviewers for theartifact inspections.During the defect collection meetings, oneof the reviewers in each team serves as thereader, moderator and recorder. While inthe meetings, reviewers brought up newdefects or discussed any defects foundduring the individual preparation. Alldefects found were recorded in the teamdefects collection meeting forms by therecorder.Four different teams were created each forthe Perspective-based, Ad hoc andChecklist-based reviewers. In order toeliminate bias in the results, team sizeswere duplicated for each of the groups. Forinstance, teams 001A and 001B in Table 1are for team size of one, while teams 002A
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 11, November 2011165http://sites.google.com/site/ijcsis/ISSN 1947-5500
Search History:
Result 00 of 00
00 results for result for
p.
Comparative Study of the Effectiveness of Ad Hoc, Checklist- and Perspective-based Software Inspection Reading Techniques
Software inspection is said to be inevitable in order to ensure software quality assurance. Nevertheless,
there have been controversies on which defect detection techniques should be applied in software document inspection. This work comparatively study the… (More)
there have been controversies on which defect detection techniques should be applied in software document inspection. This work comparatively study the… (More)
294
Reads
0
Readcasts
0
Embed Views
More From This User
Notes
Load more
