IntroductiontoSoftwareQuality Assurance

BasedonD.GalinCh14,5andR.PattonCh1

Objectives

WhatisSoftwareQuality? WhyisSoftwareQualityimportant? WhatisSoftwareQualityAssurance? SoftwareQualityfactors ElementsofSoftwareQualityAssurance Developmentandqualityplans ProcessMaturitymodels

WhatisSoftware?
AccordingtotheIEEE Softwareis: Computerprograms,procedures,andpossibly associateddocumentationanddatapertainingto theoperationofacomputersystem.

SoftwareErrors,softwarefaultsand softwarefailures

Bug/defect/fault consequence of a human error

results in non-conformance to requirements manifests as failure in running software

Softwaredevelopmentprocess softwareerror softwarefault softwarefailure

NineCausesofSoftwareErrors
1.Faultyrequirementsdefinition 2.Clientdevelopercommunicationfailures 3.Deliberatedeviationsfromsoftwarerequirements 4.Logicaldesignerrors 5.Codingerrors 6.Noncompliancewithdocumentationandcodinginstructions 7.Shortcomingsofthetestingprocess 8.Userinterfaceandprocedureerrors 9.Documentationerrors

Majorityofdefectsareintroducedinearlierphases
Phase Requirements Design Code Others Percentage of Effort to fix defects defects 56 82 27 13 7 1 10 4

Developmentprocessrelationto defects

Relativecostoffixingdefects
Phase in which found Requirements Design Coding Unit/Integration testing System/Acceptance testing Production Cost Ratio 1 36 10 15 40 30 70 40 1000

Requirementsarethe topreasonfora projectsuccessor failure

WhatisSoftwareQuality?

Conformancetorequirements Lack of bugs Low defect rate (# of defects/size unit) High reliability (number of failures per n hours of operation)

Measured as Mean Time To Failure (MTTF) probability of failure-free operation in a specified time

WhatisSoftwareQuality?
AccordingtotheIEEE Softwarequalityis: (1) Thedegreetowhichasystem,component,orprocess meetsspecifiedrequirements. (2) Thedegreetowhichasystem,component,orprocess meetscustomeroruserneedsorexpectations.

WhatisSoftwareQuality?
AccordingtoPressman Softwarequalityis: Conformancetoexplicitlystatedfunctionaland performancerequirements,explicitlydocumented developmentstandards,andimplicitcharacteristicsthat areexpectedofallprofessionallydevelopedsoftware

ImportanceofSoftwareQuality

Softwareisamajorcomponentofcomputersystems (about80%ofthecost)usedfor

communication(e.g.phonesystem,emailsystem) healthmonitoring, transportation(e.g.automobile,aeronautics), economicexchanges(e.g.ecommerce), entertainment, etc.

Softwaredefectsareextremelycostlyintermof

money reputation lossoflife

ImportanceofSoftwareQuality

Severalhistoricdisastersattributedtosoftware

1988shootingdownofAirbus320bytheUSSVincennescryptic andmisleadingoutputdisplayedbytrackingsoftware 1991patriotmissilefailureinaccuratecalculationoftimedueto computerarithmeticerrors LondonAmbulanceServiceComputerAidedDispatchSystem severaldeaths OnJune3,1980,theNorthAmericanAerospaceDefenseCommand (NORAD)reportedthattheU.S.wasundermissileattack. Firstoperationallaunchattemptofthespaceshuttle,whoserealtime operatingsoftwareconsistsofabout500,000linesofcode,failed synchronizationproblemamongitsflightcontrolcomputers. 9hourbreakdownofAT&T'slongdistancetelephonenetwork causedbyanuntestedcodepatch

ImportanceofSoftwareQuality

Ariane5crashJune4,1996

maidenflightoftheEuropeanAriane5launcher crashedabout40secondsaftertakeoff lostwasabouthalfabilliondollars explosionwastheresultofasoftwareerror

Uncaughtexceptionduetofloatingpointerror:conversion froma64bitintegertoa16bitsignedintegerappliedtoa largerthanexpectednumber ModulewasreusedwithoutpropertestingfromAriane4

ErrorwasnotsupposedtohappenwithAriane4 Noexceptionhandler

ImportanceofSoftwareQuality

MarsClimateOrbiterSeptember23,1999

MarsClimateOrbiter,disappearedasitbegantoorbit Mars. Costabout$US125million Failureduetoerrorinatransferofinformation betweenateaminColoradoandateaminCalifornia

OneteamusedEnglishunits(e.g.,inches,feetandpounds) whiletheotherusedmetricunitsforakeyspacecraft operation.

ImportanceofSoftwareQuality

MarsPolarLanderDecember,1999

MarsPolarLander,disappearedduringlandingon Mars Failuremorelikelyduetounexpectedsettingofa singledatabit.

defectnotcaughtbytesting independentteamstestedseparateaspects

ImportanceofSoftwareQuality

Internetvirusesandworms

Blasterworm($US525millions) Sobig.F($US500millions1billions)

Exploitwellknownsoftwarevulnerabilities

Softwaredevelopersdonotdevoteenoughefforttoapplying lessonslearnedaboutthecausesofvulnerabilities. Sametypesofvulnerabilitiescontinuetobeseeninnewer versionsofproductsthatwereinearlierversions.

Usabilityproblems

Monetaryimpactofpoorsoftwarequality(Standishgroup1995) 175,000softwareprojects/yearAverageCostperproject

ImportanceofSoftwareQuality
Largecompanies$US2,322,000 Mediumcompanies$US1,331,000 Smallcompanies$US434,000 cost$81billion

31.1%ofprojectscanceledbeforecompleted

52.7%ofprojectsexceedtheirbudgetcosting189%oforiginalestimates

cost$59billion

16.2%ofsoftwareprojectscompletedontimeandonbudget(9%forlarger companies) Largecompaniesdeliveredsystemshaveapproximatelyonly42%of originallyproposedfeaturesandfunctions 78.4%ofsmallercompaniesprojectsgetdeployedwithatleast74.2%oftheir originalfeaturesandfunctions.

TheSoftwareQualityChallenge

Theuniquenessofthesoftwareproduct

Highcomplexity Invisibilityoftheproduct Limitedopportunitiestodetectdefects(bugs)

onlyopportunityisProductdevelopment

Theenvironmentsinwhichsoftwareisdeveloped

Contracted Subjectiontocustomersupplierrelationship Requirementforteamwork Needforcooperationandcoordinationwithotherdevelopmentteams Needforinterfaceswithothersoftwaresystems Needtocontinuecarryingoutaprojectwhiletheteamchanges Needtocontinuemaintainingthesoftwaresystemforyears

SoftwareQualityFactors

McCall'ssoftwarequalityfactormodel Softwarequalityfactors

Productoperationfactors
Correctness Reliability Efficiency Integrity Usability

Productrevisionfactors
Maintainability Flexibility Testability

Producttransitionfactors
Portability Reusability Interoperability

Correctness

SoftwareQualityFactors

accuracy,completenessofrequiredoutput uptodateness,availabilityoftheinformation maximumfailurerate resourcesneededtoperformsoftwarefunction softwaresystemsecurity,accessrights abilitytolearn,performrequiredtask

Reliability

Efficiency

Integrity

Usability

SoftwareQualityFactors

Maintainability

efforttoidentifyandfixsoftwarefailures(modularity, documentation,etc)

Flexibility

degreeofadaptability(tonewcustomers,tasks,etc)

Testability

supportfortesting(e.g.logfiles,automaticdiagnostics,etc)

SoftwareQualityFactors

Portability

adaptationtootherenvironments(hardware,software)

Reusability

useofsoftwarecomponentsforotherprojects

Interoperability

abilitytointerfacewithothercomponents/systems

WhatisSoftwareQualityAssurance?
AccordingtotheIEEE Softwarequalityassuranceis: 1. Aplannedandsystematicpatternofallactions necessarytoprovideadequateconfidencethatanitem orproductconformstoestablishedtechnical requirements. 2. Asetofactivitiesdesignedtoevaluatetheprocessby whichtheproductsaredevelopedormanufactured. Contrastwith:qualitycontrol.

WhatisSoftwareQualityAssurance?
AccordingtoD.Galin Softwarequalityassuranceis: Asystematic,plannedsetofactionsnecessarytoprovide adequateconfidencethatthesoftwaredevelopmentprocess orthemaintenanceprocessofasoftwaresystemproduct conformstoestablishedfunctionaltechnicalrequirements aswellaswiththemanagerialrequirementsofkeepingthe scheduleandoperatingwithinthebudgetaryconfines.

ObjectivesofSQAindevelopment
(1) Assuringanacceptablelevelofconfidencethatthesoftware willconformtofunctionaltechnicalrequirements. (2) Assuringanacceptablelevelofconfidencethatthesoftware willconformtomanagerialschedulingandbudgetary requirements. (3) Initiationandmanagementofactivitiesfortheimprovement andgreaterefficiencyofsoftwaredevelopmentandSQA activities.

ObjectivesofSQAinmaintenance
(1) Assuringanacceptablelevelofconfidencethatthesoftware maintenanceactivitieswillconformtothefunctional technicalrequirements. (2) Assuringanacceptablelevelofconfidencethatthesoftware maintenanceactivitieswillconformtomanagerial schedulingandbudgetaryrequirements. (3) Initiateandmanageactivitiestoimproveandincreasethe efficiencyofsoftwaremaintenanceandSQAactivities.

ThreeGeneralPrinciplesofQA

Knowwhatyouaredoing Knowwhatyoushouldbedoing Knowhowtomeasurethedifference

ThreeGeneralPrinciplesofQA

Knowwhatyouaredoing

understandwhatisbeingbuilt,howitisbeingbuilt andwhatitcurrentlydoes supposeasoftwaredevelopmentprocesswith

managementstructure(milestones,scheduling) reportingpolicies tracking

ThreeGeneralPrinciplesofQA

Knowwhatyoushouldbedoing

havingexplicitrequirementsandspecifications supposeasoftwaredevelopmentprocesswith

requirementsanalysis, acceptancetests, frequentuserfeedback

ThreeGeneralPrinciplesofQA

Knowhowtomeasurethedifference

havingexplicitmeasurescomparingwhatisbeing donefromwhatshouldbedone fourcomplementarymethods:

formalmethodsverifymathematicallyspecifiedproperties testingexplicitinputtoexercisesoftwareandcheckfor expectedoutput inspectionshumanexaminationofrequirements,design, code,...basedonchecklists metricsmeasuresaknownsetofpropertiesrelatedtoquality

SoftwareQualityAssurance

SQA:Comprehensivelifecycleapproachconcernedwith everyaspectofthesoftwareproductdevelopmentprocess Includes

comprehensivesetofqualityobjectives measurablequalityattributes(qualitymetrics)toassessprogress towardtheobjectives quantitativecertificationtargetsforallcomponentofthesoftware developmentprocesses. customerproductrequirements, customerqualityrequirements,and corporatequalityrequirements.

Takesintoaccount:

SQA
SQEincludes

Requirement Architecting LowDesign Coding

validation

verification

Verification

arewebuildingtheproductright? performedattheendofaphasetoensure thatrequirementsestablishedduring previousphasehavebeenmet

verification

verification

Validation

arewebuildingtherightproduct? performedattheendofthedevelopment processtoensurecompliancewithproduct requirements

verification

SQA
SQAincludes

DefectPrevention
preventsdefectsfromoccurringinthefirstplace

Activities:training,planning,andsimulation findsdefectsinasoftwareartifact Activities:inspections,testingormeasuring

Defectsdetection

Defectsremoval
isolation,correction,verificationoffixes

Activities:faultisolation,faultanalysis,regressiontesting

SQA

TypicalactivitiesofaSQAprocess

Requirementsvalidation. Designverification. Staticcodechecking(inspection/reviews). Dynamictesting. Processengineeringandstandards. Metricsandcontinuousimprovement