You are on page 1of 38

RequirementsValidation&

Verification

Basedonmaterialfrom:
GeraldKotonyaandIanSommerville:RequirementsEngineeringProcessesand
Techniques,


WhatareRequirementsValidation&
Verification
Goals,
Requirements

Validation

Validation:ensuresthatthesoftwarebeing
developed(orchanged)willsatisfyits Requirements
Specification
stakeholders
RequirementsValidationchecksthesoftware
requirementsspecificationagainststakeholders
Design
goalsandrequirements

System

WhatareRequirementsValidation&
Verification
Goals,
Requirements

Validation

Verification:ensuresthateachstepfollowed
intheprocessofbuildingthesoftwareyields Requirements
Specification
therightproducts
RequirementsVerificationchecksthe
consistencyofthesoftwarerequirements
Design
specificationartefactsandothersoftwareVerification
developmentproducts(design,
implementation,...)againstthespecification
System

RequirementsV&V
TheMachineanditsWorld(M.Jackson
1995)

problem solution
Hardware(C)
interface
Software(P)
domain system


Domainproperties(D) Specification(S)


Requirements(R) Verificationquestion:ifthe
Validationquestion:ifthedomain hardwarehastheunderlying
hastheunderlyingpropertiesD, propertiesC,thenbyaddingthe
thenbyaddingthepropertiesofS, propertiesthatittakesbecauseof
itwillalsohavethepropertiesR thesoftwareP,itwillsatisfythe
D,SR specificationS

C,PS
RequirementsV&V
Example
RequirementR:

Reversethrustshallonlybeenabledwhentheaircraftis
movingontherunway

DomainPropertiesD:
Wheelpulsesonifandonlyifwheelsturning
Wheelsturningifandonlyifmovingonrunway

0SpecificationS:
Reversethrustenabledifandonlyifwheelpulseson
DoesD,SR?
Arethedomainassumptions(D)right?
Aretherequirements(R)whatarereallyneeded?

RequirementsV&V

ShowingD,SRisdifficultwithnaturallanguage
ambiguities,expressiveness,...

Moreeffectivewithformalmethods
basedonmathematicallyformalsyntaxandsemantics
provingcanbetoolsupported


RequirementsV&VversusAnalysis

Bothhaveseveralactivitiesincommon
readingrequirements,problemanalysis,meetingsand
discussions,...

Analysisworkswithrawrequirementsaselicitedfromthe
systemstakeholders
developasoftwarerequirementsspecificationdocument

RequirementsV&Vworkswithasoftwarerequirements
specificationwithnegotiatedandagreedrequirements

checkthatthisspecificationisaccurate

RequirementsV&VTechniques

TypicalRequirementsV&Vapproaches:


Tracingapproaches

Prototyping

Testing

Usermanualwriting

Formalvalidation

Reviewsandinspections

Walkthroughs

Formalinspections


Checklists
RequirementsV&VTechniques
Tracingapproaches

Variouscheckscanbedoneusingtracing

Checkingthatallelicitationnotesarecovered

Checkinggoalsagainsttasks,features,requirements
Canbedonebydevelopingatraceabilitymatrix

Ensuresthatrequirementshavebeentakenintotoconsideration


ifnotthereshouldbeareason
Ensurethateverythinginthespecificationisjustified


RequirementsV&VTechniques
Prototyping

Prototypesaregoodwaytodemonstratetherequirements
andhelpstakeholdersdiscoverproblems
helpvalidaterequirements
moreaccessiblethanspecification

Differenttypesofprototypesmaybeused
horizontal,vertical
evolutive,throwaway,
...

Executablespecificationscanbeusedasprototypes

RequirementsV&VTechniques
Prototyping

Prototypingbasedvalidationsteps:
1.chooseprototypetesters
2.developtestscenarios

carefulplanningisrequiredtodrawupasetoftestscenarios
whichprovidebroadcoverageoftherequirements.

endusersshouldntjustplayaroundwiththesystemasthis
mayneverexercisecriticalsystemfeatures.
3.executescenarios
4.documentproblems

usingaproblemreportingtool

RequirementsV&VTechniques
Testing
Eachrequirementshouldbetestable


itshouldbepossibletodefineteststocheckwhetherornota
requirementhasbeenmet.

Inventingrequirementstestsisaneffectivevalidation
technique

missingorambiguousinformationintherequirementsdescription
maymakeitdifficulttoformulatetests

Eachfunctionalrequirementshouldhaveanassociatedtest

Somesoftwaredevelopmentapproaches(e.g.agilemethods)
startfromtestcases(TestDrivenDevelopment)

RequirementsV&VTechniques
Testing

RequirementsbasedTestCasesshouldbewrittensuchthat
theycanbetracedtorequirements

haverequirementidentifiersaspartofthetestcasedescription

Somerequirementsarehardertovalidateusingtesting

somenonfunctionalrequirementssuchasreliability,

exclusiverequirements(saysomethingshouldn'thappen)


RequirementsV&VTechniques
WritingUserManual

Forcesadetailedlookatrequirements
particularlyrequirementsrelatedtousability

Typicalinformationinausermanual
Descriptionofthefunctionalityandhowitisimplemented
Whichpartsofthesystemhavenotbeenimplemented
Howtogetoutoftrouble
Howtoinstallandgetstartedwiththesystem


RequirementsV&VTechniques
FormalV&V

Waystocheckifaformalspecificationhascertaindesirable
properties
completeness
consistency
mutualexclusion
particulartemporalproperties


RequirementsV&VTechniques
FormalV&V

Techniques:
Modelchecking(forFSMsandtemporallogic)
Theoremproving(moregeneralforanyformalspecification)

FormalV&Vinvolvescheckingallpossibleexecutionpaths
ofthespecification


RequirementsValidationTechniques
FormalV&V

ModelChecking
Automatedtechniqueforformallyverifyingfinitestateconcurrent
systems
Usedto

verifythatamodelofasystemsatisfiesaspecifiedproperty

checkcorrectnessofamodel
Model:expressedinaformallanguagewithstatetransition
semantics(possibilityofextendedstates)
Property:expressedinatemporallogic


RequirementsV&VTechniques
FormalV&V

ModelChecking
Property
modelcheckerverifies Model
(M) (P)
MP
ifnotatraceofstates
andtransitionsleading ModelChecker
totheviolationofPis
produced
No,
Yes

Majorobstacleisstate tracetoerror

explosion

RequirementsV&VTechniques
FormalV&V

OtherStateMachinebasedV&V
Reachabilityanalysischecksthat

allstatesarereachable

alleventsarehandled

therearenodeadlocks


RequirementsV&VTechniques
FormalV&V

OtherStateMachinebasedV&V
Conformancechecking

checksconformancebetweentwostatemachines(one
istypicallymoreconcrete/abstractthantheother)
Equivalencechecking

betweentwostatemachines

differentnotionsofequivalence(e.g.trace,
observation,congruence,...)

RequirementsV&VTechniques
Reviews

Agroupofpeoplereadandanalysetherequirements,lookfor
problems,meetanddiscusstheproblemsandagreeonactionsto
addresstheseproblems

Awidelyusedrequirementsvalidationtechnique
lotsofevidenceofeffectivenessofthetechnique

Canbeexpensive
carefulplanningandpreparation
prereviewchecking
useofchecklists

RequirementsV&VTechniques
Reviews

Differenttypesofreviewswithvaryingdegreeofformality
Readingandsigningoff:readingthedocumentandsigningoftoendorseit
Walkthroughs

Informal,oftenhighleveloverview.

Canbeledbyauthor/experttoeducateothersonhis/herwork.
Formalinspections

Verystructuredanddetailedreview,definedrolesforparticipantsand
preparationisneeded


RequirementsV&VTechniques
Reviews

Focusedinspections

reviewershaverolesandeachlooksonlyforspecifictypesoferrors.
Activereviews

reviewerisaskedtousethespecification

theauthorposesquestionsforthereviewertoanswerthatcanbe
answeredonlybyreadingthedocument.


RequirementsV&VTechniques
Reviews

Reviewprocess


RequirementsV&VTechniques
Reviews

Planreview
Thereviewteamisselectedandatimeandplaceforthereviewmeetingis
chosen.

Distributedocuments
Therequirementsdocumentisdistributedtothereviewteammembers

Prepareforreview
Individualreviewersreadtherequirementstofindconflicts,omissions,
inconsistencies,deviationsfromstandardsandotherproblems.


RequirementsV&VTechniques
Reviews

Holdreviewmeeting
Individualcommentsandproblemsarediscussedandasetofactionstoaddress
theproblemsisagreed.

Followupactions
Thechairofthereviewchecksthattheagreedactionshavebeencarriedout.

Revisedocument
Therequirementsdocumentisrevisedtoreflecttheagreedactions.Atthis
stage,itmaybeacceptedoritmayberereviewed


RequirementsV&VTechniques
ReviewTeam

Reviewsshouldinvolveanumberofstakeholdersdrawn
fromdifferentbackgrounds
Peoplefromdifferentbackgroundsbringdifferentskillsand
knowledgetothereview
StakeholdersfeelinvolvedintheREprocessanddevelopan
understandingoftheneedsofotherstakeholders
Reviewteamshouldalwaysinvolveatleastadomainexpertand
anenduser


RequirementsV&VTechniques
Reviews
Exampleofproblemscategorisation
Requirementsclarification

Therequirementmaybebadlyexpressedormayhaveaccidentallyomitted
informationwhichhasbeencollectedduringrequirementselicitation.
Missinginformation

Someinformationismissingfromtherequirementsdocument.Itisthe
responsibilityoftherequirementsengineerswhoarerevisingthedocument
todiscoverthisinformationfromsystemstakeholders.


RequirementsV&VTechniques
Reviews

Requirementsconflict

Thereisasignificantconflictbetweenrequirements.Thestakeholders
involvedmustnegotiatetoresolvetheconflict.
Unrealisticrequirement

Therequirementdoesnotappeartobeimplementablewiththetechnology
availableorgivenotherconstraintsonthesystem.Stakeholdersmustbe
consultedtodecidehowtomaketherequirementmorerealistic.


RequirementsV&VTechniques
Prereviewchecking

Reviewsareexpensivebecausetheyinvolveanumberofpeople
spendingtimereadingandcheckingtherequirementsdocument
expensecanbereducedbyusingprereviewchecking
onepersonchecksthedocumentandlooksforstraightforwardproblems
suchasmissingrequirements,lackofconformancetostandards,
typographicalerrors,etc.
documentmaybereturnedforcorrectionorthelistofproblems
distributedtootherreviewers


RequirementsV&VTechniques
Fagan'sInspectionProcess

Formalandstructuredinspectionprocess


RequirementsV&VTechniques
Fagan'sInspectionProcess

Characterizedbyrulesonwhoshouldparticipate,howmany
reviewersshouldparticipateandwhatrolestheyshouldplay
3to5reviewers
authorservesasthepresenterofthedocument
authorsupervisordonotattendthemeetingorgetaccesstothenotes
amoderatorisresponsibleforinitiatingtheinspection,leadingthe
meetingandensuringissuesfoundarefixed
allreviewersneedtopreparethemselvesusingchecklists
issuesarerecordedinspecialforms

Newinspectionif>5%ofdocumentneedtobereworked
criteriaisoftenmadetighter(reinspectionifanydocumenthastobe

reworkedbecausenewerrormaybeintroducedatcorrectiontime)
RequirementsV&VTechniques
RequirementsReviewChecklists


Essentialtoolforaneffectivereviewprocess
listcommonproblemareaandguidereviewers

Therearegeneralchecklistsandchecklistsforparticular
modelingandspecificationlanguages

Checklistsaresupposedtodevelopedandmaintained


RequirementsV&VTechniques
RequirementsReviewChecklists

Sampleofelementsinarequirementsreviewchecklist
Understandabilitycanreadersofthedocumentunderstandwhat
therequirementsmean?
Redundancyisinformationunnecessarilyrepeatedinthe
requirementsdocument?
Completenessdoesthecheckerknowofanymissing
requirementsoristhereanyinformationmissingfromindividual
requirementdescriptions?


RequirementsV&VTechniques
RequirementsReviewChecklists

Ambiguityaretherequirementsexpressedusingtermswhichare
clearlydefined?Couldreadersfromdifferentbackgroundsmake
differentinterpretationsoftherequirements?
Consistencydothedescriptionsofdifferentrequirementsinclude
contradictions?Aretherecontradictionsbetweenindividual
requirementsandoverallsystemrequirements?


RequirementsV&VTechniques
RequirementsReviewChecklists

Sampleofelementsinarequirementsreviewchecklist
Organisationisthedocumentstructuredinasensibleway?Are
thedescriptionsofrequirementsorganisedsothatrelated
requirementsaregrouped?
Conformancetostandardsdoestherequirementsdocumentand
individualrequirementsconformtodefinedstandards?Are
departuresfromthestandards,justified?
Traceabilityarerequirementsunambiguouslyidentified,include
linkstorelatedrequirementsandtothereasonswhythese
requirementshavebeenincluded?


RequirementsV&VTechniques
RequirementsReviews

Advantages
effective(evenafterconsideringcost)
allowfindingsourcesoferrors(notonlysymptoms)
authorsaremoreattentivewhentheyknowtheirworkwillbeclosely
reviewed

encouragethemtoconformtostandards
familiarizelargegroups


RequirementsV&VTechniques
RequirementsReviews

Risks
Reviewscanbedullanddraining(needtobelimitedintime)
Timeconsumingandexpensive(butusuallycheaperthanthealternative)
Personalityproblems
Officepolitics