Software Verification and Validation

Lecture No. 13
 Module 75:
Introduction to Software Verification
Introduction to Software Verification

 Software development
lifecycle models and
processes
 Waterfall model
 Spiral model
 Rapid Application
Development (RAD)
 Evolutionary prototyping
 Etc.
 Each stage can potentially
contain issues such as:
 Misconceptions
 Misunderstandings
 Errors…
 Introduction to Software Verification
 We need to spend time and
financial resources to fix these
errors or bugs in the following
manner (ref: textbook):
 Cost to fix a bug increases
exponentially (10x)
 i.e., it increases tenfold as
time increases
 E.g., a bug found during
specification costs $1 to fix.
 … if found in design cost is
$10
 … if found in code cost is
$100
 … if found in released
software cost is $1000
 Module 76:
Verification Methods
Verification Methods

 Defect prevention: Error

blocking and error source
removal.
 Defect removal:
 Inspection - this lecture.
 Testing, etc.
 Defect containment: Fault
tolerance and failure
containment (safety
assurance).
Verification Methods

 Verification is process of
assessing a software product
or system during a particular
phase
 to determine if it meets
requirements or conditions
 that were specified at the
beginning of that phase.
 It is a static process and it
includes substantiation of
artifacts such as
 SRS, SDS,
 design and
 Code / program.
Verification Methods

 We mainly divide verification

methods into
 Peer Reviews
 Walkthroughs
 Inspections
 We make use of checklists
which represent a set of time-
tested means of making sure
nothing is forgotten in any
process is a checklist
 A verification checklist will
help manage all of the details
that must be attended to in an
artifact.
Verification Methods

 From informal to formal

Informal

Walkthrough

Peer Review

Inspection
 Verification Methods

Method Family Typical Goals Typical Attributes

Minimal overhead Little/no preparation
Developer training Informal process
Walkthroughs
No measurement
Quick turnaround
Not Formal Tech. Review
Requirements elicitation Formal process
Technical Reviews Ambiguity resolution Author presentation
Training Wide range of discussion

Detect and remove all Formal process

Inspections defects efficiently and Checklists, Measurements
effectively.
Verify phase
Verification Methods

 Applications
 SRS verification
 Design verification
 Code verification
…
 Each step in adopted
application development
lifecycle requires verification
before an artifact becomes
a milestone and goes to the
next stage.
Module 77:
Walkthrough
Walkthrough

 Author of the artifact leads the

review process and the team
members ask questions and spot
possible errors against
development standards and
other issues.
 The meeting is usually led by
author of the document under
review and attended by other
members of the team.
 Review sessions may be formal
or informal.
 Before walkthrough meeting,
preparation by reviewers and
then, optionally, a formal review
report containing list of findings.
Walkthrough

 Defects/issues are noted down

so that they can be fixed and
they may be tracked to
closure.
 Main purpose of walkthrough
is to enable learning about the
content of the document under
review to help team members
gain an understanding of the
content of the document and
also to find defects
 Quality of results depend upon
personal brilliance of the
reviewer
Module 78:
Peer Review
Peer Review

 Peer Reviews are

documented and use a defect
detection process that has
peers and technical specialist
as part of review process.
 Review process doesn't
involve management
participation.
 It is usually led by trained
moderator who is NOT the
author.
 The report is prepared with
the list of issues that needs
to be addressed.
Peer Review

 Peer review requires

authors to present their
work,
 Involves a team of 2 to 7
professionals
 Report optional to author
and it presents issues in the
shape a few faults.
 Question arises: what are
types of issues.
 Issues / faults present
those situations that may
impact desired results
Peer Review

 Example–issue Classification
 Severe
 Defects that may cause
incorrect results or behavior
 Moderate
 Defects that may affect
limited areas of functionality
that can either be worked
around or ignored.
 Minor
 Defects that can be
overlooked with no loss of
functionality.
 Module 79:
Inspections for Verification
Inspection for Verification

 M. E. Fagan of IBM defined

inspections in 1976

 Inspection is led by a
trained moderator, who is
not the author. Inspection is
most formal and driven by
checklists and rules.

 This review process makes

use of entry and exit
criteria.
Inspection for Verification
 Inspections requires formal

 presenter in the shape of

moderator(NOT author),

 prior preparation of complete team,

 involvement of a team of 3 to 6
members,

 formal report by moderator,

 requires skilled staff and is

expansive.
Inspection for Verification

 Inspections can find about

60% of all defects.
 Inspections are technical, not
management.
 Inspections results can
assess/improve quality of:
 work product
 software development
process
 review process
Inspection for Verification

 Inspections reduce total

project cost, but have their
own execution costs
 Upstream defect removal is
10-100 times cheaper.
 Reviews disseminate domain
knowledge, development
skills, and corporate culture
 Module 80:
Fagan Inspections for Verification
Fagan Inspections for Verification

 We share steps involved in

Fagan Inspections:
1. Planning
2. Overview (1-to-n
meeting)
3. Preparation (individual
inspection)
4. Inspection (n-to-n
meeting)
5. Rework
6. Follow-up
Fagan Inspections for Verification

1. Planning
 Entry criteria: what to
inspect
 Team size: about 4
persons
 Developers/testers from
similar projects
 Inspectors are not authors
2. Overview
 Author-inspectors meeting
 General background
information
 functional/structural/inf
o., intentions
Fagan Inspections for Verification

 Assign individual tasks:

 coverage of important
areas
 moderate overlap
3. Preparation or individual
inspection
 Independent
analysis/examination
 Code as well as other
document
 Individual results:
 questions/guesses
 potential defects
Fagan Inspections for Verification

4. Inspection (generic:
collection)
 Meeting to collect /
consolidate individual
inspection results
 Team leader/meeting
moderator (1)
 Reader/presenter:
summarize/paraphrase for
individual pieces
 Defect identification, but
not solutions
 No more than 2 hours
 Inspection report
Fagan Inspections for Verification

5. Rework
 Author's response
 Defect fixing (solutions)
6. Follow-up
 Resolution verification
by moderator
 Re-inspection?
 Fagan inspection in
practice
 Widely used in industry
 Evaluation studies
 Variations and other
inspections
 Module 81:
Examples of Checklists for SRS document Inspections
Examples of Checklists for SRS Inspections

 Importance of checklists
 Represent industrial best
practices as templates
 Provide a means of use of
preexisting knowledge base
and lessons learnt
 Help conduct inspections /
audits in an efficient and
appropriate manner
 Artifact specific checklists
 We discuss example checklists
for
 SRS, Code…
 We have SRS and checklist as
input to inspection
Examples of Checklists for SRS Inspections

 General (SRS
requirements) Checklist
 functional overview of system
provided?
 software and hardware
environments been specified?
 assumptions that affect
implementation stated?
 Has every definition, acronym
and abbreviation been defined
in glossary, if they are correct?
 Are requirements, interfaces,
constraints, definitions, etc.
listed in appropriate sections?
Examples of Checklists for SRS Inspections

 Functional Requirements
Checklist
 Have all requirements
communicated by customer
been specified?
 Are all inputs to a function
sufficient to perform the
required function?
 Are undesired events /
inputs considered and their
required responses
specified?
 Are all use case
descriptions provided?
 Are all misuse cases
documented?
Examples of Checklists for SRS Inspections

 Interface Checklist
…
 Non-functional Requirements
Checklist
…
 Requirements Quality
…
…
Examples of Checklists for SRS Inspections

 By and large, checklist for

requirements verification
must check if the document
is:
 Complete, consistent,
correct
 Precise, unambiguous,
clear
 Relevant, testable,
understandable
 Traceable, feasible,
prioritized
 Free from design details
 Module 82:
Examples of Checklists for Code Inspections
Examples of Checklists for Code Inspections

 Code checklists are language

specific
 Example code inspection
checklist:
 Code formatting:
 Are proper alignments
followed (left margin) 
 Are code block starting
point/ending point
easily identifiable.
 Are naming conventions
(Pascal, CamelCase,
Hungarian notation
followed. 
Examples of Checklists for Code Inspections
 Coding best practices:
 Are all similar values
grouped under enumerated
data types
 Are comments properly
given
 Are comments provided for
what is done rather why is
done
 Are there nested if/else
statement in the code?
 Is there a separation of
concerns through
implementation of any
framework
…
Examples of Checklists for Code Inspections

 Structure
…
 Variables
…
 Comments
…
 Loop and branches
…
 General
…
 Module 83:
Examples of Checklists for User Documentation Inspections
Checklists for User Documentation Inspections

 Example of documentation
review checklist:
 Does title page include
required information
 The purpose of document is
clear and complete.
 All known audiences /
customers / users are
described thoroughly and
accurately.
 Scope of document is
accurate and complete.
 Product version numbers /
release dates are accurate.
 Table of contents clear
Checklists for User Documentation Inspections

 All steps in the procedure are

accurate and complete.
 All screen shots are accurate
and complete.
 All supporting texts are
accurate and complete.
 All corresponding screen shots
accurately display current
version of software and clearly
relate to the step text.
 All charts, graphs, and
diagrams are labeled
accurately and consistently.
…
 Module 84:
Code Review Example
Code Review Example

 We need
 Moderator
 Team members
 Authors
 Checklists
 Reporting forms
 Artifact for
inspection/review (Code)
 Let author presents code and
moderator conducts
 Planning, Overview,
Preparation and
inspection with two
team members
 Code Review Example
 Checklist
 Follows coding conventions
 Names are simple and if possible
short and are spelt correctly
 Names contain units where
applicable
 Enums are used instead of int
constants where applicable
 All class, variable, and method
modifiers are correct.
 There is no commented out code
 There is no dead code
 Debugging code is absent
 No stack traces are printed
 Variables not used with null values
 Code is not repeated or duplicated
 There is an else block for every if
clause even if it is empty
 No complex, negatively named or
long Boolean expressions
 No empty blocks of code
 Constructors don’t accept null values
 Arrays checked for out of bound
 Catch clauses are appropriately used
 Exceptions not ignored, if caught.
 Files/Sockets/Cursors and other
resources are properly closed.
Code Review Example
Code Review Example

 Issues Classification
 Severe
 Defects that may cause
incorrect results or behavior
 Moderate
 Defects that may affect
limited areas of functionality
that can either be worked
around or ignored.
 Minor
 Defects that can be
overlooked with no loss of
functionality.
 Code Review Example
 Example Issue Log Form
Code Review Example

 Issues found
 Severe
 Try-catch block not
implemented
 Moderate
 Constructor not present
 Missing class, function and
variable comments.
 Minor
 No naming convention
followed e.g., Hungarian
notation is not followed for
variable names, function
names etc.
Code Review Example

 Next steps:
 Review meeting
 Consolidated and
comprehensive listing of
issues prepared.
 Shared with key
stakeholders
 Authors are given time to
rework
 Moderator (or representative)
verifies if the corrections are
made.