Software Testing:

Verification and Validation

• Verification
“Are we building the product right?”

• Validation
“Are we building the right product?”

Barry Boehm, 1979

Ch.19 - Verification & Validation Tuesday, December 7, 2021 1

Verification and Validation
Techniques

• Static Techniques
– Software Inspections
(against source code)

• Dynamic Techniques
– Software Testing
(requires executable program)

Ch.19 - Verification & Validation Tuesday, December 7, 2021 2

Verification and Validation
Static techniques

• Software Inspections
– of requirements documents
– of design documents (design reviews)
– of source code (code reviews)
– automated static analysis

Ch.19 - Verification & Validation Tuesday, December 7, 2021 3

Verification and Validation
Dynamic techniques

• Software Testing
– specification vs. implementation
• Defect testing [Ch.20]
– verifying non-functional requirements (e.g.
performance; reliability)
• Statistical testing [Ch.21]
– automated dynamic analysis (if applicable)

Ch.19 - Verification & Validation Tuesday, December 7, 2021 4

Verification and Validation
Goals

• Establish that software is fit for purpose,

not “bug-free”
• “Good enough” depends on:
– Software function (critical nature?)
– User expectations
– Market
• competition, price

Ch.19 - Verification & Validation Tuesday, December 7, 2021 5

Testing vs. Debugging

• Verification and Validation

– looking and categorizing existence of system
defects [example] [bug list]
– “What?”

• Debugging
– locating and correcting these defects
– “Why?”

Ch.19 - Verification & Validation Tuesday, December 7, 2021 6

Regression Testing

• Canned test runs to verify that new defects

were not introduced during “debugging”
session.
• Not exhaustive
• Targeted to a particular interface
– components, sub-systems, integrated system
• Different levels (lengths) of regression tests
– Targeted regressions
Ch.19 - Verification & Validation Tuesday, December 7, 2021 7
The Test Plan
• Planning should begin right after
requirements specification
– Acceptance tests can be written then
• System, sub-system tests can be written
against designs

Ch.19 - Verification & Validation Tuesday, December 7, 2021 8

The Test Plan

Requir ements System System Detailed

specification specification design design

System Sub-system Module and

Acceptance
integration integration unit code
test plan
test plan test plan and tess

Acceptance System Sub-system

Service
test integration test integration test

Ch.19 - Verification & Validation Tuesday, December 7, 2021 9

Software Inspections (code reviews)

• >60% of program errors can be detected in

code review [Fagan86]
• >90% if more formal approach used (e.g.
“Cleanroom” process) [Mills87]
(We’ll talk about Cleanroom later)

Ch.19 - Verification & Validation Tuesday, December 7, 2021 10

Software Inspections (code reviews)
• Why are reviews more effective for finding defects
in systems/subsystems (i.e., before acceptance
testing)?
– Bugs are often masked by other bugs
– Leverage domain/programming knowledge
• inspectors are skilled programmers

• Common practice: code reviews and then

acceptance testing
– reviews can also help with development of tests

Ch.19 - Verification & Validation Tuesday, December 7, 2021 11

Software Inspections (code reviews)

• Sample procedure:
– Announce review meeting in advance (a week?)
– Provide design document, implementation
overview, and pointer to code
– Reviewers read code (and make notes) in
advance of meeting
– During meeting, directives recorded by Scribe
– Testers/documenters attend too
Ch.19 - Verification & Validation Tuesday, December 7, 2021 12
Automated Static Analysis

• CASE tools that catch program curiosities

that are usually indicative of bugs:
– unreachable code
– uninitialized variables
– unreferenced variables

• Programming language dependent

– e.g., LINT (for C)

Ch.19 - Verification & Validation Tuesday, December 7, 2021 13

Automated Dynamic Analysis

• Tools which do bookkeeping while program

is run/tested.
• Can find some dynamic problems that
compiler cannot catch (depends on
language…)
• C/C++ tools: Purify, BoundsChecker

Ch.19 - Verification & Validation Tuesday, December 7, 2021 14