Verification and

Validation
1 ST SEMESTER 2017
Outline
1. Verification and validation in the software development process
2. Debugging
3. Static analysis techniques
4. Dynamic analysis techniques
5. Verification and validation standards

2017 AGUSTN YAGE CARMEN GIL 2

Outline
1. Verification and validation in the software development process
1. Introduction
2. Verification and validation
3. The verification and validation process
4. Verification and validation ecosystem

2017 AGUSTN YAGE CARMEN GIL 3

 1. Introduction
Basic definitions
Error (synonym of mistake)
It is a human action while performing some activities while the engineering process
Fault (synonym of defect)
It is the result of an error, that means a fault is the representation of an error
Two types:
Commission: that is when an engineer enter something in a software that is incorrect (bad)
Omission: that is when an engineer makes an error and something is missing. (the worst)
Failure
It only happens when a fault executes an executable representation. Therefore it is related to comission faults
Incident
When a failure occurs. An incident is a symptom associated to a failure
Test
It is the act of exercising software
Two goals:
Find failures
Demonstrate correct executions
Test case
It represents a program behavior. It has a set of inputs and their corresponding outputs

2017 AGUSTN YAGE CARMEN GIL 4

 1. Introduction
Testing Venn Diagrams

Specification expected vs Program observed Program Program

behaviors (observed)
Three areas:
S: specification written (omission)
P: behaviors implemented (commission)

S P: behavior specified and implemented S P

Specification
(expected)

2017 AGUSTN YAGE CARMEN GIL 5

 1. Introduction
Venn Diagrams
Specification Program 1. Behavior specified, implemented and tested
(expected) (observed)
S P 2. Behavior specified, implemented but not tested
2 3. Behavior not specified, implemented and tested
5 6 4. Behavior specified, not implemented and tested
1 5. Behavior specified but not implemented nor tested
4 3 6. Behavior not specified, implemented and not tested
7. Behavior not specified, not implemented and tested
7

T Test cases
(verified)

2017 AGUSTN YAGE CARMEN GIL 6

1. Introduction
Types of testing
Specification-based Inputs Outputs
Functional testing
Black box

Code-based
Structural testing
White box testing
Glass box testing
Clear box testing An engineers Black Box: a
system understood in terms
of its inputs and outputs

2017 AGUSTN YAGE CARMEN GIL 7

 1. Introduction
Which is better Specification-based or Code-based? Program behaviors

Both are needed

If all specified behavior have not been
implemented structural testing will never be able S P
to recognize them
If the program reveals behaviors that have not Specification-based Code-based
been specified, this will never be revealed by (establishes (Seek faults)
functional test cases confidence)

2017 AGUSTN YAGE CARMEN GIL 8

1. Introduction
Levels of testing

Requirements System
Specification Testing

Preliminary Integration
Design Testing

Detailed Unit
Design Testing

Coding

2017 AGUSTN YAGE CARMEN GIL 9

1. Introduction
But we also need some mathematical concepts:
Sets
Set theory (see these concepts in the external presentation)
Equivalence classes (see these concepts in the external presentation)
Graphs (see these concepts in the external presentation)
Basic definitions
Undirected graphs
Directed graphs
Paths
Components (weak and strong)

2017 AGUSTN YAGE CARMEN GIL 10

2. Verification and validation
Some previous questions
How could we know if the product that has been built is the product that the customer really needs?
How is possible to know if the developed product will work properly?

Some facts
We have reduced resources
Mathematics demonstrations are desirable to show correctness

Conclusions
Empirical verification is required

2017 AGUSTN YAGE CARMEN GIL 11

2. Verification and validation
Some figures
An expert developer produces 1 defect each 10 lines of code
If we were able to detect 99% of these defects (too optimistic)
That means 1 defect each 1000 lines (1 KLOC)
Normally a software system has more than 150 KLOC
By 2020 it is expected than cars will have around 100 MLOC in embedded software

2017 AGUSTN YAGE CARMEN GIL 12

2. Verification and validation
In addition
Cost of late detection increases along the development life cycle

2017 AGUSTN YAGE CARMEN GIL 13

2. Verification and validation
To remove all the existing defects is impossible, so that
The minimum amount of achievable defects is 0,5 1 every KLOC
In commercial software it is expected to find 3 6 defects/KLOC
It could be considered as high quality software when the defects rate is less than 15 defects/KLOC

2017 AGUSTN YAGE CARMEN GIL 14

2. Verification and validation
Software quality MUST be considered at every stage in the development life cycle
It is more o less easy to analyze the source code quality, but
It is not easy to analyze the quality of:
Specification
Analysis
Design
Usability
Performance
Maintainability
Portability

2017 AGUSTN YAGE CARMEN GIL 15

2. Verification and validation
Verification (IEEE 610 Software engineering terminology)
(1) The process of evaluating a system or component to determine whether the products of a given development phase satisfy the
conditions imposed at the start of that phase. Contrast with: validation.
(2) Formal proof of program correctness. See: proof of correctness.

Have we build the product right?

Validation
The process of evaluating a system or component during or at the end of the development process to determine whether it
satisfies specified requirements. Contrast with: verification.

Have we build the right product?

2017 AGUSTN YAGE CARMEN GIL 16

3. The verification and validation process
12207 (software)

15288
(systems)

2017 AGUSTN YAGE CARMEN GIL 17

3. The verification and validation process
Verification process
Purpose
The purpose of the Software Verification Process is to confirm that each software work product and/or
service of a process or project properly reflects the specified requirements.

Outcomes
a) a verification strategy is developed and implemented;
b) criteria for verification of all required software work products is identified;
c) required verification activities are performed;
d) defects are identified and recorded; and
e) results of the verification activities are made available to the customer and other involved parties.

2017 AGUSTN YAGE CARMEN GIL 18

3. The verification and validation process
Verification process
Activities and tasks:
a) Process implementation
b) Verification
a) Requirements verification
b) Design verification
c) Code verification
d) Integration verification
e) Documentation verification

2017 AGUSTN YAGE CARMEN GIL 19

3. The verification and validation process
Validation process
Purpose
The purpose of the Software Validation Process is to confirm that the requirements for a specific
intended use of the software work product are fulfilled.
Outcomes
a) a validation strategy is developed and implemented;
b) criteria for validation of all required work products are identified;
c) required validation activities are performed;
d) problems are identified and recorded;
e) evidence is provided that the software work products as developed are suitable for their intended
use; and
f) results of the validation activities are made available to the customer and other involved parties.

2017 AGUSTN YAGE CARMEN GIL 20

3. The verification and validation process
Validation process
Activities and tasks:
Process implementation
Validation:
Prepare test requirements
Ensure that tests requirements reflects the specific facts
Conduct tests
Validate that the software satisfies the intended use
Test the software in the target environment

2017 AGUSTN YAGE CARMEN GIL 21

4. Verification and validation ecosystem
Watch Code Code Writing

Publish Results Tests Writing

Tests Execution Build

2017 AGUSTN YAGE CARMEN GIL 22

4. Verification and validation ecosystem
Eclipse
Eclipse is an open source community,
Eclipse is one of the most popular IDEs
Open development platform
Extensible with plugins

Maven
Maven is an open source tool
Designed to facilitate building and managing coding and deploying activities
Makes the build process easy
Provides a standard build lifecycle
Provides a set of plugins with external tools

2017 AGUSTN YAGE CARMEN GIL 23

4. Verification and validation ecosystem
Hudson/Jenkins
Is a web-based continuous integration server
It allows the automation of continuous integration practices.
"Continuous Integration (CI) is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily -
leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many
teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. (Martin Fowler)

Updated when changes in version control systems are detected

Triggering new builds
Testing artifacts
Sending notifications
Deploying to production servers

2017 AGUSTN YAGE CARMEN GIL 24

4. Verification and validation ecosystem
SonarQuBe
SonarQuBe is an open source platform
Facilitate quality management of software code
Provides mainly features to facilitate:
Code analysis,
Results reporting,
Hunt defects and
Time-line project evolution
Some measures are:
Duplicate code stats
Coding standards and potential bugs identification
Testing coverage
Code complexity
Design and architecture structural parameters

2017 AGUSTN YAGE CARMEN GIL 25