Introduction to Formal Methods Dr-Ing.

Abderahman KRIOUILE

Scientists Quotes
“Software engineers want to be real engineers.
Real engineers use mathematics.
Formal methods are the mathematics
of software engineering.
Therefore, software engineers should use
formal methods.”

(Mike Holloway, NASA)

linkedin.com/in/akriouile @A_KRIOUILE Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 2

Contents Today
• Need for Formal Methods • Overview of Critical-Systems
• Formal Methods Definition • Need for Formal Methods
• Formal Modeling • Formal Methods Definition
• Temporal logics • Examples of
• Model Checking Formal Techniques
• CADP toolbox

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 3 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 4

Motivations Desired properties of systems

• Problems in Developing Industrial Systems • Availability: a system must be working and
– Functionality Issues: Growing Size & Complexity able to provide its services
– Requirements issues: availability, reliability, safety, • Reliability: a system must correctly provide its
security
functionalities, as expected by users
– Application Domain Issues: Safety-Critical, Mission-
Critical or Business-Critical Systems • Safety: the system should do nothing very
– Market Issues: Time-to-delivery, Costs undesirable (causing damages to people,...)
– Maintenance Issues: Requirements change over time • Security: the system should resist to intruders

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 5 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 6

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 1

Introduction to Formal Methods Dr-Ing. Abderahman KRIOUILE

Critical Systems Safety-Critical Systems

• Safety-critical: trains & planes control, nuclear

plants control, etc.
• Mission-critical: space craft navigation
• Business-critical: bank management software,
operating systems, Smartphone, etc.

Systems whose failure can cause life losses

or serious environmental damage
Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 7 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 8

Business-Critical Systems Mission-Critical Systems

Systems whose failure can cause the loss of Systems whose failure can cause the failure
big or huge amounts of money of the goals of important missions
Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 9 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 10

Avionic software standard

Developing Safety-Critical Systems
Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016
DO-178C: Design Assurance Level(DAL)
• DAL A / Catastrophic - Failure may cause multiple fatalities,
usually with loss of the airplane.
• DAL B / Hazardous - Failure has a large negative impact on
safety or performance, or reduces the ability of the crew to
operate the aircraft.
• DAL C / Major - Failure significantly reduces the safety margin
or significantly increases crew workload. May result in
passenger discomfort (or even minor injuries).
• DAL D / Minor - Failure slightly reduces the safety margin or
slightly increases crew workload.
• DAL E / No Effect - Failure has no impact on safety, aircraft
operation, or crew workload.
11 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 12

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 2

Introduction to Formal Methods Dr-Ing. Abderahman KRIOUILE

Avionic software standard

The Ariane 5 Case
DO-178C: Design Assurance Level(DAL)
• 4 June 1996: the first flight of the Ariane 5
failed. After 40 seconds the rocket changed
Level Failure condition Objectives With independence
trajectory and exploded
A Catastrophic 71 33
B Hazardous 69 21 • The SW of the Inertial Reference Systems
C Major 62 8
ceased to work after 36 seconds.
D Minor 26 5
E No Effect 0 0 • 800 Million US$ lost
• Cause:
a variable overflow!

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 13 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 14

Formal Methods Definition Formal Model: a utility by itself!

Formal Notation • Classical Development: the product is the unique
Formal Methods formal statement (code for software)
Formal Analysis – At the end of the development process
• Formal Model
• Formal Notation – Ambiguities detection
– Semantics defined mathematically – Earlier error detection
Unambiguous – Upstream phases consolidation
• Formal Analysis – Mature product earlier
– Automated processing of models expressed in – More expensive upstream phases
formal notation • But overall more effective development

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 15 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 16

Example: NASA projects budget

Modeling For Simulation
respect
• Correlation between effort spent in upstream phases • Prototyping, design choices exploration
/ budget compliance • Validation by model-based testing

SCADE Example

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 17 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 18

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 3

Introduction to Formal Methods Dr-Ing. Abderahman KRIOUILE

Code Generation Test Generation

• Automatic code generation from models • Automatic test generation from models
– Avoid manual coding error – Compliance code / model
– Productivity gain – Model Coverage
– Earlier verification on the model – Automated Verdict
Requirement Integration Tests Requirement Integration Tests

Design Unit Tests Design Unit Tests

Code Code

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 19 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 20

Verification Formal Verification

• Classic = dynamic • Syntax Analysis
– Simulation /testing
• Verification of generic properties
– None exhaustive
– Test run the hardware – Absence of run-time error
– WCET Computing
• Formal = static – Dedicated Analysis
– No run • Verification of specific properties
– Automated
– Defined by the user
– Exhaustive / consider properties
– Expressed formally

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 21 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 22

Formal Verification Techniques (1) Static Analysis

1. Static Analysis (SW) – Static Program Analysis
– Verify predefined safety properties “something
bad cannot happen” by over-approximation
2. Model Checking (SW & HW)
– If the property holds for the over-approximation it
holds for all behaviors
3. Theorem Proving (SW & HW) All violations detected

Infeasible behaviors
All behaviors of
the program
False negative

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 23 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 24

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 4

Introduction to Formal Methods Dr-Ing. Abderahman KRIOUILE

Static Analysis Properties Examples (2) Model Checking

• Absence of run-time errors • Formal V&V by exhaustive search over the
– Division never by 0
– Sqrt never of a negative number state space
– Arithmetic operation never causes overflow/underflow • Systems modeled as Finite State Machine M
– Array-index is never out of bounds
– Dereference never applied to null-pointer • Properties expressed with a formal
• Absence of timing accidents for WCET analysis representation φ (e.g. temporal logic)
– Memory access is a cache hit
– Bus access is not blocked Precise, unambiguous semantics
• Derived global program properties • Verification via logical reasoning: M |= φ
– Stacks do not overflow
– Program terminates within a given deadline
Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 25 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 26

Temporal Logic A Temporal Logic can express

• Formalism for describing evolutions of • “an event A is always possible”
program states over (logical) time • “an event A will eventually happen”
– Atomic propositions over states
– Propositional logic operators (or, and, not …) • “an event A is always possible until an event B
– Tense operators (neXt, Until, Previous, Since, happens”
Once, …)
– Interpreted on state spaces Numerous temporal logics were proposed in the
• High-level specification style: literature.
abstraction and modularity

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 27 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 28

Model Checking Industrial Success of Model Checking

• Powerful debugging capabilities
– detecting problems in early stages of development
– exhaustive, thus effective
– provides counterexamples
• Can be integrated within industrial
p
development cycle
q
• Does not require deep training (Vs Theorem
proving)

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 29 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 30

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 5

Introduction to Formal Methods Dr-Ing. Abderahman KRIOUILE

(3) Theorem Proving Theorem Proving Example

• Consider the following argument (sequence of
• Theorem prover is based on libraries of propositions):
axioms and hypotheses. – If the prof offers chocolate for an answer, you answer the prof’s
question.
• Uses inference rules to prove the formal – The prof offers chocolate for an answer
– Therefore, you answer the prof’s question
description of the behavior of the system
• Let p be “the prof offers chocolate for an answer”
• Simplifying until reaching known axioms • And q be “you answer the prof’s question”
• The form of the above argument is :

• The argument is valid since ((p q)˄ p) q

is a tautology

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 31 Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 32

Synthesis
• Formal Methods are introduced to industry
• Certification
– Security: Formal Methods are mandatory for high
certification level
– For example: FM recommended in railway area
– In Aeronautic, DO-333 is a technical supplement
of the DO-178C introducing Formal Methods.

Dr. Abderahman KRIOUILE - Introduction to Formal Methods - ENSIAS 2015/2016 33

ENSIAS Systèmes Embarqués et Mobiles 2015/2016 6