2 - AAS Concepts and Terminology Teaser

Analysis of Software Artifacts
Departamento de Engenharia Informática, FCTUC
Analysis of Software Artifacts (ASA)

Henrique Madeira,
Departamento de Engenharia Informática
Faculdade de Ciências e Tecnologia da Universidade de Coimbra
2022/2023
Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 1
Fundamental concepts
of software quality
and software dependability
(this is just a teaser)
Henrique Madeira, 2022/2023 1

Concepts and terminology on dependability and

quality in action…
Dependability: an integrative concept
• Dependability: ”delivery of service that can justifiably be trusted, thus

avoidance of failures that are unacceptably frequent or severe” (J.-C. Laprie)
Security
Attributes


Key non-functional attributes

of software (and systems)

The problems that may

damage dependability


Different means/techniques to solve

or mitigate the effect of the threats

Causality chain

Threats: causality view
System point of view
Fault may cause Error may cause Failure
• Failure: Incorrect component and/or system response

• Error: Erroneous change in the state of the system
• Fault: Root cause
Threats: causality view
Software point of view
Error may cause Fault may cause Failure

(human) (defect/bug)
• Failure: External behaviour is incorrect

• Fault: Discrepancy in code that causes a failure.
• Error: Human mistake that caused fault
10

Two views of software systems

• Functional view
– What the software system does
– Quality is related to the match between the functionalities and the user
needs/expectations
• Non-Functional view
– How the software system does it (features such as performance, security,
reliability, availability, usability, maintainability, and many, many, more)
– Typically known as Quality Attributes of a software system
– Most of them cannot be measured directly
– The biggest technical challenges are in these non-functional attributes
11
Functional and non-functional requirements
In software engineering the functional vs non-functional views starts with

the requirements elicitation (i.e., at the very beginning of the process)
• Functional requirements
– Describes what a software system should do
– Function points is a usual metric to characterize and assess the size of the
software
• Non-functional requirements
– Define constraints (or goals) on how the system will do so
– Include basically everything that is not related to the functional aspects of the
software system
12

Robustness
(more on concepts & terminology)
• Robustness: “a software system can be said to be robust if it retains its

ability to deliver service in conditions which are beyond its normal domain
of operation” (Laprie)
• Robustness is used very often to test software interfaces such as system

calls, APIs, web services, etc. This is called robustness testing:
– In this context, robustness is defined as “the degree to which a system or
component can function correctly in the presence of invalid inputs
[IEEE90]”
– Experimental studies (Phil Koopman) show that approximately 15% of the OS
system calls (Linux, Unix, Windows) crashes when called with invalid input
parameters.
13
Resilience
(more on concepts & terminology)
• Resilience ≈ dependability + robustness
Resilience: the persistence of service delivery that can justifiably be

trusted, when facing changes (Laprie)
• Resilience considers changes in lato senso. That is, changes include all sort
of upsets:
– Hardware and software faults
– Malicious attacks
– Configuration changes
– Software and hardware upgrades
– Etc…
14

Dependability (and Resilience)

Attributes, Means, and Threats
• Hardware faults
• Software faults
• Environment faults
• Human faults
• …
15
What is a software fault?

Residual(?) software faults (bugs), originated from defects in design or
implementation of software components and its integration in a system, that escape
testing and other fault avoidance methods
Software development process (in theory...)

Requirements
Specification
Design
Code development
Test
Deployment
Correctness from the

end user point of view
16

What is a software fault?
17
Different types of software faults

• In complex systems, the failures caused by software bugs may appear in
different way, defining a very first big types of software faults (bugs):
• Bohrbugs
• Bugs that cause failures deterministically
• Easiest to find during testing
• Fault tolerance à design diversity and redundancy
• Mandelbugs
• Re-execution after a failure caused by a Mandelbug will generally not cause another
failure
• Very difficult to find and correct during testing
• Fault tolerance à simple retries and recovery-oriented computing using checkpointing
• Aging-related
• Bugs tend to be activated and cause failures after long periods of system run-time
• Difficult to find during testing (but static code analysis is effective for some of them)
• Fault tolerance à software rejuvenation
18

Software faults: a persistent problem
• Software reliability is mainly based on fault avoidance using good

software engineering methodologies
• In real systems (i.e., not toys) à fault avoidance not successful à

Fault-tolerance is needed, unless the impact of failures is
acceptable.
• Rule of thumb for fault density in software (Rome labs, USA)

– 10-50 faults per 1,000 lines of code à for good software
– 1-5 faults per 1,000 lines of code à for critical applications using highly
mature software development methods and having intensive testing
19


acceptable.
• Rule of thumb for fault density in software (Rome labs, USA)

20


• SW development methodologies
• Static analysis tools
• Software inspections
acceptable.
• Model checking
• Rule of thumb for• fault density
Testing, in software
testing, testing (Rome labs, USA)
• Verification and validation
• …
21
Classification of faults
• Caused by what?
– Physical faults
– Human-Made faults
• Why?
– Accidental faults
– Intentional non malicious faults / Intentional malicious faults
• When?
– Development faults: design, coding, configuration, upgrading
– Operational faults: in use or maintenance (operation faults, interaction faults,
configuration faults,..)
• Where (with respect to the system)?
– Internal faults
– External faults
• How long?
– Permanent faults
– Transient faults
22

Classification of faults (more detailed view)
23

Components/systems may fail

arbitrarily
Failures such as clean crashes
(i.e., stop sending outputs)
are relatively rare
24

Failures classification
25

26

Dependability means
• Fault Prevention techniques: prevent the occurrence ofTwo
faults
different
– Improve development process to avoid/minimize faults
– Use selected technologies (better components, certified softwareperspectives
tools, etc. ) with
strong technical
• Fault Tolerance techniques: to provide correct serviceimplications
in presence of faults
– Triple modular redundancy, N-Version programming, check pointing and recovery, etc.
27
Dependability means
• Fault Prevention techniques: prevent the occurrence of faults
– Improve development process to avoid/minimize faults
– Use selected technologies (better components, certified software tools, etc. )
• Fault Tolerance techniques: to provide correct service in presence of faults

– Triple modular redundancy, N-Version programming, check pointing and recovery, etc.
• Fault Removal techniques: specific techniques to reduce the presence of

faults (number, seriousness, ...)
– Development: regression and non-regression testing, static and dynamic verification, etc.
– Operation: preventive maintenance such as patches, updates, SW rejuvenation, etc.
• Fault Forecasting techniques: to estimate the present number, the future

incidence, and the consequences of faults
– Probabilistic assessment, modeling, operational evaluation,…
28

Dependability means diagram (Laprie)
Error masking
29
Dependability means diagram (Laprie)
V model
Error masking
30

Fault tolerant techniques diagram
31
Fault tolerance techniques
Fault Error Failure
Estimated using fault

forecasting techniques
Fault tolerant mechanisms
32

Fundamental concepts
of software quality
and software dependability
For a more complete view:

• Detailed slides provided
• Recommended papers
• Other (reliable) sources available in the Internet
1
33

2 - AAS Concepts and Terminology Teaser

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2 - AAS Concepts and Terminology Teaser

Uploaded by

Copyright:

Available Formats

Analysis of Software Artifacts

Departamento de Engenharia Informática, FCTUC

Analysis of Software Artifacts (ASA)

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 1

(this is just a teaser)

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 2

Henrique Madeira, 2022/2023 1

Concepts and terminology on dependability and

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 3

Dependability: an integrative concept

• Dependability: ”delivery of service that can justifiably be trusted, thus

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 4

Henrique Madeira, 2022/2023 2

Dependability: an integrative concept

• Dependability: ”delivery of service that can justifiably be trusted, thus

Key non-functional attributes

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 5

Dependability: an integrative concept

• Dependability: ”delivery of service that can justifiably be trusted, thus

The problems that may

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 6

Henrique Madeira, 2022/2023 3

Dependability: an integrative concept

• Dependability: ”delivery of service that can justifiably be trusted, thus

Different means/techniques to solve

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 7

Dependability: an integrative concept

• Dependability: ”delivery of service that can justifiably be trusted, thus

Henrique Madeira, 2022/2023 4

Threats: causality view

System point of view

Fault may cause Error may cause Failure

• Failure: Incorrect component and/or system response

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 9

Threats: causality view

Software point of view

Error may cause Fault may cause Failure

• Failure: External behaviour is incorrect

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 10

Henrique Madeira, 2022/2023 5

Two views of software systems

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 11

Functional and non-functional requirements

In software engineering the functional vs non-functional views starts with

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 12

Henrique Madeira, 2022/2023 6

• Robustness: “a software system can be said to be robust if it retains its

• Robustness is used very often to test software interfaces such as system

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 13

• Resilience ≈ dependability + robustness

Resilience: the persistence of service delivery that can justifiably be

Henrique Madeira, 2022/2023 7

Dependability (and Resilience)

What is a software fault?

Software development process (in theory...)

Correctness from the

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 16

Henrique Madeira, 2022/2023 8

What is a software fault?

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 17

Different types of software faults

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 18