3 - AAS Complexity Metrics Teaser

Analysis of Software Artifacts
Departamento de Engenharia Informática, FCTUC
Analysis of Software Artifacts (ASA)

Henrique Madeira,
Departamento de Engenharia Informática
Faculdade de Ciências e Tecnologia da Universidade de Coimbra
2022/2023
Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 1
Software complexity metrics
(this is just a teaser)
“If you cannot measure it, you cannot improve it”, Lord kelvin
Henrique Madeira, 2022/2023 1

Size matters
LoC: this is an example of
code complexity metric
100 million lines of code
How many software bugs?
From Rich Rogers, https://twitter.com/richrogersiot/status/958112741218111489
Size matters
LoC: this is an example of
code complexity metric
100 million lines of code
Rule of thumb for fault density in software

• 10-15 faults per 1,000 lines of code à for good software
• 1-5 faults per 1,000 lines of code à for critical
How manyapplications using highly
software bugs?
mature software development methods and having intensive testing
Software faults (human errors): a persistent problem

“This year’s (2022) report cites an increase in the cost of poor software
quality (CPSQ) in the U.S. to at least $2.41 trillion—up from $1.31
trillion two years ago.”
From Rich Rogers, https://twitter.com/richrogersiot/status/958112741218111489
https://www.synopsys.com/software-integrity/resources/analyst-reports/cost-poor-quality-
software.html?intcmp=sig-blog-cisq22

Many types of software complexity metrics
You will find many diagrams like this…
Adapted from: https://www.javatpoint.com/software-engineering-software-metrics

Software complexity metrics are used for many things…
• Complexity is the key element of software quality

• Code complexity metrics are used to:
• guide the definition of test cases to achieve specific goals test coverage
goals;
• predict software defects probability and estimate defect density;
• estimate reusability risk of software components;
• control quality in continuous integration/continuous deployment;
• assess/predict how programmers comprehend code;
• determine the adequate component granularity in software architectures
based on complexity thresholds (code refactoring).
• …

Code complexity metrics
• Lines of code LoC
• McCabe complexity metrics

– Cyclomatic complexity V(g): number of linearly independent paths of a code unit
• Halstead metrics
– Length: number of operators and operands
– Vocabulary: number of unique operators and unique operands
– Difficulty = (unique operators / 2) * (operands / unique operands)
– Volume: how much information the reader has to absorb to understand the code.
– Effort: amount of effort to rewrite a codebase (excluded all the work related, like understanding
specifications)
– Halstead Time: how much time is needed to rewrite a codebase.
Halstead metrics have been criticized for many reasons. Today people tend to think that these
metrics are not well adapted to modern languages…
Code complexity metrics:

many programming paradigms
Procedural programming Aspect oriented programming Feature oriented programming
Object-Oriented
Number of Classes
Number of Methods
Cohesion
Coupling
…

Code understandability
”Programs must be written

for people to read, and only
incidentally for machines to
execute.” Abelson &
Sussman
People tend to think/accept that

software complexity metrics
Human perspective of reflect the human (developers)
complexity
perception of software complexity
Understandability
McCabe’s cyclomatic complexity: V(g)
• Measures
– The number of linearly independent paths through the control flow graph
(basic set)
• V(G) = E – N + 2
– E is the number of flow graph edges
– N is the number of nodes
Predicates:
Or • Nodes with multiple exit arcs.
• Corresponds to branch/conditional
• V(G) = P + 1 statement in program.
– P is the number of predicate nodes
Henrique Madeira,
Henrique IPN-LIS, February 1, 2023
M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 10 10
10

Cyclomatic complexity: example
min = A[0];
1 1
i = 1;
while (i < N) { 2
2 T
F
if (A[i] < min) 3
3
min = A[i]; 4
i = i + 1; 5 F T
} 5 4
print min; 6 6
Henrique Madeira,
Henrique IPN-LIS, February 1, 2023
M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 11 11
11
min = A[0];
1 1
i = 1;
while (i < N) { 2
2 T
F
if (A[i] < min) 3
3
min = A[i]; 4
i = i + 1; 5 F T
} 5 4
print min; 6 6
V(G) = E – N + 2 = 7 – 6 + 2 = 3
V(G) = P + 1 = 2 + 1 = 3
Predicates: node 2 and node 3
12

min = A[0];
1 1
i = 1;
while (i < N) { 2
2 T
F
if (A[i] < min) 3
3
min = A[i]; 4
i = i + 1; 5 F T
} 5 4
print min; 6 6
V(G) = 3 à means that the program has 3 independent paths

P1 = (1, 2, 6)
P2 = (1, 2, 3, 5, 2, 6)
P3 = (1, 2, 3, 4, 5, 2, 6)
13
Path base testing
• Test cases: 1
– P1 = (1, 2, 6) 2
F T
• Test Case: A = { 5 }, N = 1 3
Expected Output: 5
F T
– P2 = (1, 2, 3, 5, 2, 6) 5 4
• Test Case: A = { 5, 9 }, N = 2 6
Expected Output: 5
– P3 = (1, 2, 3, 4, 5, 2, 6)
• Test Case: A = { 8, 6 }, N = 2
Expected Output: 6
Define at least one test case for each linearly independent path
Simplistic use of V(g):

V(g) = 3 à Very simple code V/g) > 10 à Complex code
14

State-of-the-art code complexity metric:

Cognitive Complexity from SonarSource
From SonarSource Technical Report on “A new way of measuring understandability”, 2021.

https://www.sonarsource.com/docs/CognitiveComplexity.pdf
15


16



17
Is SonarSource Cognitive Complexity (CC-Sonar)

really measuring code understandability?
Experiment
27 programmers
8 proficient
19 ordinary
3 code comprehension
tasks
EEG, HRV, eye tracker Program
Expected
V(g) LoC
No.
Goal
NASA Task Load complexity units
Index (TLX) C1 Easy 5+1 15 2

Counts the number of elements in an array that fall
within a given interval.
Computes multiplication using classic weighted digit
C2 Medium 4+7+1 45 3
algorithm.
Searches 3 dimensional objects in a 3 dimensions
C3 Difficult 22 43 1
space.
18

NASA TLX results
There is a clear (and expected) saturation effect
19
Subjective perception, cognitive load and

performance
The Spearman correlation coefficient (r s) calculated for the scores obtained for the mental effort (NASA TLX) and cognitive load
(EEG) for all participants in the three programs is r s = 0.829, with a corresponding p < 0.0001, which indicates a high correlation
between the cognitive load measured using EEG and the subjective perception of mental effort declared by the
participants
20

How code complexity metrics compare with

code comprehension difficulty
Classic Halsted metrics Basic

code
structures
21

Cognitive load (EEG) can detect even small differences in the code complexity (that
have impact on the mental effort)
22


Variables seems to play an important role. Calls to library functions and external APIs is
relevant. The semantics of the algorithm is important.
23

CC-Sonar much lower than the value recommend as threshold for code refactoring (CC-
Sonar > 15) does not guarantee that programmers easily understand the code unit.
24

A glimpse of detailed results (and 2 highlights)
à CC Sonar > 15
• Use CC-Sonar as a staring point

• Add the following:
• Score saturation
• Data complexity There is room for improvement
• Parameters and variables (e.g., Halsted Difficulty metric) à new metric DC3
• Algorithm embedded in the data operations
• Context complexity, coupling and Cohesion
• Change the way depth of nesting is considered for the score
25
A glimpse of detailed results (and 2 highlights)
We can measure code understandability in real time for each programmer?

1.00
0.90
Cognitive load (EEG) vs No. Revisits
0.80 Nrevisits
0.70
CL(E EG)
0.60
0.50
0.40
0.30
0.20
0.10
0.00
B
B
C .E
B
A
C C
C
D
C C
C .1
C .2
C C .3
C .1
C .2
C .3
C .1
C .2
C .3
C .4
C .5
C .6
C .7
C .8
C C .9
D
C .10
C .11
.12
A
1.
2.
3.
C
C
A
A
A
C
C
C
C
C
C
C
C
1.
1.
2.
2.
2.
3.
3.
BC
2
C
C
C
C
C
1.
C
C
C
C
2.
2.
2.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
3.
2.
The Spearman correlation coefficient is rs = 0.96309, p < 0.00001
26

Software complexity metrics
For a more complete view:

• Recommended papers
• Other (reliable) sources available on the Internet
1
27

3 - AAS Complexity Metrics Teaser

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

3 - AAS Complexity Metrics Teaser

Uploaded by

Copyright:

Available Formats

Analysis of Software Artifacts

Departamento de Engenharia Informática, FCTUC

Analysis of Software Artifacts (ASA)

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 1

Software complexity metrics

(this is just a teaser)

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 2

Henrique Madeira, 2022/2023 1

100 million lines of code

How many software bugs?

From Rich Rogers, https://twitter.com/richrogersiot/status/958112741218111489

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 3

100 million lines of code

Rule of thumb for fault density in software

Software faults (human errors): a persistent problem

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 4

Henrique Madeira, 2022/2023 2

Many types of software complexity metrics

You will find many diagrams like this…

Adapted from: https://www.javatpoint.com/software-engineering-software-metrics

Software complexity metrics are used for many things…

• Complexity is the key element of software quality

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 6

Henrique Madeira, 2022/2023 3

Code complexity metrics

• Lines of code LoC

• McCabe complexity metrics

Code complexity metrics:

Henrique Madeira, 2022/2023 4

”Programs must be written

People tend to think/accept that

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 9

McCabe’s cyclomatic complexity: V(g)

Henrique Madeira, 2022/2023 5

Cyclomatic complexity: example

Cyclomatic complexity: example

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 12

Henrique Madeira, 2022/2023 6

Cyclomatic complexity: example

V(G) = 3 à means that the program has 3 independent paths

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 13

Path base testing

Simplistic use of V(g):

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 14

Henrique Madeira, 2022/2023 7

State-of-the-art code complexity metric:

From SonarSource Technical Report on “A new way of measuring understandability”, 2021.

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 15

State-of-the-art code complexity metric:

From SonarSource Technical Report on “A new way of measuring understandability”, 2021.

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 16

Henrique Madeira, 2022/2023 8

State-of-the-art code complexity metric:

From SonarSource Technical Report on “A new way of measuring understandability”, 2021.

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 17

Is SonarSource Cognitive Complexity (CC-Sonar)

Index (TLX) C1 Easy 5+1 15 2

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 18

Henrique Madeira, 2022/2023 9

NASA TLX results

There is a clear (and expected) saturation effect

Henrique M adeira Analysis of Software Artifacts, DEI-FCTUC, 2022/2023 19

Subjective perception, cognitive load and