SQUORE Automotive (PDFDrive)

SQUORE Automotive:
How to implement
Automotive SPICE base
practices?
Cyril Benkimoun
03/06/2014
→ Introduction
→ SQUORE Vs Automotive SPICE (15min)
→ « Risk-based » Testing Demo (30 min)
→ Conclusion / Question-Answer (10 min)
03/06/2014 © Copyright Squoring Technologies 2

About Squoring Technologies…
Revenue → Created in 2010

→ Jeune entreprise innovante (JEI) since 2011
AeroSpace&D
efense → 2013: turnover ~=1M€
Energy
→ Aerospace Valley member
Automotive → Located in France and Germany
Others
Growing
market

Our offer…
SQUORE is an Actionable dashboard
Or Process
To Software and performance rating
System Engineering
Effectiveness
Aggregation of
From Software code
Aggregation of system process adherence
quality rating KPIs
engineering artefacts
(requirements, Models,
Aggregation of IVV plans, Tests,….)
Software static
analyzers results output
=> IT market , IT
standards
SQUORE
03/06/2014 © Copyright Squoring Technologies Automotive 4
Our References…

SQUORE: An Open and
Scalable Architecture
A Collaborative Platform for
Optimizing Software Project Management
Test Bug
Requirements Coverage User Defined
Design Tracking Checklists
Source Models
Sour
Sour Sour ce
…
Sour Sour
Code Sour
Sour
ce
ce
Code
ce
ce
Code
Code Sour
ce Code
Code Sour Sour
ce
Sour
ce
Sour ce Code Sour Code
ce ce Code Sour
Code ce
Sour Code Sour ce Code
Sour Code ce
ce Code
ce ce Code Sour
Code Sour
Code Sour Code ce
ce Code
ce Code Sour
Code
ce
Code
SQUORE Evaluation Engine

Optimized Data Analysis &
Action Plan Modeling
Dashboards & Reporting
SQUORE Server
SQUORE DataBase

SQUORE: An Intuitive Portal to
Key Performance Indicators
Breakdown into Distribution of artifacts
Quality according to their level
characteristics of performance
Objective and
impartial
evaluation
Forecast Trend analysis

version by
Predicting the
03/06/2014 © Copyright Squoring Technologies version 7
trend
→ Introduction
→ SQUORE Vs Automotive SPICE (15min)
→ Conclusion / Question-Answer (10 min)

Context…
→ Software, the new engine of Automotive
 Increase of embedded electronics for

 Passengers Safety
 Functionality Reliability
 Performance
 Comfort
 Growing software and Safety Issue
 “Good enough” Vs “Time to market”

Context…
→ Emergence of standards and regulation in the Automotive

industry
 1998: MISRA C Standard (Safety programming rules)
 2005: HIS Source Code Metrics (Complexity quality

model)
 2005: Automotive SPICE (Process Assessment Model in

the development of “software-based” systems)
 2011: ISO 26262 (Safety standard for systems)

SQUORE Vs Automotive SPICE
→ Implementing requirements from Automotive SPICE is a key

to success…
→ SQUORE thanks to its core features:

 Quality and Analysis model
 Decision model
 Role-based dashboard
 Capitalisation database …
… is a pragmatical way to comply with Automotive SPICE

The SPICE
Process Reference Model …
Squoring Confidentiel 12
Specify a Test Strategy
MAN.6 Measurement
BP
Perform, retrieve measurement activities BP7 Analyse measures
5-6
BP8 Use measurement information for decision making
ENG.6 Software Construction

BP1 Define a unit test strategy
BP2 Develop unit verification criteria
ENG.7 Software Integration

BP1 Develop software integration test strategy
ENG.8 Software Testing

BP1 Develop software test strategy
BP8 Develop regression test strategy and perform regression testing

Supporting SPICE Processes:
Support Process Group (SUP)
SUP.1 Quality Assurance

BP5 Assure quality of work product
BP8 Report quality assurance activities and results
SUP.2 Verification
BP3 Conduct verification
BP5 Report verification results
HIS metrics Tailored Quality and Analysis models

for Standard compliance claiming
MISRA rule checking
Role-based dashboards for reporting
results
Import of verification data from
3rd party tools Export to "Change Management"
Compliance Table to HIS

source code metrics : i.e. the
percentage of functions that
comply with the metric
thresholds
Almost 90% of the functions

have a cyclomatic complexity
lower or equal to the xHIS
threshold (i.e 10).
Such a low percentage is
questionable regarding the
HIS standard requirements.
HIS metrics
The value of each HIS metric
for each function is accessible
Compliance claiming at the function dashboard
level.

Compliance Table to the full

MISRA standard :
The project complies with
almost two thirds of the
MISRA coding standards …
42 of them are violated at
least once in the overall
source code.
Compliance Table to the

applicable MISRA Subset
(applicable the project).
The project compliance is
much higher ... Hopefully !
MISRA rule checking However, 24 MISRA coding
standards remain
unsatisfied
Compliance claiming

Rule Checking - Reporting Results

Getting access to all non conformities Total number of non conformities found
in the code
Variation since the selected
previous version.
Just click on the

line # icon to
visualize the
related non
conformity in the
effective code.
e.g. Automated import and processing of

raw data from Rational Test-RealTime
Import of verification data

from 3rd party tools
Reporting Results

Select the Action Items to be

Export To "Change Management" exported to the Change
Sort by level of priority to focus Management supporting tool
on the "high value" actions first
All key Advices

generated by the
SQUORE Decision
Model are aggregated
in the Artefact "Action
Plan" /
The Action Items related to the ROM Consumption

problem still remains since Internal Delivery 301 …
Engineering Process Group (ENG)
ENG.6 Software Construction

BP1 Define a unit test strategy
BP2 Develop unit verification criteria
ENG.7 Software Integration

BP1 Develop software integration test strategy
ENG.8 Software Testing

BP1 Develop software test strategy
BP8 Develop regression test strategy and perform
regression testing
Tailored Decision Models for "Risk-Based" Testing / Optimizing Test Strategy
Tailored Analysis Model for Verification Criteria

Engineering Process Group (ENG)
The function has evolved from a basic

function to a quite complex function
including some advanced computation
features (i;e. call to critical standard
mathematical librariy).
The Unitary Test Strategy is

automatically tailored according to
this context recommending special
testing
Tailored Decision Model for

for "Risk-Based" Testing /
Optimizing Test Strategy
21
Management Process Group (MAN)
MAN.3 Project Management

BP3 Determine and maintain estimates for project attributes
BP
11
Review and report progress of the project
MAN.6 Measurement
BP8 Use measurement information for decision making
Tailored Analysis Models for Quality Criteria and Project Plan
Tailored Decision Models
Role-Based Dashboard for Progress Reporting

Trend Analysis
Tailored Analysis Models for

Project Plan
(e.g. Delivery Management)
Trend Analysis

The project Quality Trend (e.g. by delivery)

"Is it better or is it worth ?"
The overall level of performance of the

project regarding the applicable Quality
criteria: "Is it good or is it bad ?"

Quality Criteria
The applicable Quality Model i.e. the
standardised breakdown of product
characteristics (e.g. ISO SQUARE 25010) :
Trend Analysis a rating is provided at each level of the
breakdown


Quality Criteria
(e.g. Memory Consumption, Technical Debt)
Trend Analysis

MAN.6 Measurement
BP5 Perform measurement activities
BP6 Retrieve measurement data
BP7 Analyze measures
Capitalisation Database
Statistical Packages

Identify projects with unusual

measures…
Find what were the specifics of
project life-cycle processes
c
v
Capitalisation Database Study the distribution of the

data of all past projects stored
Statistical Packages in the database.
Identify best practices.
cv
And establish new appropriate
thresholds for future projects

Capitalisation Database
Statistical Packages Use the correlation matrix to find relations between
product characteristics (e.g. complexity) and
product quality of service (e.g. Number of bugs)

Process Improvment Process Group (PIM)
PIM.2 Process Assessment

BP6 Analyze the assessment data
BP7 Report the assessment results
BP8 Maintain the assessment records
PIM.3 Process Improvement

BP7 Confirm process improvment
Tailored Analysis Models for Assessing Process Capability and Performance
Capitalisation Database and Statistical Features
Process Assessment Review Forms

Stability Index : the percentage of unchanged code.

100% means that no existing lines has been changed.
Average value of the

stability index: 99% !
By average, no more than
1% of the overall code of
the project is changed
from one version to
another.
Statistical Features :
e.g. HIS Code Stability Index
Trend using a SPC graph
98% of the code of
the version 501 has
remained unchanged
in the version 502 …
At this time, the value observed was outside i.e. 2% of the code
the usual values: the process performed in an has been modified or
different way. This should be studied deleted.

The Technical Debt has been reduced

from version 402 to version 403 …
while the code size still increased.
Indeed, a good performance of the
EngineeringProcesses !

→ Introduction
→ SQUORE Vs Automotive SPICE(15min)
→ Conclusion / Question-Reponse (10 min)

Practical use case: Implementing
a risk-based Test Strategy
Minimizing the risks while improving the overall quality?
 A « risk-based » test strategy refering to ASIL Level, Code

quality, test coverage …

« Risk-based » Testing Strategy
→ Decision Criteria for Risk Based Testing.
What shall I test right now ? Focus on changed, critical,

Component X or Component Y? and complex software
components first!
X Y
Software Components
→ Decision Criteria for Risk Based Testing.
Number of previous defects

Component Coupling
Software Quality Index :
Criticality: - HIS Complexity
ISO 26262 ASIL - MISRA Standard compliance
- Unitary test coverage
- ...
Code stability
…
X Y
Software Components
→ Decision Criteria for Risk Based Testing…. Example at Function level.

For all new or modified functions …
… and containing calls to
critical libraries
Stability Index
< 100%
or
New function
Call to mathematical
libraries
Control Flow
Complexity
… that have some complexity…

→ Decision Criteria for Risk Based Testing…. Example at component level.
 In case of changes in a safety-critical

Component
Stability Index
Stability Index
< 80%
< 100%
Strong Coupling
ASIL C or with Components
ASIL D
… and a strong coupling with others

Components

Demo
→ Using SQUORE for « Risk-based » Testing
 Objectives:
 Optimize your test effort at each delivery using risk-
based criteria
 Means
 Assessing your code quality and standard compliance
 Identifying Critical resources
 Optimizing the test strategy at artefact level

SQUORE: Agregating Information
Flows from SDLC processes
→ A typical Information Flow set up on a SQUORE Project:
Change Mgnt. Conf. Mgnt.
Req. Mgnt.
Mantis Doors SVN
Jira Reqpro GIT
Coding ClearQuest Reqtify ClearCase
… … …
Source
Source
Code
Source
Code
Source
Code
Source
Code
Code
Pc-Lint – QAC GCOV

CodeSonar, Junit
Klocwork RTRT
… …
Rule Checking Test - V&V

→ Introduction
→ SQUORE Vs Automotive SPICE
→ « Risk-based » Testing Demo
→ Conclusion

A SQUORE Package:
SQUORE for Automotive
SQUORE A dedicated SQUORE configuration to fit the
Automotive specific needs and contraints of the targeted users
 “Out of the box” Quality and Analysis models based on

applicable Automotive standard and regulation:
e.g. HIS Metrics, MISRA rule set, ISO 26262, ...
 Predefined Decision Models for Management and Engineering:

e. g. Unit and Integration Test Strategies, …
 Embedded C/C++ Source code Parsers

 Connectors to usual “in-used” COTS:
e.g. RTRT, Polyspace, QA-C, Doors…
 Checklists and Review form supporting Key Processes and Work

Products
© Copyright Squoring Technologies
03/06/2014 41
Conclusion
→ Models can be extended to others project artifacts:

 Requirement
 Change Management
…

Conclusion
→ Read our Expert Opinion about « Risk-based » Testing

 Download it from our blog
 http://blog.squoring.com/

SQUORE Automotive (PDFDrive)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SQUORE Automotive (PDFDrive)

Uploaded by

Copyright:

Available Formats

SQUORE Automotive:

→ SQUORE Vs Automotive SPICE (15min)

→ « Risk-based » Testing Demo (30 min)

→ Conclusion / Question-Answer (10 min)

03/06/2014 © Copyright Squoring Technologies 2

Revenue → Created in 2010

03/06/2014 © Copyright Squoring Technologies 3

SQUORE is an Actionable dashboard

03/06/2014 © Copyright Squoring Technologies 5

SQUORE Evaluation Engine

03/06/2014 © Copyright Squoring Technologies 6

Forecast Trend analysis

→ SQUORE Vs Automotive SPICE (15min)

→ « Risk-based » Testing Demo (30 min)

→ Conclusion / Question-Answer (10 min)

03/06/2014 © Copyright Squoring Technologies 8

→ Software, the new engine of Automotive

 Increase of embedded electronics for

 “Good enough” Vs “Time to market”

03/06/2014 © Copyright Squoring Technologies 9

→ Emergence of standards and regulation in the Automotive

 1998: MISRA C Standard (Safety programming rules)

 2005: HIS Source Code Metrics (Complexity quality

 2005: Automotive SPICE (Process Assessment Model in

 2011: ISO 26262 (Safety standard for systems)

03/06/2014 © Copyright Squoring Technologies 10

→ Implementing requirements from Automotive SPICE is a key

→ SQUORE thanks to its core features:

03/06/2014 © Copyright Squoring Technologies 11

BP8 Use measurement information for decision making

ENG.6 Software Construction

ENG.7 Software Integration

ENG.8 Software Testing

03/06/2014 © Copyright Squoring Technologies 13

SUP.1 Quality Assurance

HIS metrics Tailored Quality and Analysis models

Compliance Table to HIS

Almost 90% of the functions

03/06/2014 © Copyright Squoring Technologies 15

Compliance Table to the full

Compliance Table to the

03/06/2014 © Copyright Squoring Technologies 16

Rule Checking - Reporting Results

Just click on the

e.g. Automated import and processing of

Import of verification data

03/06/2014 © Copyright Squoring Technologies 18

Select the Action Items to be

All key Advices

The Action Items related to the ROM Consumption

ENG.6 Software Construction

ENG.7 Software Integration

ENG.8 Software Testing

Tailored Decision Models for "Risk-Based" Testing / Optimizing Test Strategy

Tailored Analysis Model for Verification Criteria

The function has evolved from a basic

The Unitary Test Strategy is

Tailored Decision Model for

MAN.3 Project Management

Tailored Analysis Models for Quality Criteria and Project Plan

Tailored Decision Models

Role-Based Dashboard for Progress Reporting

Tailored Analysis Models for