See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326563648

FMEDA-Based Fault Injection and Data Analysis in Compliance with ISO-26262

Conference Paper · June 2018

DOI: 10.1109/DSN-W.2018.00075

CITATIONS READS
4 1,606

3 authors:

Kuen-Long Lu Yung-Yuan Chen

National Taipei University National Taipei University
22 PUBLICATIONS   94 CITATIONS    64 PUBLICATIONS   309 CITATIONS   

SEE PROFILE SEE PROFILE

Ryan Huang
Industrial Technology Research Institute
32 PUBLICATIONS   167 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The feasibility study for automotive Ethernet communication systems View project

Development and Verification of System-Level Fault-Robust Techniques for Safety-Critical FlexRay Network Systems View project

All content following this page was uploaded by Kuen-Long Lu on 14 September 2018.

The user has requested enhancement of the downloaded file.

 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops
FMEDA-Based Fault Injection and Data Analysis in
Compliance with ISO-26262
Kuen-Long Lu Yung-Yuan Chen
Industrial Technology Research Institute Department of Electrical Engineering,
(ITRI) National Taipei University
HsinChu, Taiwan, R.O.C New Taipei City, Taiwan, R.O.C
Department of Electrical Engineering, chenyy@mail.ntpu.edu.tw
National Taipei University
New Taipei City, Taiwan, R.O.C
sone@itri.org.tw
Abstract—With the growing demand on automotive
electronics for the advanced driver assistance systems and
autonomous driving, the functional safety becomes one of the
most important issues in the hardware development. Thus, the
safety standard for automotive E/E system, ISO-26262, becomes
state-of-the-art guideline to ensure that the required safety level
can be achieved. In this study, we base on ISO-26262 to develop a
FMEDA-based fault injection and data analysis framework. The
main contribution of this study is to effectively reduce the effort
for generating FMEDA report which is used to evaluate
hardware's safety levelġġbased on ISO-26262 standard.
Keywords—ISO-26262, Functional Safety, FMEDA, Fault
Injection
I. INTRODUCTION
Automotive E/E systems, electronic control unit (ECU),
micro-controller unit (MCU), system-on-chip (SoC) and
intellectual property (IP) used, become prevalent in the safety-
critical automotive applications, which require a stringent
dependability while the systems are in operation. Therefore,
the safety and reliability issues must be addressed in the
development of safety-critical hardware (HW) system.
Nevertheless, the incorporation of the safety/reliability
requirements into the design specification will raise the design
complexity significantly. Thus, an important and valuable
research topic is emerged: how to develop an effective safety
process following the guidelines of international functional
safety standard, i.e. ISO-262626, to assist designer in tackling
the complexity of the HW design and verification. Therefore,
we need to incorporate the safety standard into the present
automotive HW design and verification process such that the
new integrated safety design process can facilitate the
designers in assessing and enhancing the safety/robustness of
an automotive hardware in an efficient manner.
ISO-26262 [1] was first published in 2009 for needs
specific to the application sector of electrical and/or E/E
systems within road vehicles. The primary purpose for this
standard is to conduct a safety life cycle for the electronic
systems. In ISO-26262, a safety life cycle includes: concept
2325-6664/18/$31.00 ©2018 IEEE 275
DOI 10.1109/DSN-W.2018.00075
Li-Ren Huang
Information and Communication Research
Laboratories
Industrial Technology Research Institute
(ITRI)
HsinChu, Taiwan, R.O.C
lrhung@itri.org.tw
phase, product development phase and the product and
operation planning phase. During the safety life cycle, the
considering issues cover initialization of the product concept,
specification establishment, product design and pre-production
test. All these issues are treated with functional safety
consideration. At the product development phase, V-model is
adopted as the primary design, verification and validation flow.
This phase is further divided into three different levels: system
level, hardware level and software level. For these levels,
functional safety requirements are verified and validated
through Failure Modes and Effect Analysis (FMEA), Fault-
Tree Analysis (FTA), Failure Mode Effect and Diagnostic
Analysis (FMEDA) and safety-related metrics. Among these
safety analysis, FMEDA is adopted to verify the safety at HW
level by calculating the HW architecture metrics to evaluate the
HW safety level.
In this study, we develop a FMEDA-based fault injection
and data analysis framework in compliance with the functional
safety standard ISO-26262 for safety-critical automotive SoC.
The effectiveness of the framework is demonstrated by an
exemplary RISC-based SoC to show how to reduce the effort
for generating FMEDA report. This paper is organized as
follows: FMEDA and fault injection in ISO-26262 are
introduced in Section II. In Section III, our FMEDA-based
Fault Injection and data analysis framework is proposed. Case
study appears in Section IV. Then conclusions are given in
Section V.
II. FMEDA AND FAULT INJECTION IN ISO-26262
A. FMEDA in ISO-26262
The definition of FMEDA is a systematic analysis
technique to obtain subsystem / product level failure rates,
failure modes and diagnostic capability. The main purpose of
FMEDA in ISO-26262 is to evaluate HW architectural metrics
and safety goal violations due to random HW failures for
providing sufficient information to improve the gaps if the
demanded HW safety level is not fulfilled. The HW
architectural metrics include single-point fault metric (SPFM),
latent-fault metric (LFM) and probabilistic metric for hardware
failure (PMHF). SPFM reflects the robustness of the item to
single-point and residual faults either by coverage from safety
mechanisms or by design (primarily safe faults). A high SPFM
implies that the proportion of single-point faults and residual
faults in the hardware of the item is low. LFM reflects the
robustness of the item to latent faults either by coverage of
faults in safety mechanisms or by the driver recognizing that
the fault exists before the violation of the safety goal, or by
design (primarily safe faults). A high latent-fault metric implies
that the proportion of latent faults in the hardware is low.
Finally, PMHF is a probabilistic metric for evaluating the
violation of the considered safety goal due to random hardware
failure. In ISO-26262, safety is classified into four different
automotive safety integrity levels (ASIL) – A, B, C and D
where ASIL D represents the highest safety level. With higher
ASIL, the stricter safety requirements including HW
architectural metrics are needed to be fulfilled. The following
Table I shows target values for SPFM, LFM and PMHF
respectively under different target ASILs. It is worthy to note
that there is no target value for ASIL A. The details of how to
calculate SPFM, LFM and PMHF can be found in [2] and [3].
Table I TARGET VALUES FOR HW ARCHITECTURAL METRICS
UNDER EACH ASIL
B. Fault Injection is ISO-26262
Fault injection is highly recommended for HW safety
verification, the main merits are summarized as follows:
• Supporting the evaluation of the HW architectural metrics:
- Evaluating the diagnostic coverage of a safety mechanism
• Evaluating the diagnostic time interval and the fault reaction
time
• Confirmation of fault impact
• Evaluating the completeness and correctness of a safety
mechanism
- Demonstrating the completeness and correctness of the
safety mechanism to detect faults and control their effect
- Demonstrating completeness and correctness of the
functionality of the safety mechanism with respect to
requirements.
Currently, commercial EDA tools like Synopsys Z01X [4],
Cadence Incisive Functional Safety Simulator (IFSS) [5] and
Questa Verification Solution developed by Mentor Graphics [6]
and related study in [7-9] all support HW fault injection and
simulation to evaluate diagnostic coverage of a safety
mechanism. However, this tools do not support the FMEDA-
based data analysis so the designers still need to conduct
FMEDA report manually. In this study, we develop a
FMEDA-based Fault Injection and Data Analysis tool called
FIDA to generate the FMEDA report automatically so that
designer’s effort can be reduced. Detailed introduction of
FIDA is given in next section.
III. FMEDA-BASED FAULT INJECTION AND DATA ANALYSIS
The proposed framework with FIDA to generate FMEDA
report includes three main phases: (1) HW design parsing
276
phase; (2) fault simulation and data analyzing phase and (3)
FMEDA report exporting phase. The execution flow is shown
in Fig. 1. We need to emphasize that FIDA is not a stand-alone
tool. FIDA relies on the commercial hardware description
language (HDL) simulator and waveform comparator to
generate the fault simulation and data analysis results.
However, it also means that FIDA is not restricted to specific
HDL tools. Theoretically, FIDA supports all Verilog HDL
simulators and waveform comparators that can compare two
different value change dump (VCD) files.
Fig. 1 FIDA framework
In HW design parsing phase, the users can use any HDL
simulator or compiler to parse their Verilog codes for
generating the fault injection targets such as Input/ġ Output
ports, signals, clock and reset, etc. Users can also specify
number of faults injected and the fault distribution among these
fault injection targets. Phase 2 is constituted by three steps as
Fig. 1 shows.
In step “Fault Injection”, FIDA generates a set of test
benches with fault injection commands and script files for
automatically executing fault simulations in next step. Firstly,
we define a fault injection campaign constituted by: (a)
number of injected faults and the distribution of different
injection targets; (b) fault types like stuck-at-fault or bit-flips;
(c) fault duration and fault instance time and (d) permanent or
transient faults.
Users need to specify items (a) and (d) as the inputs for
FIDA. Then items (b) and (c) can be generated by FIDA
automatically. FIDA determines the fault type and duration for
each fault according to the permanent or transient faults
specified.
According to the specified number of injected faults, the
same number of test benches are generated by FIDA. In each
test bench, FIDA inserts commands to control the injection of
each fault and dumping fault simulation results. These inserted
commands are Verilog built-in commands like “force”,
“release” and “dump” and hence can be executed in any HDL
simulator.
In the step “Fault Simulation”, the only action needed is to
execute a FIDA command. This command will perform a set of
script files generated by FIDA in step “Fault Injection” and
then all fault simulations are executed automatically. After all
fault simulations are finished, waveform files in VCD format
for all simulated faults are dumped and stored in pre-specified
directory. Besides, a fault-free simulation is also executed to
generate the golden waveform.
Then in step “Data Analysis”, for each injected fault FIDA
compares (a) the fault simulation waveform with the golden
waveform and (b) SRAM contents of the fault simulation and
the fault-free simulation. If comparing results of (a) and (b) are
both different then the fault simulation result is identified “SoC
failure”. FIDA repeats the comparison for each injected fault
and accumulate the amount of “SoC failures”. Once all fault Matrix multiplication (Matrix). Two simulation configurations
simulation results are compared, the total number of SoC are shown in the following: one is for failure mode
failures is acquired. Then FIDA can calculate the diagnostic classification and the other is for FMEDA.
coverage (DC) or called failure mode coverage (FMC) in
FMEDA report through dividing the number of SoC failures by
total number of injected faults. Finally, the HW architecture
metrics can be calculated and filled in FMEDA report.
On the other hand, a failure mode analysis report is
generated to help users further recognize which failure modes
are main contributors among all SoC failures. Currently, FIDA
classifies SoC failures into 12 different failure modes, as Table
II summarized.
Table II SoC failure mode classification
Failure mode Description
Simulation ends incorrectly (earlier than
EIT/ID
expectation) with incorrect results
Simulation ends incorrectly (earlier than
EIT/CD
expectation) with correct results Fig. 3 ORPSoC block diagram
Simulation ends incorrectly (earlier than
EIT/ND
expectation) with no results A. Simulation configuration 1:failure mode classification
Simulation ends incorrectly (later than In this simulation configuration, we inject permanent faults
LIT/ID
expectation) with incorrect results to better demonstrate the failure mode classification for the
LIT/CD
Simulation ends incorrectly (later than original ORPSoC without safety mechanism protection. Table
expectation) with correct results III shows the failure mode classification results for three test
LIT/ND
Simulation ends incorrectly (later than programs. For each test program, the same 1000 permanent
expectation) with no results faults were injected to observe the fault impact on different
IIT/ID Simulation breaks down with incorrect results program features.
IIT/CD Simulation breaks down with correct results
Table III failure mode classification results
IIT/ND Simulation breaks down with no results
CT/ID Simulation ends normally with incorrect results
CT/CD Simulation ends normally with correct results
CT/ND Simulation ends normally with no results
FIDA automatically classifies failure modes according to
the corresponding waveform comparing results. Failure mode
classification is based on a series of comparisons between
fault-free and fault simulation from time and value aspect as From Table III we can observe that LIT/ID is with highest
Fig. 2 shows. For time domain, FIDA compares end time of occurrence among all failure modes which provides a useful
fault simulations with fault-free simulation for the adopted guide for devising effective safety mechanisms.
benchmark. For value domain, FIDA will dump and compare
the contents of data memory at simulation end time for fault B. Simulation configuration 2: FMEDA
simulations and the fault-free simulation for the adopted In this simulation configuration, we inject permanent faults
benchmark. In summary, users only need to prepare the into ORPSoC without and with safety mechanism. It is worthy
Verilog codes and specify related parameters for fault to note that for each fault simulation there is only single fault
simulation, then FIDA will take over the remaining tasks for injected. Thus only SPFM is calculated by FIDA. FMEDA
generating the FMEDA report. Thus, compared to traditional report with LFM is not demonstrated in this paper due to
FMEDA generation by hands, FIDA effectively reduces the limitations on space. The safety mechanism Triple Module
effort through automated FMEDA generation. With the help of Redundancy is adopted for whole OR1200 CPU in ORPSoC.
proposed framework, number of re-design iterations due to FMEDA reports for ORPSoC without and with TMR
insufficient safety level is expected to be reduced also. protection are shown on left hand side and right hand side of
Table IV respectively. Analyzing fault simulation results, we
IV. CASE STUDY observe that faults injected in certain sub-parts of the OR1200
For demonstrating FIDA effectiveness, we adopt a CPU do not cause any effect on the SoC operation. We identify
OpenRISC-core-based SoC – ORPSoC – as the case study [10]. these sub-parts as “NES” which stands for “No Effect on SoC”
This SoC is developed in Verilog RTL. Fig. 3 shows the block in Table IV (red boxes). These NES sub-parts will be excluded
diagram of the ORPSoC. from FMC and SPFM calculation. Comparison of these two
A set of benchmarks including three common programs are FMEDA reports provides the evidence of SPFM improvement
adopted: Fibonacci sequence (Fib), Bubble sort (Sort) and contributed by TMR safety mechanism.

277
 V. CONCLUSIONS
In this study, a FMEDA-based fault Injection and data
analysis framework in compliance with ISO-26262 is proposed.
Through the proposed three-phase framework with developed
tool – FIDA, fault simulations and data analysis for the
simulation results are executed automatically. Furthermore
FIDA can also automatically perform failure mode
classification and FMEDA report generation so that the
designer can rapidly recognize the weakness of HW and
establish safety mechanism to improve the safety level.
Therefore the effort to achieve the demanded HW's safety level
is effectively reduced.
ACKNOWLEDGMENT
The authors acknowledge the support of MOST under
Contract Number 106-2221-E-305-011.
REFERENCES
[1] ISO/FDIS 26262, “ISO-26262 Road Vehicles --Functional safety,” ed:
International Organization for Standardization, 2011.
Fig. 2 Failure mode classification
Table IV FMEDA generated by FIDA
278
View publication stats
[2] S. H. Jeon, J. H. Cho, Y. Jung, S. Park, and T. M. Han, “Automotive
hardware development according to ISO 26262,” in 13th (ICACT) 2011,
pp. 588-592.
[3] Y. C. Chang, L. R. Huang, H. C. Liu, C. J. Yang, and C. T. Chiu,
“Assessing automotive functional safety microprocessor with ISO 26262
hardware requirements,” in Proc.Int.Conf. Symposium on VLSI
Design,on Automation and Test (VLSI-DAT), 2014, pp. 1-4.
[4] ZOIX User Guide Version 2016.03-6, July, 2016.
[5] Randal Childers, “Enabling ISO 26262 Qualification By Using Cadence
Tools”, Cadence white paper, 2014.
[6] DOUG SMITH, “How Formal Reduces Fault Analysis for ISO 26262”,
Mentor Graphic white paper, 2017.
[7] Ludovic Pintard, Jean-Charles Fabre, Karama Kanoun, Michel Leeman,
and Matthieu Roy, "Fault Injection In The Automotive Standard ISO
26262: An Initial Approach", European Workshop On Dependable
Computing,vol. 7869: Springer Berlin Heidelberg, May 2013.
[8] N. Adler, S. Otten, M. Mohrhard, M. K. D, x00Fc, and Glaser ller,
“Rapid safety evaluation of hardware architectural designs compliant
with ISO 26262,” in Proc.Int.Conf. Symposium on Rapid System
Prototyping (RSP), Montreal, QC, 2013, pp. 66-72.
[9] M. Kooli and G. Di Natale, “A survey on simulation-based fault
injection tools for complex systems,” in Pric.9th.Int.Conf.On Design &
Technology of Integrated Systems In Nanoscale Era (DTIS), 2014, pp.
1-6.
[10] https://opencores.org/,OpenRISC SoC offical website.