International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194

A Procedure to Verify and Validate an FPGA Level

Testing as Per DO-254
Dr. Manju Nanda#, P Rajshekhar Rao*
#,*
Aerospace Electronics & Systems Division, CSIR- National Aerospace Laboratories, Bangalore, Karnataka, India

Abstract:-This paper describes the how advanced verification
like constrained random verification or directed tests which
improves the system performance in FPGA level testing or
hardware target level testing which outcomes safety analysis and
performance based scenarios. Basically recent avionics
industries facing while testing the hardware design using FPGA
level mode of testing in to different feeded inputs, capturing with
different techniques, which will improves the system verification
on random based approach methods. Finally this paper tells
about to improve the hardware design data (HDD) needs to
trace the hardware requirement document (HRD) accurately
using the randomization methods as carried out by constrained
random verification.
Keywords: DO-254, FPGA, directed tests, verification
 Configuration Management Process
 Certification Liaison Process
Integral processes are parallel processes that shall be adopted
throughout the lifecycle of the project. While traversing in the
standard hardware lifecycle processes, the transition criteria
shall be met before transiting to next process in the sequence.
Means that, before start of next process the previous process’s
output must be available as an input. So transition criteria
shall define the minimum data/outcome of any process to start
next process [2].
Planning Development Process
PLANNING REQUIREMENTS DESIGN
IMPLEMENTATION
/INTEGRATION

Integral Processes
I. INTRODUCTION
Detailed
Conceptual
Hardware Planning Requirement Design Implementation

V
Design Process
alidation and Verification Process conducts the hardware Process Capture Process
[High-Level]
Process
[Low-Level]
Process

requirements review to determine whether the captured

hardware requirements satisfy the objectives of RTCA/DO-
254. This review also reduces the risk of an applicant
producing hardware requirements that does not meet
RTCA/DO-254 objectives or other certification criteria.
Hardware requirements review should take place after the
initial completion of the hardware requirements capture
process. The reviewed and approved hardware requirements
documents serve as primary driving factor for design,
implementation and testing processes[1].
To ensure the completeness and correctness of requirements,
all requirements of the hardware will be reviewed as per
RTCA/DO-254.
A. Hardware processes.
The processes involved in the hardware design lifecycle
process are:
 Hardware Planning Process
 Hardware Design Processes
 Requirement Capture Process
 Conceptual Design Process
 Detailed Design Process
 Implementation Process
 Product Transition Process (Not Applicable
in this case)
 Verification/Validation Process
 Integral Processes
 Process Assurance Process
Fig. 1. Hardware Design Lifecycle Processes [4]
B. Relationship between Processes and Activities
The processes established as per RTCA/DO-254 guidelines
are carried out by different teams. During the planning
process, the activities to be performed in every process are
defined in the planning document. After review, they are
brought under configuration control. The activities related to
certification liaison process and system process is not brought
under the configuration control process established for
hardware development [3].
The system team provides the Functional requirements
document and system design. The design and development
team designs the IP-core as per the system requirement. The
system requirements allocated to hardware are translated to
high level and low level requirements. The output of hardware
development process is verified by an independent
verification team.
Table 1: Hardware process and activities
Hardware Design Processes Planned Activities
Plan to meet all objectives of
lifecycle process
Define Hardware
Development Lifecycle
Planning Process
Define Hardware
Development lifecycle
standards
Define Program Milestones

www.rsisinternational.org Page 48
International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194

Hardware Design Processes Planned Activities  Identification of evidence reports to be produced out
and Reviews
of verification and validation independently
Define Transition Criteria
among processes  Identification of analysis/test tools and equipment to
Define Development & be used to meet verification and validation process
Testing Environment and objectives.
assisting tools
Define Means of Compliances  Describes the means of independence to ensure
of hardware design assurance verification purpose for those objectives requiring
objectives for certification independence.
authority
Define problem reporting and
 Identification of organizational responsibilities for
tracking resolution implementing independent verification process
mechanisms
Define/Document system's II. DETAILED DESIGN PROCESS
allocated and derived
requirements for hardware The Detailed Design Process produces detailed design data
item
Requirement
Define/Document design using the hardware item requirements and conceptual design
Capture Process data as the basis for the detailed design such as HDL/RTL.
constraints
Traceability to be maintained Hardware Design Standard shall be followed during the
across the different levels of Detailed Design phase of development lifecycle [4].
requirements
Design Hardware items
architecture
A. Detail Design Process Objectives
Define functional blocks and
Design
Conceptual Design
its interfaces etc.,
 Detailed design is developed from the hardware item
Process requirements and conceptual design data
Process Define Electrical interface
characteristics between design  Trace Data: Derived requirements are fed back to the
blocks conceptual design process or other appropriate
Design HDL code from
hardware item requirements processes.
Detailed Design and conceptual design data  Trace Data: Requirement omissions or errors are
Process Develop test methods and provided to the appropriate processes for resolution.
interface data for verification
of design B. Detail Design Process Inputs and Outputs
Perform simulation and
Implementation synthesis Fig. 2. Shows the inputs and outputs of detail design process
Process Capture design constraints
Perform target timing analysis
Generate compliances with INPUTS OUTPUTS
respect to design requirements
and Design assurance level. Hardware Design
Description HDL /RTL
Generate design deviation
report (if any) and raise & Hardware Block
Design
track the problem till it gets Diagram
Detailed Constraints

resolved Design Process

Verify the traceability of all
requirements from top to
Integral Verification/ bottom and vice versa
Processes Validation Review of design artifacts
Process Generate review checklists
Generate verification
plans/methods and
compliance documents
Identify the acceptance test
criteria
Generate test cases, execution
of test cases and log test results
Perform formal
simulation/testing on target
C. Tasks
Hardware Verification Plan should include following details
in the document.
 Methods to be adopted for Verification & Validation
of hardware items independently
Hardware
Coding Standard
Hardware VV
Standard
Integral Process
Integral Process Records
Fig. 2. Detail Design Process Inputs and Outputs [4]
C. Detail Design Process Activities
 Generates the detailed design data for the hardware
item includes interconnection data, component data,
HDL and test methods.
 Test features should be designed in, where necessary,
to facilitate verification
 Constraints on the design should be identified.
 Derived requirements produced during the detailed
design process should be fed back to the conceptual
design or other appropriate processes.

www.rsisinternational.org Page 49
International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194

 Requirement omissions and errors discovered during Validation and Verification Process ensures that the test cases,
the detailed design process should be provided to the methods, procedures, pass/fail criteria and results/coverage
appropriate process for resolution are reviewed to determine whether the developed test cases,
methods, procedures, pass/fail criteria satisfy the objectives of
III. TESTING PROCESS RTCA/DO-254.
The Mentor Graphics Questa Sim shall be used for following Verification of the implementation is the verification (e.g.
testing purpose and same is represented in Fig. 3. post layout simulations) of the detailed design after place and
route and of the device itself.
 RTL Behavioral Simulation
 Post Synthesis Simulation B. Reviews and Analysis
 Post Place & Route Simulation The entry criteria to perform the review of test cases,
methods, procedures, pass/fail criteria and results/coverage
INPUTS OUTPUTS are release of initial version of those data items and other
Questa Tool supporting hardware verification data will be under
Design VHDL Verification
Code & Test RTL Behavioural Results/Coverage
configuration control as appropriate. Designers will perform
Simulation
Bench Reports the review of these test cases, methods, procedures & pass/fail
criteria and provide the feedback to the testing team.
Questa Tool

Synthesis Model Verification

The verification on device level, to assess unacceptable
of Netlist & Test Post Synthesis Results/Coverage robustness defects through RBT (requirement based testing) to
Bench Simulation Reports
cover the normal and abnormal input conditions and operating
conditions. HVVP document will be prepared to justify for
Questa Tool
level of implementation (RTL, post layout, physical device,
Place/Route Model Post Verification
of Bitstream & Test Place/Route Results/Coverage board level) and the type of the planned verification activities
Bench Reports
Simulation (test, simulation, analysis, inspection etc.,) [5].
C. Requirements-Based Test Coverage Analysis
Fig. 3. : Testing Tool Usage
Mentor Graphics Reqtracer shall be used along with HDL
A. Simulation and On-Target Testing
designer to generate requirement coverage analysis. Test cases
Mentor Graphics Questa Sim shall be utilized for performing shall be generated to perform requirement test coverage for all
the simulation activities of the designed IP-core. On target the individual requirements.
testing is limited for performing functionality test of the IP-
D. Elemental Analysis
core.
Elemental Analysis (code coverage) will be performed on
B. Test Execution and Test Results Compilation
RTL code and on Post Synthesis verification model at
Test cases execution for Behavioral Simulation, Post integrated levels.
Synthesis Simulation, Post Translate simulation and Post PnR
(Place & Route) simulation will be performed and associated V. HARDWARE DESIGN METHODS
results will be generated.
 Take the design under test (DUT) and simulate the
C. Elemental Analysis Resolution design using the test bench provided.
Elemental Analysis (code coverage) will be performed on  Create the required input and output text files to store
RTL code and on Post Synthesis verification model at the input and output values for the design
integrated levels. respectively.
 Simulate the behavioral model using ISim tool.
IV. INTEGRAL PROCESSES
A. *To compare and verify :
Integral processes are integral part of any hardware
development lifecycle process. It is a parallel path from start  Create a new test bench where the inputs are to be
of planning process till the product conformity process. The given manually and their corresponding outputs are
Integral processes include Validation/Verification, obtained.
Configuration Management, Process Assurance and  The objective is to view the results obtained and
Certification Liaison processes. compare them with the outputs of the given test
bench.
A. Validation & Verification Process Objectives and
Activities B. *Procedure to create the test bench:

www.rsisinternational.org Page 50
International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194

 After a thorough analysis of the design, add a new -- <type_name> : std_logic;

source to the top module. -- end record;
 Select the VHDL test bench option, where the test --
bench created that is the<.tb file> of the given -- Declare constants
design. --
 Within the test bench, create a process for the reset, -- constant <constant_name> : time :=
clock and/or other periodic signals. <time_unit> ns;
 Within the stimulus process, force values to the -- constant <constant_name> : integer :=
inputs and give a delay depending on the clock and <value;
reset signals. --
-- Declare functions and procedure
C. *Procedure to force values: --
* Without constraint random verification -- function <function_name> (signal <signal_name> : in
<type_declaration>) return <type_declaration>;
 For std_logic_vector the syntax is as -- procedure <procedure_name> (<type_declaration>
follows: <constant_name> : in <type_declaration>);
 <input_name> <= ”<value>”; --
(value must have length equal to the vector length )
end func_pkg;
 For std_logic the syntax is as follows:
 <input_name> <= ‘<value>’; package body func_pkg is
 After ending the process , end the architecture, save
the test bench and do the behavioral check syntax. ---- Example 1
 Obtain the output waveforms via simulation using -- function <function_name> (signal <signal_name> : in
<type_declaration> ) return <type_declaration> is
the ISim tool.
-- variable <variable_name> : <type_declaration>;
 Compare the results obtained from the new test -- begin
bench with the results obtained from the previous test -- <variable_name> := <signal_name> xor
bench . <signal_name>;
 If the results of the two test benches match, then we -- return <variable_name>;
can conclude that the former test bench is -- end <function_name>;
functioning properly. ---- Example 2
* With constraint random verification -- function <function_name> (signal <signal_name> : in
<type_declaration>;
1) Create a vhdl package file func_pkg.vhd and copy -- signal <signal_name> : in
paste the code given below:
<type_declaration> ) return <type_declaration> is
library IEEE; -- begin
use IEEE.STD_LOGIC_1164.all; -- if (<signal_name> = '1') then
use IEEE.STD_LOGIC_arith.all; -- return <signal_name>;
use IEEE.STD_LOGIC_signed.all; -- else
use ieee.numeric_std.all; -- return 'Z';
use ieee.math_real.all;--(contains the functions uniform and -- end if;
trunc) -- end <function_name>;
package func_pkg is
shared variable seed1:integer:=844396720; ---- Procedure Example
shared variable seed2:integer:=821616997; -- procedure <procedure_name> (<type_declaration>
impure function RANDOM_NUM_GEN(constant <constant_name> : in <type_declaration>) is
lower_value:in integer; --
constant upper_value:in integer; -- begin
constant busWidth:in integer)return integer; --
-- end <procedure_name>;
impure function RANDOM_NUM_GEN(constant
-- type <new_type> is lower_value:in integer;
-- record constant upper_value:in integer;
-- <type_name> : std_logic_vector( 7 downto 0); constant busWidth:in integer)return integer is

www.rsisinternational.org Page 51
International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194

variable result :integer; Hardware design data includes plans, standards, design,
variable tmp_real:real; procedures, methods and results/analysis produced in project
begin lifecycle. Hardware lifecycle data should follow some
assert(lower_value<(2**busWidth)) characteristics as listed below:
report"RANDOM_NUM_GEN():lower_value RANGE is
exceeded"  Unambiguous
severity failure;  Complete
assert(upper_value<(2**busWidth))  Verifiable
report"RANDOM_NUM_GEN():upper_value RANGE is  Consistent
exceeded"  Modifiable
severity failure;  Traceable
uniform (seed1,seed2,tmp_real); VII. RESULTS
result:=integer( trunc ((tmp_real*real(upper_value-
lower_value))+real(lower_value))); A. Trace Data Objective Evidence of Compliance
return result;
Traceability connects between two or more elements in a
project such as functions, requirements, concept, design,
end RANDOM_NUM_GEN;
verification data and test results shall be prepared across all
traceable data. The outputs of all traceability activities are the
end func_pkg;
traceability reports showing the correlation between the
project elements. Traceability reports are reviewed for
2) Check the syntax of the code.(if there are correctness internally within organizations, and audited by
modifications to the range) certification authorities.
3) To use the given code in a test bench
Tracing hardware lifecycle data right from requirements till
EXAMPLE: the test results shall be carried out using COTS tools
(ReqTracer). Hierarchy of the traceability for different
Inside the architecture block (after the begin statement) a artifacts is shown in Fig. 4.
process is added. Say A,B are input buses of std_logic_vector
System
(7 downto 0) to operate a random number for each buses ,call Requirements
the function RANDOM_NUM_GEN() and then call
conv_std_logic_vector() to convert it to bit vector stream. Hardware
That is: Requirements

use work.func_pkg.all; --library

……… Conceptual Design
Unit level test Plan
Integration level
Description test Plan
Use ieee.std_logic_arith.all;
Testing:process
Begin RTL/HDL and Unit level
Unit level Integration Integration
Coverage
Test level Test level Test
A<= constraints Test Bench
Report Bench Report
Report
conv_std_logic_vector(RANDOM_NUM_GEN(20,256,A’LE
NGTH),A’length); Fig.4. : Hardware traceability hierarchy
B<=
conv_std_logic_vector(RANDOM_NUM_GEN(50,200,B’LE VIII. CONCLUSION
NGTH),B’length); At a conceptual design stage, a block-level description of the
End process; design consistent with the requirements documents, detailing
……. all of the major components and potential sources will be
If any parameter i.e.upper_value or lower_value is exceeded included in the FPGA/CPLD design document. Any derived
,the severity failure is indicated and the simulation is stopped. requirements and any errors or omissions detected in the
(NOTE: the code given above will generate random values in requirements document from this process are fed back to the
an a periodic fashion and no value is repeated, only values requirements document.
between the specified range is generated (CONSTRAINED)).
To get the random numbers in a periodic manner for a given At detail design stage, detailed design work will begin on
period of time use a loop with a sensitivity list. creation of the HDL for major components as well as the
definition of test features and design failure mitigation. Errors
VI. HARDWARE ITEM DATA or omissions detected in the earlier steps are fed back to the
requirements document.
A. Hardware Design Data

www.rsisinternational.org Page 52
International Journal of Research and Innovation in Applied Science (IJRIAS) | Volume III, Issue VI, June 2018|ISSN 2454-6194
During the implementation phase, the actual system is
implemented. In the case of programmable logic, the design is
synthesized, place-and-route completed, and Bitstream is
generated. Errors or omissions detected in the earlier steps is
fed back to the requirements document.
REFERENCES
[1]. X. Y. Chen W. J. Xu X. Yang Y. W. Xia "SystemVerilog
assertions and its application" China Integrated Circult vol. 16 no.
19 pp. 19-24 2007.
[2]. Y. W. Xia Verilog Digital System Design Tutorial Beijing:Press
Of Beihang University 2003.
[3]. X. Jin W. Y. Ding C. Yan "Verification and testing of IP core with
a RISC architecture inside" Semiconductor Technology vol. 28 no.
11 pp. 32-35 2003.
www.rsisinternational.org
[4]. T. J. Foster D. L. Lastor P. Singh "First silicon functional
validation and debug of multicore microprocessors" IEEE Trans.
Very Large Scale Integr. (VLSI) Syst. vol. 15 no. 5 pp. 495-504
May 2007.
[5]. J. Rose J. Luu C. W. Yu O. Densmore J. Goeders A. Somverville
K. B. Kent P. Jamieson J. Anderson "The VTR project:
Architecture and CAD for FPGAs from Verilog to routing" Proc.
ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays pp. 77-
86 2012.
[6]. S. Tasiran K. Keutzer "Coverage metrics for functional validation
of hardware designs" IEEE Design Test Comput. vol. 18 no. 4 pp.
36-45 Jul./Aug. 2001.
[7]. M. Karimibiuki K. Balston A. J. Hu A. Ivanov "Post-silicon code
coverage evaluation with reduced area overhead for functional
verification of SoC" Proc. IEEE Int. High Level Design Validation
Test Workshop pp. 92-97 2011.
Page 53