An Overview of Static and Simulation-Based
Techniques for Systems-on-Chip Verification
João Luiz Carneiro Carvalho
Departamento de Engenharia Elétrica
Universidade Federal da Bahia, Escola Politécnica
Salvador, Brazil
joao.luiz.cc@gmail.com
Abstract — The effort of verifying in Integrated Circuits has
increasing progressively in the last decade, mainly due to the
complexity of large integrated circuits, such as Systems-on-Chip
(SoC). This demand has motivated the semiconductor industry,
including EDA vendors, to develop industry standards and
powerful EDA tools trying to lower the difficulty of verifying
complex designs. Several solutions are available to attack the
problem in different strategies, but it is possible to use them
combined to increase the robustness of Verification. This paper
discusses the contemporary use of most used techniques, but
focus mostly on Simulation-based Verification.
Keywords — Functional verification; System-on-Chip; Simulation;
I. INTRODUCTION
In the last decades, hardware verification has gained focus
in the semiconductor industry. This emphasis is mainly
fostered by market pressures and also by increasing design
complexity [1]. Hardware verification is placed throughout the
Integrated Circuit (IC) implementation flow, including the
comparison of chip implementation against the documented
specification or reference model and the analysis of static
properties like portability, synthesizability and manufacturing
process rules. The part that verifies if the IC has been
implemented according to its intent and without functional
bugs is called Functional Verification [3].
As the Integrated Circuit complexity grows, the cost and
complexity of functional verification also increase.
Furthermore, the cost to repair a functional bug in design
grows significantly as the project advances to final phases in
design flow. In this case, the cost of an eventual re-design
during finishing procedures like tapeout and packaging may be
impracticable in terms of project budget and schedule (time-to-
market) [1].
On the other hand, a complex digital hardware description
(RTL) has uncountable states and several modes of use so that
the hardware correctness and completeness assurance can
demand more resources than available [4]. In this context,
recently the semiconductor industry have been investing efforts
to make functional verification more efficient by promoting
verification languages, tools, techniques and methodologies
that reduce the engineering costs during verification [5]. Even
so, hardware verification still represents a critical part of
integrated circuits design and discussion about the next-
generation tools and methodologies are far from over.
Wagner Luiz Alves de Oliveira
Departamento de Engenharia Elétrica
Universidade Federal da Bahia, Escola Politécnica
Salvador, Brazil
wlao@ufba.br
This paper presents the most used functional verification
techniques for digital IC in semiconductor industry, based on
the recent literature, EDA vendors and market study. The
section II describes the scope of functional verification. The
sections III and IV refer to different types of functional
verification, whereas the section V presents contemporary
details of using described techniques.
II. FUNCTIONAL VERIFICATION
From an abstract specification to the manufactured chip,
universities, research institutes and design centers follow a
reasonably consolidated design flow. Each step inside this
process represents an incremental approximation of the final
chip, so that in latter phases of IC development the hardware
description contains physical information directly related to
manufacturing process. In parallel with design flow there are
verification activities, where outputs produced by each design
step are verified against the primary hardware specification or
implementation rules. Figure 1 illustrates the standard flow of
digital circuit design.
Figure 1: Standard IC Design Flow
The starting point of Functional Verification is a
Verification Plan. This is the most important document for
verification team because it defines the design features that will
be tested and the tools and strategies to be applied. A
verification plan also defines metrics to indicate the
verification progress and conclusion criteria.
In verification domain, there are basically two types of
verification techniques: Static and Simulation-based.
Alternative techniques to verify SoCs include Hardware
Accelerated Simulation and Field-Programmable Gate Array
(FPGA) Prototyping [10] [11] [12]. Whereas Hardware
Acceleration consists of connecting the design simulator to an
external hardware to save CPU cycles, FPGA prototyping
involves porting the HDL description in any stage of
development (since it is synthesizable) into a FPGA device
placed in a development board or a custom printed circuit
board, then actually running it. Because these techniques make
use of external hardware and this paper focus exclusively on
computer-based verification, they will not be discussed here.
So far, simulation-based verification is the most used
technique (as discussed in next sections), but there are
limitations in this approach. Firstly, simulation tools require an
environment1 specially built to generate stimulus to be driven
to Design Under Verification (DUV) – such environment is
also referred to as testbench. Usually, user-defined stimuli are
limited to typical and predictable scenarios so that the
comprehensiveness of tests is limited in terms of coverage of
design states [2].
To overcome the weak coverage of user-defined scenarios,
several test cases and test sequences are applied, resulting in
huge code to be executed and countless CPU cycles to
complete test batches. Finally, the task of describing tests
depends on the programming of scenarios by Verification
Engineer, which can be numerous and error-prone. Defining
only a fraction of the possible states that hardware assumes can
lead to a weak verification [5].
Furthermore, even if the hardware described behaves
correctly in typical and corner cases, a RTL badly designed can
infer extraneous or inefficient circuits during logic synthesis,
and hence compromising the design integrity. In this context,
design guidelines and good practices of coding are included as
essential practices in hardware development, but EDA tools
that statically scan HDL description looking for suspicious
structures play an important role in verification.
III. STATIC VERIFICATION
The purpose of static verification is to achieve high
confidence of design correctness by formal analysis without
depending on user-defined and error-prone testbenches.
Several tools and techniques explore static aspects of HDL
design, including syntax, semantics and functional behavior.
The most common are described in the following topics.
A. Lint Checking
The name “lint” is generally used to define a program that
examines source code for suspicious or non-portable constructs
of a software or hardware description. In hardware domain, lint
tool checks the design for HDL quality metrics like code
consistency, portability, synthesizability, testability, etc. [6].
Lint checking is actively used in hardware design, especially in
intermediary phases of development. Due to the ease of use
and testbench independency, this technique is actively used by
any project member involved with design or verification.
B. Formal Verification
Formal verification (FV) makes use of mathematical
representation to prove the correct implementation of design
[2]. Using FV the design is exhaustively explored to prove its
1
The environment is usually written in a Hardware Verification Language
(HVL), like SystemVerilog, for instance.
properties and find bugs. A FV environment is composed of the
design itself, the formal specification, a set of properties, and
an element that generates valid stimuli, the FV driver.
Appling FV for complex hardware is not an easy task. The
expected behavior must be formally specified, hidden design
features or assumptions must be modeled and long detailed
proofs can difficult the review. Because of that, formal
verification is not very spread in semiconductor industry.
However, the rigor of mathematical proofs is an important
aspect to be considered when verifying hardware of critical
applications, such as medical equipment and aircrafts.
C. Logic Equivalence Checking
Eventually, synthesis tools infer different logic or remove
original structures from RTL design. Generally it is induced by
optimization algorithms of synthesizer, but in most cases the
reason is bad logic in HDL or inconsistent synthesis
parameters. In these circumstances, designer needs to verify if
the functional behavior of synthesized logic remains intact, but
simulation may not be sufficient (consider the huge size of
design state space).
The best way to check if a post-synthesis design works
exactly as its pre-synthesis version is comparing both
combinational logic. If RTL description is correctly mapped to
netlist, the number of registers remains the same and each
combinational path between registers and ports are logically
equivalent. The process of verifying the equivalence between
two HDL descriptions is called Logic Equivalence Checking.
IV. SIMULATION-BASED VERIFICATION
Static Verification is a powerful technique, but it generally
requires deep mathematic background to obtain effective
results and often demands a huge effort for debugging. On the
other hand, simulation by user-defined stimulus may be less
rigorous than static techniques, but the advantage of simulation
is the possibility to debug design using internal signals and
known stimuli. When simulating a design, user is able to
follow the execution of HDL and monitor the value of any
particular signal during all simulation time.
Recently, several techniques and methodologies have been
created to strengthen the Simulation-based verification. Some
of them are discussed in the next topics.
A. Directed Tests
Possibly the simplest and most intuitive way to verify a
design is writing a program (testbench) with a sequence of
fixed stimuli to verify one feature at time. In more
sophisticated approaches, a testbench can be built to drive
multiple sequences of stimuli and hence verifying a set of
design features. However, in both cases every fixed sequence
must be directly defined by user before running the testbench,
and for that reason this technique is called Directed Test.
Designing a set of pre-defined stimuli to drive has the
advantage of the full control by the user of what is been driven.
However, the verification members usually specify directed
tests that represent typical scenarios or possibly imaginable
corner scenarios.
Although the user can track the efficiency of executed tests
by analyzing functional coverage and state coverage of design
(it will be discussed later), defining multiple directed tests to
verify each scenario induces many testbench code to be
written, and consequently many code to be maintained.
Additionally, the simulation time required to hit a corner case
is greater than introduction of random stimuli.
B. Constrained-Random Simulation
The problem of generating user-defined stimuli relies on
the high dependence of the identified and modeled valid
scenarios by verification user. To overcome this problem, an
alternative approach is generating random stimuli, increasing
the chance of rapidly reach extreme or rare states of design,
also named corner cases. In the other hand, using random
stimuli leads to invalid and unreal design states, which can
generates a false bug or an unnecessarily verified scenario.
As expected, hardware in normal operation usually has
constraints in its states, events and signals. For example, if a
configuration register is 8-bit wide but the minimum and
maximum value are, respectively, 1 and 99, any value outside
this range is considered illegal. Then, the constraint of such
register is assuming only the valid values.
These constraints can be modeled by user in the object that
randomly generates the stimulus. During simulation, the
simulator generates randomized data based on the designed
constraints, choosing values that strictly fit the user-defined
criteria. The SystemVerilog language contains structures to
define signal constraints, as shown in the algorithm below.
Algorithm 1. Example of SystemVerilog Constraint
// data field
rand bit[7:0] cfg_reg;
// valid config. values must be between 1 and 99, inclusive
constraint valid_cfg { cfg_reg inside {[1:99]}; }
C. Coverage-oriented Verification
Inside the verification environment, coverage metric is an
important parameter to assess verification progress. The main
motivations to use coverage metrics in Functional Verification
are the need to identify visited states and the need to define an
end-of-verification point, where the verification is considered
done when the coverage is 90%, for example. Full coverage
(100%) is very challenging to achieve because of the large
number of states and the particularity of some corner states,
which are difficult to arrive in simulation.
The Algorithm 2 demonstrates a structure to collect
coverage data, named coverpoint. In this example, an object
representing a transmission packet has fields for source and
destination identifications. The value of each source and
destination IDs, including the combination of these values, are
captured by the coverpoint. Every time a packet is captured,
the coverpoint is notified and the coverage report is updated.
Algorithm 2. An exemple of covergroup in SystemVerilog.
Packet pkt;
...
covergroup cov1;
s: coverpoint pkt.src_id {
bins src[8] = {[0:7]};
} environment. The adoption of advanced techniques such as
d: coverpoint pkt.dst_id {
bins dst[8] = {[0:7]};
}
cross, s, d;
endgroup cov1;
D. Assertion-Based Verification
An assertion is a positive statement about a property of
design. If the property does not remain true in all simulation
time, an error indication is issued for the assertion [8]. An
expression defines the assertion by using a combination of
signals or a sequence of events that represents one design
feature to check on-the-fly the consistency of that feature.
Therefore, assertions verify the hardware, but they depend on
stimuli generated by the testbench.
The place of assertion does not affect its behavior. For
example, the Algorithm 3 shows an assertion placed inside the
module m. Obviously, the assertion cannot be synthesized
together with the module logic, but its presence inside the
instance gives full visibility of module internal signals and
enables the RTL designer to use assertions inside his/her
module during its development, improving the reliability of
design.
Algorithm 3. A simple assertion inside a Verilog module.
module m( input wire a, clk)
reg a, b;
always @(posedge clk) begin
a <= c;
b <= !c;
end
assert property (@(posedge clk) a != b)
else $error(“a and b are equal”);
endmodule
As mentioned before, assertions depend on external stimuli
(from testbench) to activate their continuous verification. The
Assertion-Based Verification relies on using assertions to
support the verification of design features, without monitoring
the input and output stimuli [8].
V. RECENT SCENARIO
HDL simulators are widely used in IC design centers [7].
The advantages discussed in the previous sections encourage
the engineers for the simulation throughout the project.
However, simple simulations based on directed tests are no
longer satisfactory to fully implement the scenarios and metrics
contained in the Verification Plan.
In Functional Verification context, the previously presented
approaches are distinct techniques to address the problem of
verification complexity. Fortunately, many of them can be used
in a combined fashion, which enhance the verification
environment robustness. Modern EDA tools support and
encourage the use of mixed different verification techniques
such as constrained-random stimulus generation, code and
functional coverage and assertions.
A study carried by Wilson Research Group and Mentor
Graphics Corp. [7] demonstrates the adoption of verification
techniques presented in this paper. The study compares
different surveys conducted between 2007 and 2012, allowing
the analysis of hardware verification evolution in the enterprise
constrained-random simulation and functional coverage is
increasing significantly, as illustrated in Fig. 2.
 VI. CONCLUSION
80%
70% As the design complexity increases, several verification
70% 66%
62% 63% tools and techniques emerged as an evolution of old ones or as
60% a new approach to be adopted. Although the simulation-based
48% verification is widely used across different industry domains,
50%
41% 40% other approaches have been progressively applied to attend the
40% 37%
2007 demand of increasing time and resources yield. As a result, the
30% 2012 tools and methodologies have been evolving to take the best of
20% all available techniques, and the semiconductor industry and
EDA vendors had been working to standardize the use of these
10%
techniques and verification methodologies.
0%
Functional Coverage Code Coverage The UVM is the greatest example of a successful
Constrained Random Simulation Assertions methodology that integrates advanced techniques to attack the
Figure 2. Simulation-based Verification Techniques in 2007 and 2012. complexity of SoC verification. The advent of an open
standard also enabled the cooperative contribution to the
In addition to the previously mentioned techniques, the international community, which allowed small companies to
verification teams realized that a successful functional make use of most advanced verification methodologies.
verification also depends on methodology. The verification
elements, like bus drivers, monitors and transactors should be REFERENCES
developed in a form that allows code reuse and standardization.
[1] B. Wile, J. C. Goss and W. Roesner, “Verification in the Chip Design
In the last decades, many verification methodologies has Process” in “Comprehensive Functional Verification: The Complete
emerged as international standards (Accellera UVM, Mentor Industry Cycle” San Francisco, CA: Elsevier Inc, 2005, pp. 5-31.
AVM, Synopsys VMM, Cadence eRM, etc.) and describing [2] B. Wile, J. C. Goss and W. Roesner, “Introduction to Formal
each one in this paper could be very extensive. However one Verification” in “Comprehensive Functional Verification: The Complete
verification Methodology have been emerging considerably in Industry Cycle” San Francisco, CA: Elsevier Inc, 2005, pp 439-486
last years, as pictured in figure below: Accellera Universal [3] M. Mintz and R. Ekendahl. “Hardware Verification With SystemVerilog:
Verification Methodology (UVM). The picture shows that An Object Oriented Framework,” New York, NY: Springer, 2007.
adoption of UVM by study participants grew almost 600% in [4] J. Bergeron, C. Eduard, A. Hunter and A. Nightingale, “Verification
two years (Fig. 3). Methodology Manual for SystemVerilog,” New York, NY: Synopsys Inc
and ARM Limited, printed by Springer, 2006.
[5] A. Molina and O. Cadenas, “Functional Verification: Approaches and
45% 41% 42% 41% Challenges,” in Latin American Applied Research Journal, vol 37 pp.
40% 65-69, 2007.
34% [6] HAL User Guide, Product Version 14.1, Cadence Design Systems Inc.,
35% 32%
San Jose, CA, 2014, pp. 14-23
30% 27%
[7] H. Foster, “2012 Functional Verification Study,” Wilson Research Group
25% and Menthor Graphics Corp., Bristol, UK, 2013.
20%
20% 2010
14% [8] E. Cerny et al., “The Power of Assertions in SystemVerilog”, New York:
15% 2012 Springer, 2010.
11%
10% 7% [9] Universal Verification Methodology (UVM) 1.1 User’s Guide, Accellera
5% Systems Initiative, 2011.
[10] M. Gschwind et al., “FPGA prototyping of a RISC processor core for
0%
embedded applications,” IEEE Trans. Very Large Scale Integr. (VLSI)
OVM Cadence eRM Syst., vol. 9, no. 2, pp. 241-250, Apr, 2001.
Accellera UVM Synopsis VMM Others
[11] A. Jain et al., “Accelerating System Verilog Uvm Based Vip to Improve
Figure 3. Standard Methodologies adoption in 2010 and 2012.
Methodology for Verification of Image Signal Processing Designs Using
HW Emulator,” International Journal of VLSI design & Communication
The UVM is a standard methodology that provides an Systems (VLSICS), vol. 4, no. 6, Dec. 2013
open-source framework which integrates the previously [12] E. Glocker et al., “Emulated ASIC Power and Temperature Monitor
techniques and many others like Transaction-Level Modeling System for FPGA Prototyping of an Invasive MPSoC Computing
(TLM), Object-Oriented Programming (OOP), verification Architecture,” Workshop on Resource Awareness and Adaptivity in
components reuse, etc. Due to the abundance of features in Multi-Core Computing, Paderborn, Germany, May 2014.
UVM, its complexity and learning curve increased
substantially. A detailed explanation of UVM resources can be
found in [9].