See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/257947089

The State of Verification Reuse in VLSI and Architecture for Reusable Test Bench

Conference Paper · October 2007

CITATIONS READS
0 127

1 author:

Usha Mehta
Nirma University
42 PUBLICATIONS   131 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Usha Mehta on 03 June 2014.

The user has requested enhancement of the downloaded file.

Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007
The State of Verification Reuse in VLSI
and
Architecture for Reusable Test Bench
Usha S. Mehta
Assistant Professor (EC Dept.)
Institute of Technology,
Nirma University, Ahmedabad
ushasmehta2002@yahoo.co.in, usm_ec.it@nirmauni.ac.in
Abstract
It is well known that verification today constitutes about
70% to 80% of the total design efforts, thereby making
it the most expensive component in terms of cost and
time, in the entire design flow of any ASIC. A number of
academic and industrial research laboratories have
been carrying out research on verification reuse and
static and dynamic methodologies for verification reuse
of ASIC and SoC based on different approaches. In this
paper the main area covered is the verification reuse .
Many of the issues related to intrinsic limitations of
these approaches taken. Today’s scenario in market has
brought to focus the need to carry out design and
verification concurrently. For the design and
verification task to proceed concurrently there is a need
to understand concepts of preparation for verification
before real Design under Verification (DUV) is ready to
be verified. As designs become more complex,
verification will have to be carried out using the divide
and conquer approach and it may require a different
verification approach for different block of design. The
aim of this paper is to provide information on the state
of the art in the area of reusable verification component
i.e Reusable TestBench(RTB) and architecture for
Reusable test Bench. The author concludes the review
by presenting issues, which form the current focus for
research. Most of the observations are author’s own
observations during her M. Tech. dissertation on
“Verification of Ethernet e Verification Components” at
eInfochips, Ahmedabad. So the architecture is
sometimes explained with the example of Ethernet DUV.
Index Terms—Verification Reuse, Reusable Test Bench
(RTB), System On Chip (SoC) , Design Under Verification
(DUV)
I. Introduction
When verification is a tough job for ASIC, it is
expected to get even worse for SoC designs because of
System on Chip (SoC) designs inherit all the well
known verification and validation difficulties associated
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore,
India
with complex ASIC designs, besides added by re-using
Intellectual Property (IP) cores. In any ASIC or
complex SoC design flow, verification is very
important; any behavioral or functional bug escaping
this phase will not be detected in the subsequent
implementation phases and will surface only after the
first silicon is integrated into the target system, resulting
in costly design and silicon iterations.
Verification of a design itself suggests that it should
be well checked for all possible valid inputs, its
performance during erroneous input conditions or
erroneous environmental conditions, all possible
implementation (device dependency) and all possible
future uses of the design.
For this purpose very natural method is to generate
all possible inputs, apply it to design under verification
and to check the response of design for these inputs.
This method, as inputs are required in this method, is
called dynamic verification method. The Functional
verification method is a dynamic verification method.
Dynamic verification deals with generation of input
stimuli, driving the stimuli to design under verification,
collect the output from design under verification and
validate the output for applied input.
Now as any design is required to be verified before
further processing as stand alone and later on as a part
of big SoC also, it is advisable to make test bench
reusable at various stage so that it can be used at
various levels of process flow. Reusability means that
the same input stimulus i.e. test cases can be used for
stand alone verification as well as a part of SoC
verification. The main requirement of such Reusable
Test Bench (RTB) is TRUST among users. To create a
trust on Reusable Test Bench and to be confident for
its result, it is must that this reusable test bench should
be containing all possible scenarios and corner cases for
stand alone verification as well as future post integration
verification. For assurance of this purpose this
verification component must be rigorously verified with
available standard or specifications of design, third
party models or golden reference of the design for
which it is built and checked that it has a provision to
detect all possible errors that can occur in design.
62
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007
II. Reusable Test Bench
Improving verification productivity is an economic
necessity. When the design company people try to
verify each of the design modules from its big SoC
design themselves, they are required to create test bench
for each particular module. This is a very lengthy
process itself. So if for some standard modules like
processors, protocol or architecture which follows some
international standards like IEEE, if the ready made test
bench is available in market, such design companies
will be at very ease. Such ready made options of test
bench which can be also used at various steps of
verification process are called Reusable test Bench.
Verification reuse directly addresses higher
productivity, increased chip quality and overall
verification investment. Reusable Methodology is the
breakthrough technology required to create reusable
verification environments and to ensure that all
verification components effectively interoperate.
Today's complex chips commonly incorporate many
different protocols, interfaces and processors.
Assembling appropriate verification environments
requires efficient integration of reusable, plug-and-play
verification components. Achieving reusability requires
that all components be built and packaged uniformly.
Reusability becomes even more challenging when
design teams all over the world create verification
components that need to fit together seamlessly. Every
aspect of the component, including basic naming
conventions and coding styles, debug message
conventions, user interfaces, and interactions between
components must be standardized in order to assure
interoperatability. World leader companies in the
development phase face many verification challenges.
Verification reuse is essential for productivity and to
reduce time-to-market for these companies. Here
Verification Reuse suggests the ready made test bench
for a particular application like a protocol, processor or
so on. When we think of any ready made, plug and play
kind of test bench, it should contain some of the generic
features as listed below:
• Reusable Test bench should be ready-to-use, out-of-
the-box verification environment, typically focusing
on a specific processor, protocol or architecture, such
as Ethernet, PCI Express, AHB, PCI, or USB.
• It must consist of a complete set of elements for
stimulating, checking, and collecting coverage
information on device under test (DUT).
• This test bench expedites creation of a more efficient
test bench for DUT.
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore,
India
• It must be able to work with both Verilog and VHDL
designs and if possible with other languages also.
• It must be able to work with all HDL simulators.
• User should be able to use this test bench as a full
verification environment or add it to an existing
environment.
• The reusable test bench should be optimized for use in
a coverage-driven verification process by providing:
(1) Generation that randomizes to all possible behaviors
of the environment
(2) Generation that can be constrained to a narrower set
of behaviors or all the way to a directed test.
(3) Self-checking
(4) Coverage measurement for stimulus conditions,
checking triggers and device response.
• The reusable test bench behavior should comply with
the IEEE Standards or other specific standards that
industries generally follow for that particular
application. For example, a reusable test bench for
Ethernet protocol should be comply with the IEEE std
802.3, 2000, IEEE Draft P802.3ae/D4.0, RMII
Consortium, SGMII and SMII specifications by Cisco
systems-1998 and so on..
• Normally an RTB contains agents for different
possible configurations or extensions. Each agent
contains a signal map block—containing the RTB
signals mapped to the DUT, configuration block-that
has the signals for configuration of the agent, Bus
Function Model (BFM)- to drive and generate the
stimuli, a monitor to check the DUT behavior and
collect coverage information and to collect the output
data and then emits an event for each output data
received, collector- to check for the protocol
violation, checker – to automate the output response
checking process and a scoreboard to check the data
items integrity.
III Architecture of Reusable test Bench
The architecture of Reusable Test Bench is shown in
fig. 1, page no.3. It contains various components like
Active agent, passive agent, monitor, checker, score
board, bus function model and so on. The function of
each block is explained in detail.
63
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007

Fig. 1 Architecture of Reusable Test Bench

RTB & VE MGMT Sequences Item coverage

agent ve Agent Sequences
Config
&
Ethernet error logger Coverage Active

Ethernet RTB MGMT Sequence Item checker

management error Driver
Sequence Driver

VE monitor
Passive
BFM MGMT BFM
Monitor

BFM checker

DUT
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore, 64
India
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007
Role of Active Agents
The active agents drive traffic to the DUT with the help
of the sequence driver. The sequence driver generates
various sequences and these sequences produce input
stimuli for DUV. The stimuli (like Ethernet packets for
Ethernet DUV or random lists of data) are injected into
the DUT by the Bus Function Model (BFM). The BFM
injects them on the input port of DUV (like TX lines for
the MAC agent and Rx lines for the PHY agent).
• The active agents generate input stimuli packets
depending on the constraints provided by the user on
various item fields. Active agents also contain a
monitor to do the checking and collecting coverage.
The management sequence driver generates various
management sequences that are injected into the DUT by
the management BFM. The management BFM injects the
sequences on the MDIO (Management Data
Input/Output) line. The management sequences are
related to functions of loop back, power down mode,
isolation, control, and status operations.
Proceeding with the Ethernet example, the sequences
from the sequence driver and the management sequence
driver verify the DUT as follows:
The sequences from the sequence driver verify the DUT
for the following functions:
• MII like collision, false carrier indication, TX and Rx
errors.
• GMII like carrier extension, bursting, collision, false
carrier indication, TX and Rx errors.
• XGMII like starting error, data error, remote fault, and
local fault. RMII like Rx error, collision, and so on.
• SGMII, TBI like invalid code in data, special, and
undefined category.
• XAUI like deskew error.
• XSBI like block with invalid sync header and type
field.
The sequences from the management sequence driver
verify the DUT for functions of management interface
like loop back, power down, isolation, speed selection,
and so on.
Role of Passive Agents
The passive agents consist of:
* A monitor
* A scoreboard
The passive agent has both the TX and Rx collectors in
the monitor, by default. The TX monitor senses signals on
the TX path and the Rx collector senses signals on the Rx
path. The monitor collects the packets and emits events
on the status of traffic to and from the DUT. The monitor
contains predefined coverage definitions and you can
create additional coverage definitions and protocol checks
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore,
India
to meet your test bench requirements. The monitor also
contains predefined checks that verify the DUV’s
adherence to the specifications. The monitor in the
passive agent checks for the protocol violation on the TX
line and monitor in the passive agent checks for protocol
violation on the Rx line. The scoreboard unit verifies the
data integrity by comparing the input and output data.
Flow of Data within the Agents
The BFM initiates new stimuli for transmission by calling
the sequence (seq.) driver if transmission of previous
stimuli is over and other required conditions match. If an
stimuli needs to be transmitted, then the sequence driver
generates the required stimuli and passes it to the BFM. If
a management packet needs to be transmitted, then the
management sequence driver generates the required
management stimuli and passes it to the management
BFM. During reception cycle both the active and passive
agents collect list of bits, list of di-bits, list of nibbles or
list of bytes depending on the type of interface. The
agents then re-group the data in the collector and check
for related errors.
Monitor and BFM Architecture
The monitors of the RTB are completely passive. The
BFM drives and generates the data. The BFM can make
use of the monitor or duplicate some of the monitor's
logic.
Most passive activity is done by the monitor, while all
active interactions with the DUT are done by the BFM.
For example, the monitor collects the packets and then
emits an event for each data received.
Monitor Architecture.
1. TX Collector: The TX Collector collects
packets from TX data path. It is disabled
for ACTIVE TX agents and enabled for
PASSIVE agents by default.
2. RX Collector: The RX Collector collects
packets from RX data path. It is disabled
for ACTIVE RX agents and enabled for
PASSIVE agents by default.
The monitor has predefined checks to verify protocol
adherence of DUT and has predefined coverage definition
too.
IV Architecture for Verification
Environment of Reusable Test Bench
For verification of protocol based DUV, given
architecture of RTB can be used. For getting the
qualitative means for how well this RTB is working, it is
required to verify this RTB before use. It should be
carefully checked that whether it fulfills the requirement
of DUV verification or not. It should be also checked for
65
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007
any bug exists in it. It should be checked for whether its
functionality can be further extended and whether it fully
checks all the concepts of specification or standard for
given DUV . So it is required to build the mock
verification environment (VE), and configuring the RTB
for different kinds of possible DUVs in future. The goal
here is to make RTB fault (bug) free and to achieve 100%
coverage with reference to standards or specifications for
given design. The architecture of this VE is given in fig.
2.
First step for defining verification environment is to
decide its features. The features required for a good
verification environment is:
ƒ Item generation coverage and checking.
ƒ Coverage of all kinds of sequences.
ƒ Data integrity checks for the BFM.
ƒ Sequences to generate various complex
scenarios.
VE architecture
Based on above-mentioned features, we have prepared
VE architecture, which follows the guidelines from
Verisity and Specman Ellit. This topic discusses various
modules of VE.
Item checker
This module checks whether the item generated by the
sequence is without any contradiction or not. It also
checks that the sequence driver generate all kind of
sequence items.
Item Coverage
This module covers the item generated by the sequence.
Here VE coverage definition is different from RTB
coverage. The RTB coverage defines all the rules and
items design protocol supports. For VE coverage item list,
it should cover all the checkers that are introduced in the
RTB, scoreboard items and different possibilities of RTB
configuration in addition to items listed in RTB coverage
list.
While defining coverage items, some questions related to
specifications must be answered. Below are some sample
questions.
• Have all packet/transaction types been tried?
• Have all CPU opcodes and operand
combinations been tested?
• Have all legal state transitions occurred?
• Have all instruction types been interrupted?
• Have all cases of resource contention been
tested?
• Have all queue limits been stressed?
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore,
India
BFM checker
This unit checks whether the item fields (virtual fields)
constrained by the sequence driver; is driven as per
constrains through the BFM or not. Verification target:
Verify that the virtual fields of the item are driven
correctly from the BFM to the monitor.
VE Sequences
The VE sequences are built on the basic
sequences provided by RTB to generate
complex scenarios. The VE should have its own
sequence library to simulate various complex
scenarios so that checking of the checks can be
done through them.
Agent Configuration Coverage
This module covers the various fields of the agent config
struct.
Error Logger
This module is used to expect errors based on packet
injected by Ethernet BFM of agent. When DUT
completes its packet collection, occurred errors are copied
to this module. It compares both expected and occurred
errors and gives error on mismatch.
Management Error Logger
This module is used to expect errors based on
management packet injected by MGMT BFM of agent.
Rest of the functionality is same as Ethernet Error Logger
module.
VE Monitor
This unit checks the validity of the checkers by
comparing expected errors and actually occurred errors of
error logger unit. Whether a check is fired only when
there is an error condition violating the protocol; is
checked by the VE monitor.
Data Collected Correctly
Verification target: verify that the monitor collects items
correctly. It uses scoreboard to verify data integrity.
Checker Is Correct
Verification target: verify that the checks fired are
correct.
Generate the sequence to generate both erroneous/non-
erroneous behavior and check against expected behavior
with a separate checker.
Whenever a sequence is intended to fire checks from
RTB monitor, expected errors list is updated with the list
66
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007
of expected error tag names. The errors occurred during
the execution of sequence are logged into occurred errors
list. At the end of sequence both the lists are compared
and result is indicated. If both the lists match then a
message saying: “Expected and occurred errors are
matched” is displayed else a message saying: “Errors
expected but not occurred:” or “Errors occurred but
not expected” is displayed. Please refer the figure 3 and
4 with “ERROR” to a different directory with all the details
For checker, coverage buckets are defined for each of
them with their corresponding tag name. Whenever a
dut_error occurs, the bucket for that particular check is
filled.
Coverage is correct
Verify that coverage definitions are exercised at least
once and coverage is being collected correctly.
Configuration check / coverage
This module checks and covers for the configuration of
the RTB. It also checks that the RTB is configured as per
the user configuration.
It checks for the following.
ƒ Topologies
ƒ User configuration.
Scoreboard
The scoreboard is used to check the data integrity
between the BFM of one agent on one end and the
monitor of other agent on the other end.
Following is the list of scoreboard instances and
corresponding input items, which are to be compared.
V Implementation
The above architecture is successfully implemented for
Ethernet Verification using e Language and Specman
Elite during Author’s M. Tech. dissertation “Verification
of SoC using eVC”.
VI Automation
For all above tasks of verification like input
generation, simulation, checking, coverage analysis, and
so on, the automation plays a great roll in saving time,
efforts and money. The design must be checked for the
valid input combinations including the specific critical
inputs where errors are most prone. To check the
behavior of design under invalid input combinations, the
design should be checked with erroneous input stimuli
also. So automation techniques are employed to generate
both types of stimuli. When you have a hundreds of test
cases to apply to design in proper sequence in absence of
you, to collect the reports from simulator in prescribed
format, to analyze the report for any error, to recreate the
test case which had error in output, to generate the
Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore,
India
coverage report and so on, there are a number of tasks
which requires automation. Fig. 1 and 2 gives an example
that how this automation can be applied.
In case of normal input applied to design, the output
should be error free. So a matching automated mechanism
can be scripted which will check all the simulators report
for a string which contains a string “ERROR” or
something like that. The script will also store such reports
like test case number, date of execution, no. of errors in
file, line no of “ERROR” and so on.
In case of erroneous input applied to design, for each
predefined error in put, a list of expected output errors
must be created. So when output is generated, a script can
be made to list all the errors in output. Then this list must
be compared with the expected errors list and which ever
are common errors must be neglected for the analysis.
Now there will be two types of error which are “expected
but not occurred” and “occurred but not expected”. So
bug analysis is required for these two error types. Script
can help in all above tasks. So now a days, for a
verification engineer, to understand the operating system
very well and to automate the process in very efficient
way is one key to success.
VII. Conclusion
The paper focused on why a reusable test bench is
required and also why it is required to be verified before
putting it in to user’s hand. The architecture for RTB and
VE are explained for protocol checking by use of checker
as well as data integrity by use of scoreboard. The
coverage results can be used as a measure of quality for
RTB. As day by day, the world is moving with more and
more functions on smaller chip, the demand of RTB will
be definitely higher and to get revenue out of it and to
generate trust on it, this kind of comprehensive
verification of RTB will be must in coming days.
VIII. Acknowledgements
The author is thankful to CEO, eInfochips,
Ahmedabad for providing the facilities to work on such a
imerging topic.
IX. References
[1] Janic Bergeron, Writing Testbenches : Functional
Verification of HDL Models, ISBN – 1-4020-7401-8
[2] Guide lines for eVC and eRM from Verisity.
[3] Usha S Mehta : “Verification of SoC using Ethernet
eVC’, M. Tech. Project, 2004
[4] http:/ www. einfochips.com
67
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007

Expected Output (without error)

Input

No
match
Normal match
Design Under Matching
Behavior Mechanism
Verification

Real Bug
Output
Real Output (with or without error)

Figure 3 Comparison of real output with expected output in

case of normal input

Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore, 68
India
Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007

Input List of
With Expected
Directed Errors Error

Expected No
Design and match
Matching
Under occurred
Mechanism
Verification errors are
matching

Normal
Behavior

Output List of
Occurred
Errors

Expected but not Occurred but not

occurred error expected error

Bug

Figure 4 Comparison of real output with expected output in

case of input containing directed errors

Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore, 69
India
 Proceedings of the International Conference on Computer, Communication, Control & Instrumentation – 3CI-2007 – November 21st to 23rd
2007

Departments of Electronics & Communication Engineering and Instrumentation Technology R.V.College of Engineering, Bangalore, 70
India

View publication stats