Verification of Digital Systems, Spring 2017

11. Testbench Basics 1

Verification Testbench

February 23rd 2017

Nagesh Loke, ARM Cortex-A class CPU Verification Lead
Jacob Abraham, Rajesh Ganesan, Kshitiz Gupta, UT Austin

What to expect?

This lecture aims to:

provide an idea of what a testbench is

help develop an understanding of the various components of a testbench

build an appreciation of the complexity in a testbench

highlight why it is as much a software problem as is a hardware problem

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 2

What is a testbench?

A testbench helps build an environment to test and verify a design

The key components of a testbench are:

Stimulus

Checker

Stimulus

is used to drive inputs of the design to generate a high-level of confidence

should be able to exercise all normal input scenarios and a good portion of critical combinations
with ease

A checker

is a parallel & independent implementation of the specification

is used verify the design output against the modeled output

What are we verifying? opcode[2:0]

An ALU has

an input clock
A[7:0]

two 8-bit inputs as operands

a 3-bit opcode as an operator

a 16-bit output
OUT [15:0]

Performs the following operations:

ADD, SUB, AND, OR, XOR, MUL, XNOR

B[7:0]

clk

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 3

What did they do?

What are some of the

issues with this approach?

What should the approach be?

A bit of planning goes a long way

A Verification plan talks about:

various design features and scenarios that need to be tested

architecture of the testbench

reuse in higher level testbenches

Testbench should have the ability to:

test as many input data and opcode combinations as possible

test different orders of opcodes

stress key features/combinations

use more machine time and less human time

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 4

SystemVerilog

SystemVerilog as a hardware verification language provides a rich set of features

Data Types & Aggregate data types

Class, Event, Enum, Cast, Parameterization, Arrays, Associative arrays, Queues and manipulating methods

OOP functionality

Classes, Inheritance, Encapsulation, Polymorphism, memory management

Processes

fork-join control, wait statements

Clocking blocks

Interprocess synchronization & communication

Semaphores, Mailboxes, named events

Assertions

Functional Coverage

Virtual Interfaces

Constraints

Data Types

SystemVerilog introduces several new data types.

Verilog variables are 4 state variables (0, 1, X, Z) wire, reg.

SystemVerilog has both 4 state and 2 state variables (0, 1).

Testbench can use 2 state variables for loop iterators etc.

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 5

Integer Data Types

2 state data types

bit (user defined size)
bit [7:0] addr;

byte (8b, signed)
byte addr;

int (32b, signed)
int addr;

4 state data types

reg (user defined size)
reg [7:0] addr;

logic (user defined size)
logic [7:0] addr;

logic type can be used instead of reg or wire types.

Non Integer Data Types

time - 64b, unsigned

real - similar to double in C

10

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 6

Misc. Data Types

Arrays can have packed and unpacked dimensions

packed dimension is specified before the variable name

unpacked dimension is specified after the variable name

the following declaration is for a 4 entry register file where each entry is 8b wide

bit [7:0] register_file [3:0];

11

Misc. Data Types

enum define enumerated types where variables can take any value from a set of all
possible values

enum { ADD=0, SUB=1, MUL=2, AND=4, OR=5, XOR=6, XNOR=7 } OPTYPE;

struct define structures containing multiple variables of same or different types

they can be packed or unpacked like arrays

struct { bit [9:0] src_addr; bit [1023:0] data; bit [3:0] crc} packet;

union define a variable that can be accessed as many different types (byte, int, etc.)

union {int a; byte b;} var;

string define a string type variable that can hold a sequence of characters

typedef add user defined types from the basic types

12

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 7

Misc. Data Types

dynamic arrays the size of the array varies at run time based on need or behaviour.

to allocate storage new and delete methods can be used

dynamic arrays are declared as shown below

int dynamic_array[];

13

Misc. Data Types

associative arrays a set of key:value pairs usually used for sparse data or when the size
of the array is not known

they can be declared as shown below

int associative_array[int];

Methods that can be used on associative arrays are:

first()

last()

next()

prev()

exists()

num()

delete()

14

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 8

Misc. Data Types

queue a variable-size, ordered collection of homogeneous elements

Can be used to model a LIFO or FIFO

Elements can be inserted and deleted using an index even if they are not the first or the last one.

int collection[$] = {10, 20, 30, 40}

Methods associated with queue data types are:

size()

insert()

delete()

push_front()

push_back()

pop_front()

pop_back()

15

Interfaces

Interfaces in SystemVerilog are a neat way to denote a bundle of wires.

An interface can be used as a module port instead of individual signals. It improves readability of the
code and makes updates to the code easier.

It can contain parameters, constants, variables, functions and tasks etc. in itself.

interface dut_in;
logic [31:0] A, B;
logic clk, rst;
endinterface

module dut(dut_in _in);

//one can use _in.A, _in.B in this module

endmodule
16

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 9

Interfaces

CLK RST

Master Intf Slave

CLK_I CLK_I
RST_I RST_I
ADR_O ADR_I
DAT_I DAT_O

Master Modport
DAT_O DAT_I

Slave Modport
WE_O WE_I
SEL_O SEL_I
STB_O STB_I
ACK_I ACK_O
CYC_I CYC_O

17

Interfaces
interface WB (input CLK, input RST);
logic [31:0] ADR, RDAT, WDAT;
logic [15:0] SEL;
logic WE, STB, ACK, CYC;

modport Master (output ADR, WDAT, WE, SEL, STB, input RDAT, ACK, CYC);
modport Slave (input ADR, WDAT, WE, SEL, STB, output RDAT, ACK, CYC);

endinterface

18

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 10

Virtual Interfaces
Testbench Design

Driver
Virtual IF

IF DUT

19

Object Oriented Programming

Procedural Programming Paradigm

A program is broken down into data variables and procedures (functions) that operate on that data.

A program itself is a sequence of procedure calls, where the main program hands over data to the
procedures.

Object Oriented Programming Paradigm

A program is broken into classes (data and methods) and objects (which are instances of classes).

Objects operate on their own data.

A program is a set of messages to objects.

Why Object Oriented Programming Paradigm?

Modular

Has Reusable Components

Easy to maintain and enhance

Based on interaction between objects

20

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 11

Object Oriented Programming Constructs

An object may contain data (data fields - strings, integers, etc.) and methods (member
functions) that operate on the data.

Encapsulation - The concept of keeping the data and the methods/functions that work
on that data in one structure. This also allows to keep the data safe, as the only allowed
ways that the data can be modified is usually through the methods in that class.

Objects are created as instances of a class. Class is a user-defined type in OOPS.

One of the most important features of classes (and OOPS in general) is inheritance. It
allows for new classes to be created by inheriting properties from previously defined
classes. The new class is called the derived class while the old class is referred to as
the base class.

21

Object Oriented Programming Constructs

Class Shape:
Data area(int), perimeter(int), color(string)
Methods getarea(), getperimeter(), getcolor()

Class Triangle(inherited from Shape): Class Quadilateral(inherited from Shape):

Inherited Data area(int), perimeter(int), color(string) Inherited Data area(int), perimeter(int), color(string)
Added Data length of the 3 sides Added Data length of the 4 sides
Methods getarea(), getperimeter(), getcolor() Methods getarea(), getperimeter(), getcolor()

Class RightTriangle(inherited from Triangle): Class Rectangle(inherited from Quadilateral):

Add methods like getHypotenuse() Add methods like isSquare()

22

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 12

Object Oriented Programming Constructs

class packet; class eth_packet extends packet;

bit [9:0] src_addr;
bit [1023:0] payload;
bit [15:0] dst_addr;
bit [3:0] crc;
protected int packet_id;
virtual function void print_packet();
$display(packet: %d, packet_id);
endfunction
endclass : packet
protected int frame_no;
function void print_packet();
super.print_packet();
$display(frame: %d, frame_no);
endfunction
endclass : eth_packet

23

Object Oriented Programming Constructs

24

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 13

Object Oriented Programming Constructs

25

Object Oriented Programming Constructs

26

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 14

Randomization Constructs

Variables within a class can be declared as rand or randc to be generated randomly.

rand bit [7:0] data;

randc is used when a cyclic random variable is needed. It ensures that all unique
elements are exhausted before any repetition occurs.

randc bit [1:0] state;

cycle 1 {10 00 01 11 }

cycle 2 {10 11 00 01 }

cycle 3 {01 10 11 00 }

These variables will be randomized every time the randomize() function is called on an
object of the class. randomize() is a built-in function used for randomization.

To pre-process or post-process the randomized variables, pre_randomize() and
post_randomize() functions can be used.

27

Constraints

Constraints are necessary to direct the randomization towards interesting corner cases

Constraint blocks are defined using the keyword constraint

Constraint blocks can be specified using any of the below expressions

set membership

distribution

implication

if else

iterative

variable ordering

28

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 15

Constraints Set Membership

class packet;
rand bit[9:0] src_addr;
rand bit[1023:0] data;
rand bit[3:0]crc;
constraint src_addr_set_const;
endclass : packet

constraint packet::src_addr_set_const {
src_addr inside {[100:127], [1000:1023]};
}

29

Constraints Distribution

class packet;
rand bit[9:0] src_addr;
rand bit[1023:0] data;
rand bit[3:0]crc;
constraint src_addr_dist_const;
endclass : packet

constraint packet::src_addr_dist_const {
src_addr dist {
[100:127] :/ 50
[1000:1023] :/ 50
}
}

30

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 16

Constraints Implication

class packet;
rand bit[9:0] src_addr;
rand bit[1023:0] data;
rand bit[3:0]crc;
constraint src_addr_impl_const;
endclass : packet

constraint packet::src_addr_impl_const {
(src_addr == 10d100) (data !inside {0})
}

31

Constraints Variable Ordering

class packet;
rand bit[9:0] src_addr;
rand bit[1023:0] data;
rand bit[3:0]crc;
constraint src_addr_vo_const;
endclass : packet

constraint packet::src_addr_vo_const {
solve src_addr before data;
}

32

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 17

Constraints Iterative

class eth_packet;
rand byte data[10];
constraint each_elem;
endclass : eth_packet

constraint eth_packet::each_elem {
foreach(data[i])
data[i] > 32;
}

33

Process Control

SystemVerilog provides three fork constructs for process control

fork join

fork join_any

fork join_none

In each case, the only difference is how long does the parent process wait after the fork
and before continuing execution.

In the first case: fork join the parent process waits for the completion of all child
processes to complete before continuing execution

In the second case: fork join_any the parent process waits for atleast one process
to complete before continuing execution

In the last case: fork join_none the parent process doesnt wait for the child
processes to complete before continuing execution

34

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 18

Process Control

wait and disable constructs can be used to control individual threads of execution

wait is used when the parent process is outside the fork join_any or join_none
case, but the parent wants to wait for the children to finish before continuing execution.

disable is used when the parent process is outside the fork join_any or join_none
case, but the parent wants to kill the children before continuing execution.

35

Components of a testbench

The top testbench now looks different

It creates an interface handle

It instantiates the DUT

36

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 19

Testbench TOP

A program block is the main entry point

A bfm object and a scoreboard object are
created

All the components are started

A fork/join process ensures that they all
start in parallel

We exit the fork statement at 0 time

Simulation is stopped when $finish is called

Multiple initial blocks execute in parallel

37

Transaction class

The ALU transaction class:

Uses an enum type for optype

Uses rand to declare inputs that need
to be driven

Has a print utility that can be used with a
transaction handle/object

38

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 20

BFM/driver

The BFM/driver class:

Has a handle to a virtual interface

Declares a alu_trxn data type

Has a constructor

drive() task:
Does not end
Creates a new transaction
Randomizes the transaction
Passes the handle to drive_trxn() task

drive_trxn () task
consumes time
drives the input signals based on the values in the
trxn class
Uses clocking block and non-blocking assignments
Adheres to pin level timing of signals

39

Scoreboard

The Scoreboard:

Functionality is to continuously check the output
independent of the input stimulus

check() task:
Collects information from the interface and
populates the trxn class
Calls a compute_expected_out function

compute_expected_out() task
Implements the model of the deisign
Takes in the inputs and gets an expected output
Compares the actual output against the expected
output
Issues an error message if the comparison fails

40

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 21

How does the testbench look like?

alu_tb
clock Process control
generation logic logic

clk
A OUT
alu_bfm ALU
ALU
B

alu_sb

41

How do we know we are done?

With a random testbench it is difficult to know what scenarios have been exercised

Two techniques are typically used to get a measure of whats done

Code Coverage

No additional instrumentation is needed

Toggle, Statement, Expression, Branch coverage

Functional Coverage

Requires planning

Requires instrumenting code

SystemVerilog provide constructs to support functional coverage

Provides detailed reports on how frequently coverage was hit with the test sample

Coverage closure is an important aspect of verification quality

42

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 22

Coverage Point

Identify an integral variable or expression to be covered

bit clk, rst;

bit [3:0] counter_out;
always @ (posedge clock)

Coverage point is declared using the keyword coverpoint

Selectively disable coverage with an iff statement

Dont collect coverage when reset is active

43

Coverage Point

covlabel: coverpoint counter_out iff(!rst) {

bins min = {0};
bins low = {[1:3]};
bins med = {[4:7]};
bins high = {[8:14]};
bins max = {[15]};
}

Keyword bins is used to collect sampled values into various named bins

Simulator keeps a count for each bin

44

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 23

Covergroup

Keyword bins is used to collect sampled values into various named bins

Simulator keeps a count for each bin

covergroup cg_cnt@(posedge clk);

option.name = Counter Cover Group;
covlabel1: coverpoint counter_out iff(!rst) {
bins min = {0};

}
covlabel2: coverpoint {
}
endgroup : cg_cnt
45

Cross Coverage

Specified using the cross construct

covergroup cg_cnt@(posedge clk);
covlabel1: coverpoint counter_out iff(!rst) {
bins low = {[0:7]};
bins high = {[8:15]};
}
covlabel2: coverpoint updn iff(!rst) {
bins up = {0};
bins dn = {1};
}
covlabel1_x_covlabel2: cross covlabel1, covlabel2
endgroup : cg_cnt
46

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 24

Cross Coverage

Specified using the cross construct

covergroup cg_cnt@(posedge clk);

covlabel1: coverpoint counter_out iff(!rst) { }
covlabel2: coverpoint updn iff(!rst) { }
covlabel1_x_covlabel2: cross covlabel1, covlabel2
{
bins low_dn = binsof (covlabel1.low) && binsof (covlabel2.dn);
bins high_up = binsof (covlabel1.high) && binsof (covlabel2.up);
}
endgroup : cg_cnt

47

Transition Coverage

covlabel: coverpoint counter_out iff(!rst) {

bins one = {0 => 15};
bins two = {15 => 0};
bins three = {7 => 8 };
bins rest = default sequence;
}

48

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 25

What did we go over

Built a directed and random testbench

Discussed various components of a testbench

Modularized and built in complexity into a testbench for a reason

Demonstrated that verification and testbench development requires good planning and
software skills

49

Useful pointers

https://verificationacademy.com/

SV Unit YouTube Video

EDA Playground

http://testbench.in/

Search for SystemVerilog on YouTube

50

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 26

ARM Cortex A72 CPU

51

ARM CCN-512 SoC Framework

52

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017
 Verification of Digital Systems, Spring 2017
11. Testbench Basics 27

What are the challenges of verifying complex systems?

Typical processor development from scratch could be about 60 man years

Multiple parallel developments, multiple sites and it takes a crowd to verify a processor

The challenges are numerous especially when units are put together as a larger unit
or a whole processor and verified

Reuse of code becomes an absolute key to avoid duplication of work

Multiple times it is essential to integrate an external IP into your system

The IP can come with its own verification implementation

This requires rigorous planning, code structure, & lockstep development

Standardization becomes a key consideration

So how do we solve this?

53

Department of Electrical and Computer Engineering, The University of Texas at Austin

Loke, Abraham, Ganesan, Gupta, February 23, 2017