SV VCS UVM Design Challenges

Tackling SystemVerilog and UVM Testbench Debug Challenges
Interactive Debug with Verdi and VCS
Eugene Ho
September 2018
1 © 2018 Synopsys, Inc. 1

Agenda
Motivation
Efficient SV/UVM Testbench Debug
Rapid Root-Cause Analysis with Reverse Debug
What-if Analysis to Improve Testbench Debug Turn-Around Time
© 2018 Synopsys, Inc. 2

Verification Complexity Exponentially Increasing
10x Billion+
1T Gates SoCs
• 200M+ gates mobile SoC
• 50+ IPs per SoC
• 100+ power domains
Logic States & Transitions

10x • 300+ clocks Functionality
• 30+ interface protocols Power
• 10M+ lines of SW 100G
Performance
10x • 10M+ gates ASIC
• 5-10 IPs per ASIC
• 10+ power domains 10G
• 10-15 clocks Block-level
• 1-2 interface protocols testing to
• 100K lines of SW SW Bring-up
• 1-5M gates ASIC 1G
• 1-2 clocks
• 10K lines of SW 100M
10M
Shrinking
Time-to-
1995 2000 2005 2010 2015 2020 Market
250nm 90nm 40nm 28nm 14nm 10/7/5nm Windows
Source: Synopsys

Debug - Expanding Scope & Exploding Complexity
Constraints Coverage
Software TestBench UPF
Embedded
RTL/Gate Analog
Processor
Bus Protocols VIP

Agenda
Motivation

Testbench Debug Complexity
Verification Approach Environment

Migration from v2k to SystemVerilog/UVM Languages & Methodologies
Expertise mismatch In-house/vendor VIP
Testbench Debug
Complexity
Scale Volume
Block to sub-system to system Amount & complexity of functionality to verify
Signal-level to Protocol-level Use of IPs in design

Testbench/UVM/SV Debug Challenges
• Unexpected values in the testbench

– Why did foo() return 5 and not 3?
• Problems in the UVM domain
– Was this class type overridden?
• Visualize testbench topology
– Did the test generate 4 or 5 agents?
• Constraints debug
– Why did this randomize() call fail?
• What if analysis
– What would happen if I reran this sequence with a different value?
– How would the design change if I forced this register to 0?
• Simulation is not running as expected
– Why is my simulation hanging? Why is my simulation terminating prematurely?

Interactive Testbench Debug Mode
Full-visibility Testbench Debug in Verdi & VCS
• On-demand Debug
• Step forward/back/into
Full Interactive Reverse Debug
Control
testbench constructs
Control
• Go back in time without
restarting simulation OR
setting checkpoints
Members & Source Code
Local Variables Debug
Stack Browser
• Full Visibility
• Source code, dynamic
objects, watch list etc.
• Call stack, members, local
variables etc.
Watch Views
Class Object
Browser Browser SmartLog Log • Powerful Capabilities
Viewer
• Trace value assignment
into testbench
• Perform what-if analysis
• Enable day-to-day and batch
simulation debug
Key Elements for Efficient SystemVerilog TB Debug
Class Browser
Class hierarchy, methods,
instances, member values
Object Browser Stack and Locals

All objects with member Call stack of thread(s)
values and UVM hierarchy Frame locals with values

Simulation Control & TB Behavior Tracking
Simulation Control Breakpoints
Source with Value

Annotation
SmartLog: Powerful
Interactive Console
Watch
Class instance
breakpoint

Making Debug UVM Aware
Factory Domains/
Resource DB Phases
Sequences Registers Hierarchy

Agenda
Motivation

Improved Productivity with Reverse Debug
Restart
Rerun Simulation
Simulation
Simulation Simulation
Time 0 Time 0
Breakpoint Stepped beyond Done

line of Interest
Basic Interactive Debug
Done
Simulation
Time 0
NO
Restart Saves Hours of Simulation
Needed
Go Back to ANY
Point in Time Step back
w/ Reverse Debug
Finding Failures
Diagnose and Debug - Moving forward and backward in time
Done
Simulation
Time 0 Test FAIL
Advance forward to later time

without re-simulation
Jump back to Step forward through
previous point in time code to find issue
Undo/Redo Go To Reverse Controls

Last simulation control command Previous/Next Run - Next – Step – Step Out – Step in
Forward & Backward Value Assignment Thread – Next in Thread – Step in TB
Reverse Debug Use Case - Stepped Over Code
• Stepped over a function but needed to step inside?

• Restart the simulation and run there again?
Stepping forward through the

simulation…
Use reverse debug to go back and step into the function:

Go to Previous/Next Value Assignment
Active Driver for SVTB
• Example :
Why does the class member m_sequence_id have the value 98?
Sets the context to the value assignment of the member with full visibility and navigation
(source view, call stack, locals, …)

What-if Analysis
Fix and Validate in One Simulation
Simulation
Time 0 Test FAIL
Step forward through

Jump back to code to find issue
previous point in time No need to recompile
Force signal – Expect new outcome

No need to re-run
simulation
Simulate forwards
Done
New outcome, test does

not stop/fail here anymore Test PASS
Repeat if needed

Agenda
Motivation

Debug Scenario using Reverse What-If Analysis
• Scenario of no activity on interface signal
– Expected transitions on both sig_request[0] and sig_request[1] signals
– No activity on the sig_request[1] signal, why?
No Activity on
sig_request[1]

• After some analysis it is suspect some code in a UVM sequence is the problem
• Set a breakpoint on the suspected line of code
Set Breakpoint

• Set a conditional breakpoint to break on the suspected class object using object IDs
Set Breakpoint

• Use Reverse Debug to run back to the conditional breakpoint
• The waveforms will show the full activity of the simulation
Reverse Debug
Breakpoint Hit
Variable itr is
the incorrect
value of 0

• Within Interactive Debug change the value of itr signal using What-If Analysis
• No need to close down Verdi, re-compile, make source code change, and re-simulate to see effect
Right Click 
Change Value

• After applying new value of 20 for itr it is shown in the member pane
• The waveform are at the time of the breakpoint due to future waveform activity is removed
Run simulation
forward based on
itr=20
Variable itr is now

a value of 20
Waveform at time 0
showing no
transitions - yet

• After running the simulation forward, activity is now on both sig_request[0] and sig_request[1]
• What-If Analysis confirm this is a testbench problem with the itr variable within the UVM sequence
Activity on both
sig_request[0]/[1]

• The itr variable is not correctly retrieving the expected value for the UVM configuration database
• This can be further debugged using the UVM resource_db debug view
UVM Debug
Reverse Debug
uvm_config_db call
to retrieve/get the itr
var for configDB
Incorrect
uvm_config_db
set/get command.
This is the UVM/TB
bug!!

Summary
• Interactive Debug
– Full-featured, full visibility debug for
SV/UVM testbench
– UVM-aware debug
– Constraint debug
• Reverse Debug
– Run forward and backwards for
improved debug efficiency
• UVM-Aware Debug
– Phase Debug
– Sequence Debug/ Transaction Debug
– Resource DB Debug
• What-If Analysis
– No need to restart a interactive debug
session
– Test cause & effect changes during
interactive debug session

Thank You
Enhance Verification management methodology
by Verdi Planner
KienPham
Renesas Design Vietnam
September 21, 2018

SNUG Southeast Asia – Singapore
SNUG 2018 31
INTRODUCTION
 Full Name: Pham Trung Kien

 Company : Renesas Design Vietnam Co.,Ltd.
 Title : Project Manager
 History:
- 2008 ~ 2013 : SoC Mobile
- 2013 ~ 2018 : RH850 MCU
SNUG 2018 32
Agenda
Overview of verification management

Verification management by VerdiPlanner
Achievement
SNUG 2018 33
Overview of verification management
 Verification management challenging

 VerdiPlanner introduction
SNUG 2018 34
Verification management challenging
Total verification item by projects Verification types
System Usecase
verification verification
+45%
+15% Static
Unit
connection
verification
verification
Random verification …..
 Verification item volume increase drastically, how to manage without any missing ?
 Reporting time, specially in regression for 30K items, is feasible for human work ?
 Verification type become more diversity, how to merge them in verification report ?
SNUG 2018 35
Verification management challenging
Customer
requirement “Mission to find a solution to
1 succeed the verification
Defining Target Defining test item management in large scale
specification list
and complicated design”
Creating test
Creating RTL code
Pattern
2
Defining implement Creating With condition :
specification simulation
environment
 Minimize change of design flow.
3  Not broken simulation environment.
Synthesize RTL Executing  Cost investment is acceptable.
to Gate netlist simulation
4
Next ME/BE process Making verification
Result report
SNUG 2018 36
VerdiPlanner introduction
Hierarchy Verification Plan file

Verification plan in Excel or xml format
Automatically generated
.hvp from the verification plan
(no editing required)
Simulation
execution
result
Synopsys
coverage Verification End user
database planner application
Merging & matching
Log file Generated verification

VerdiPlanner
PASS/FAIL result report
judgement in text
Verdi Planner: A tool that provides functional verification planning and collecting the associated verification execution results
SNUG 2018 37
Verification management by VerdiPlanner
 VerdiPlanner adoption flow

 Simulation result automation management
SNUG 2018 38
VerdiPlanner adoption flow
Customer
requirement
1 Add some keyword to current

Defining Target Defining test item template, convert from Excel to *xml
specification list format. Just need to do one time.
Creating test
Creating RTL code
Pattern
2
Defining implement Creating
specification simulation Merge test pattern
environment
simulation log file, make
3 Pass/Fail judgement
Synthesize RTL Executing
to Gate netlist simulation
4
Next ME/BE process Making verification
Execute VerdiPlanner at
Result report post simulation
SNUG 2018 39
 Convert verification plan from original template to unified format

In each area, a unified keyword will be added
Existing verification plan
Defining test item
list
1 2 3
Creating test
Pattern
Creating
simulation Verification item Pattern Judgement
environment content measurement area
Updated verification plan
Executing
simulation
Making verification
Result report
SNUG 2018 40
 Create verification plan hierarchy. One Chip hierarchy is possible.

Create an upper-level verification plan by including two or more
verification plans (XML) for each module.
Defining test item
list
Creating test
Pattern
Creating
simulation Top verification plan
environment
Lower hierarchy verification plan
Executing
simulation
Making verification
Result report
SNUG 2018 41
 Execution simulation, Making pass/fail determinations and export to text file

(1) Simulation log file after execution, the judgement template depend on specific
environment
Defining test item (2) Simple & unified judgement (Pass/Fail) is translated, it will be input to Verdi Planner.
list
“fail”
Creating test
Pattern
“pass”
Creating
simulation
environment
Conversion Determinations keyword by testbench:
utility “Error”, “Normal End”
Executing
simulation sim_check.txt
Making verification
Result report
SNUG 2018 42
Defining test item

list
 Examining the results & generate verification result report

Creating test Annotate the verification execution results in the Hierarchical
Pattern
verification plan with URG Collaboration using Synopsys VCS-MX.
The annotation results are created as .ann.xml.
Creating
simulation
environment source /common/appl/Env/Synopsys/vcs-mx_vL-2016.06-
SP2:/common/appl/Env/Springsoft/verdi3_2016.06-SP2
Executing bs –os RHEL6
simulation hvp annotate -plan ./top_plan.xml
-dir ../sim/simv
-userdata sim_check.txt
VerdiPlanner -runurg -report ./urgReport
Making verification
Result report
SNUG 2018 43
Simulation result automation management
After Verdi Planner execution, verification result will be filled automatically from text file Pass/Fail judgement.
Hierarchical verification plan report & hyperlink permit easy tracking/detect Pass/Fail result by a click.
TOP plan Summary report
Module report
Urg Coverage
report
SNUG 2018 44
Achievement
 ~10x reporting time reduction

 Verification missing findings when applying VerdiPlanner
 Automatically collect, check & export the verification result
 Integrate visual verification progress from VerdiPlanner database
SNUG 2018 45
Achievement
~10x reporting time reduction
Verification result report update (man-hour)

14
12
10
0
Block 1 Block 2 Block 3 Block 4 Block 5
Manual VerdiPlanner
SNUG 2018 46
Achievement
Verification missing findings when applying VerdiPlanner
Verification item missing cases
OK
OK
Missing
Missing detection result

Missing verification item “Zero verification item missing after
35
applying VerdiPlanner”
30
25
20
15
10
0
Block 1 Block 2 Block 3 Block 4
Block 1 Block 2 Block 3 Block 4

Missing verification item 14 33 27 18
SNUG 2018 47
Achievement
Automatically collect , check & export the verification result
SNUG 2018 48
Achievement
Integrate visualize verification progress from VerdiPlanner database
6/7/2018 6/8/2018 6/9/2018 6/10/2018 6/11/2018 6/12/2018 6/13/2018 6/14/2018 6/15/2018 6/16/2018 6/17/2018 6/18/2018
TOP_BLOCK T o t a l i t e m 1619 1619 1619 1619 1619 1619 1,619 1,619 1,619 1,619 1,619 1,619
Completion item 389 389 389 389 389 632 1,000 1,021 1,021 1,100 1,100 1,350
Remained item 1230 1230 1230 1230 1230 987 619 598 598 519 519 269
Digestibility 24.03% 24.03% 24.03% 24.03% 24.03% 39.04% 61.77% 63.06% 63.06% 67.94% 67.94% 83.38%
SNUG 2018 49
Conclusion
 Could be used as flexibility “add on” of environment, no need to update

design flow.
 Automatic aggregation of verification execution results, avoid human
mistake, simplify work flow, reduce reporting time overhead.
 Unify verification plan and verification result report.
 “Real time” observe simulation progress.
 Be able to display functional coverage and code (structure) coverage in one
report.
SNUG 2018 50
Thank You
SNUG 2018 51
Additional Slides
Debugging Constraints

Current Challenges in Constraint Debug
Constraints
Constraint to describe
complexity valid
grows Unfamiliar
Extensivecode is to
reuse difficult to
expedite
with design stimulus debug development
Input Stimulus (Generators & Undesirable solution

Coverage to see what has
Drivers) distribution increases
been generated
verification cost
Behavior Coverage
Checker Observer
Design Under Test (DUT)
Iterative compile and run times impact turn-around time

Constraint inheritance example
Solver Partitions GUI perspective
How VCS handle SV constraints
• Partitioning
– Simpler to solve many small pieces than one large puzzle
– Unrelated variables are places into separate partitions
– No interaction between partitions
– May have hundreds of unrelated partitions
– Related variables are solve in a single partition
– Become state variables in later partitions
Solving Partiton 1
class packet; rand bit[7:0] a; // rand_mode = ON
...
rand bit [7:0] a;
a = 10; Partitioning
rand bit [7:0] b;
rand bit [7:0] c; Solving Partition #2
constraint cst { b < c; } rand bit[7:0] b; // rand_mode = ON
endclass rand bit[7:0] c; // rand_mode = ON
...
Small class example constraint cst {b < c;}
...
b = 20;
© 2018 Synopsys, Inc. 54 c = 100;
Debug Starts Here
• Find the randomize call to debug first • Run until breakpoint
Simulation will stop before the

class object is randomized
• Step into the constraint debugger
Equivalent UCLI command
Equivalent UCLI command

Debug Starts Here
• Locate the key random variables of interest
Source code
Examine the partitions

and constraints
Cross probe into the source
Constraint Debugger

Interactive Constraint Editing
Enabling what-if analysis without re-compiling
• Change variable value

• Modify rand_mode
• Modify constraint_mode
• Enable/Disable existing constraints
• Add new constraints
Interactive constraint
commands
Example of a
rt constraint (runtime)
No Re-Compile!
On-the-fly Re-randomization
Immediate feedback on cause-effect relationship
No Restart
Turning OFF
“src_range_cst”
constraint block fixes
the failure
Original Re-Randomize
Randomize

Solution Distribution Debug
Early assessment of stimulus quality starts here!

References
• Synopsys Verification Home

https://www.synopsys.com/verification.html
• Synopsys Verdi Debug Platform

https://www.synopsys.com/verification/debug.html

SV VCS UVM Design Challenges

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SV VCS UVM Design Challenges

Uploaded by

Copyright:

Available Formats

Tackling SystemVerilog and UVM Testbench Debug Challenges

Interactive Debug with Verdi and VCS

1 © 2018 Synopsys, Inc. 1

© 2018 Synopsys, Inc. 2

Logic States & Transitions

© 2018 Synopsys, Inc. 3

Software TestBench UPF

Bus Protocols VIP

© 2018 Synopsys, Inc. 4

© 2018 Synopsys, Inc. 5

Verification Approach Environment

© 2018 Synopsys, Inc. 6

• Unexpected values in the testbench

© 2018 Synopsys, Inc. 7

Object Browser Stack and Locals

© 2018 Synopsys, Inc. 9

Simulation Control Breakpoints

Source with Value

© 2018 Synopsys, Inc. 10

Sequences Registers Hierarchy

© 2018 Synopsys, Inc. 11

© 2018 Synopsys, Inc. 12

Breakpoint Stepped beyond Done

Basic Interactive Debug

Advance forward to later time

Undo/Redo Go To Reverse Controls

• Stepped over a function but needed to step inside?

Stepping forward through the

Use reverse debug to go back and step into the function:

© 2018 Synopsys, Inc. 15

© 2018 Synopsys, Inc. 16

Step forward through

Force signal – Expect new outcome

New outcome, test does

© 2018 Synopsys, Inc. 17

© 2018 Synopsys, Inc. 20

© 2018 Synopsys, Inc. 21

© 2018 Synopsys, Inc. 22

© 2018 Synopsys, Inc. 23

© 2018 Synopsys, Inc. 24

© 2018 Synopsys, Inc. 25

Variable itr is now

© 2018 Synopsys, Inc. 26

© 2018 Synopsys, Inc. 27

© 2018 Synopsys, Inc. 28

© 2018 Synopsys, Inc. 29

September 21, 2018

 Full Name: Pham Trung Kien

Overview of verification management

 Verification management challenging

Total verification item by projects Verification types

Random verification …..

Hierarchy Verification Plan file

Merging & matching

Log file Generated verification

 VerdiPlanner adoption flow

1 Add some keyword to current

 Convert verification plan from original template to unified format

 Create verification plan hierarchy. One Chip hierarchy is possible.

 Execution simulation, Making pass/fail determinations and export to text file

Defining test item

 Examining the results & generate verification result report

 ~10x reporting time reduction