Accelerating SOC Verification

using Process Automation and Integration

Seonghee Yim, Hanna Jang, Sunchang Choi, Seonil Brian Choi
Samsung Electronics Co., LTD.

1
 Index
• Motivation: How to fill the Gap?
• Related works
– Diverse verification environment and workflow
– SOC development &verification flow
• Proposed system
• System functional components
• Automation system
• Experimental results
• Conclusion

2
 How to Fill the Gap?

• Verification gap increase between the amount of desired verification and that of performed
– Due to design complexity and simulation performance
3
 Diverse Verification Environments and Workflows

• 3 Phases of design and verification steps

– IP-level / Block-level / SoC-level ENV 1 ENV 2 ENV n

Workflow #1 Workflow #2 … Workflow #1

• Increasing complexity of design/verification Metadata #1 Metadata #2 Metadata #1

Designers/
– Increase of verification units Verif. engineers /
Projects /
– Increase of segmentation of design/verification Teams
ENV 1-1 ENV 2-1 ENV n-1
organization (engineers, projects and teams)
Workflow #1 Workflow #2 … Workflow #1
– Increase of in/out data set according to units
Metadata #1 Metadata #2 Metadata #1

• Diverse verification envs. and workflows

– Increase of combinations according to
verification units, in/out data sets and phases [Diverse Project Environments]

4
 SOC Development &Verification Level
 Verification Level (VL)

• VL1: IP-level design/verification is performed

• VL2
– IP-level design/verification and Block-level design/verification are mainly performed
– SOC top-level design/verification starts
• VL3/VL4: Blocks and SoC is mainly performed

 In VL4, regression test for blocks and SoC is performed mainly 5

 Proposed System
• Hybrid integration method to merge existed verification environments and processes
– Top-down : existed method
• forcing current environments and processes into a unified one to be integrated
• Overhead of manual tasks for forcing
– Bottom-up :
• Creating a new interface to unify current various environments and processes
• Too various verification environments to create a unified interface
– Proposed hybrid method
• To increase reusability of the existed various environments and its processes
• Creating a new interface but reuse almost verification environments and processes
• Automation
– Maximize an efficiency of environment building for design/verification by automating manual tasks
– Enhance an environment quality by removing human errors

6
 Proposed integrated platform and queue
• Proposed integrated platform and internal queue for automation and verification
– Diverse and well-structured verification environment in each design stage (IP, block, SOC level)
– Standardized workspace structures
– Support all environments in the proposed system rather than formalize them

[Integrated Platform and Internal Queue for Automation and Verification] 7

 Proposed integrated platform and queue
1) Flow Controller
• Flow controller : unify and automate workflow that exists in each team and step of verification stage
– Similar with Continuous Integration (CI, Jenkins)
– Treats each workflow as a unit of work
• Workflow is consisted of a sequence of steps called as scenario
• Multiple workflows are dispatched to a job queue.

ID SID USER STATUS ENV BLOCKING UPDATE DE

35 1 penning pend Input1 0 aaa.v 1
36 1 hnjang pend Input2 0 2
37 1 space pend Input3 0 abc.tb 1
38 1 aaa pend Input4 0 ccc.sv 3
39 1 bbb pend input5 0 bbb.v;fff.sv 4
8
 Proposed integrated platform and queue
2) Regression manager, 3) Job scheduler
• Regression manager
– To manage regression test cases
– To control and leverage internally developed job queues and commercial regression tools
– A large amount of jobs provided by regression managers and information of their status and results
can be checked at a glance
• Job scheduler based on Scenario
– To schedule jobs based on a priority or to reserve the time to execute (reservation time)

[Job status managed through proposed automation system] 9

 Proposed integrated platform and queue
4) Triage Scenario (TS) module, 5) Data Manager
• Triage Scenario (TS) module
– To classify issues or failures based on data generated from Regression
– To narrow down a region or target range of possible issues dynamically
– To re-run if necessary to provide sufficient information for engineers for rapid debugging
– Density-based clustering technique
• Data Manager
– To collect all data while every job execution and to use them for further analysis
(all design changes, test results, logs and so on)

[Dashboard and Mail Service System]

10
 Scenario based Automation System
• Scenario
– Various user-workflow types
– Central Regression, Local Regression, RF board test

• Step
– Each component of the Scenario
– Unit operations of users

• Automation
– To automate manual tasks for integrating each
verification environments
– To reduce human errors

11
 Experimental Results (1/2)
• Experiment Condition
– w/ Flagship Mobile AP SOC
– # of regression : more than 10,000 (includes IP, Block, and SOC top scenarios)
• Comparison result of verification time
– A, B, C in Table 1 : verification time for each verification target (IP, block, SOC)
– T : unit time, A1 at the VL1 is defined as 4T

[Table 1. Verification time by each verification step]

Verification Level VL1 VL2 VL3 VL4
Change Events Before After Before After Before After Before After
Max(A1, B1) Max(A2, B2, C2)
IP level change A1 || A1_IP_fail A2+B2+C2 || A2_IP_fail
|| B1_Block_fail || B2_Block_fail
Max(B2, C2) Max(B3, C3) Max(B4,C4)
Block level change B1 B1 B2+C2 B3+C3 B4+C4
|| B2_Block_fail || B3_Block_fail || B4_Block_fail
SOC level change C2 C2 C3 C3 C4 C4
* AN = 4T, 6T, 0, 0 (N : number of verification level, T: unit time), BN = 16T, 20T, 24T, 20T, CN=0, 60T, 90T, 120T 12
 Experimental Results (2/2)
• Result of verification time reduction
– shows efficiency for the verification time before and after each verification step
– Verification time reduction : Up to 43% (more than 20%)

[Reduction of verification time by each verification step ]

13
 Conclusion
• Integration method for workflows and verification environments is proposed
• The key “re-use the existing workflows through the automation system” supports
– Rapid set-up and unifying scattered information of each verification level

– Optimization of verification environment using the unified information

• All information generated at each phase is linked and is used efficiently

• By applying the proposed system, verification time can be reduced and this reduced time has helped
improve the quality of verification

• Through additional research, plan to re-order the test list of error cases by clustering each block by
density-based clustering

14