Analytic Solutions For

Semiconductor
May 2020

ni.com
 ni.com

Through the acquisition of OptimalPlus,

NI accelerates companies’ digital
transformation initiatives by coupling
NI leadership in automated test with
new advanced product analytics for
enterprises.

“We’re confident NI’s enterprise

software strategy unlocks the value
of test data by embracing digital
transformation and bringing it to the
analog world.”
Eric Starkloff
NI CEO AND PRESIDENT
 It’s A
Changed World
Technological innovation has
transformed our lives.

Products and devices are

more intelligent and connected.
These products rely on
thousands of electronic
components that must be
more reliable than ever before.

ni.com
 Car innovations and
90% new features are
driven by electronics1
AUDI SAYS

Warranty costs
22% related to electronics
and semiconductors2

Automotive Car recall increase

3x from 2014-2016 due 1 CAR FAILURE Reliable
Innovation
to electronics3 electronics
Reliability
Challenge is a must
EVERY HOUR4
90M Ignition switch failure
cars Failure to park
$25B Takata airbag recall

15x Drive per day4:

22.5 1.5hr traditional car vs.
hrs 22.5hr autonomous car

1 Automotive change drivers for the next Decade, EY, 2016

2 BMW - AEC Automotive electronics reliability workshop, 2017
ni.com 3 NHTSA Recall Data
4 Audi, DVCon Munich, 2017
 ni.com

SEMICONDUCTOR AUTOMOTIVE ELECTRONICS

2005 ~230
FOUNDED EMPLOYEES
Big data analytics Open innovation Product-Centric
with expertise in platform for edge approach taking
manufacturing deployment of real- I4.0 and IIoT to the
Analyzing huge
time analytics and next level
volumes of data
AI/ML

100bn
DEVICES PER YEAR
Ready-made Lifecycle visibility Cloud Or On-Prem
solutions for across supply chains AWS (partner),
Automotive and and industries Azure, GCP
ROI track record and Semiconductor
loyal trusted partner industries
 Trusted By Leading Brands
Customers

Supply Chain

ni.com
 Challenges We Address In The Semi Industry

1 Growing chip
complexity, including
3 Efficiently manage
fragmented supply
5 Continuous pressure
on profitability
advanced materials, chains – suppliers,
processes, and sites, equipment,
packages systems

2 Quality requirements
get more demanding
4 Need to improve time
to market of new
6 Where and how to
use AI/ML to maintain
products competitiveness

ni.com
 Our Vision
Lifecycle Analytics Through Product-Centric Approach
DESIGN AND ENG. MANUFACTURING CUSTOMER
DESIGN CHIP BOARD MODULE PRODUCT IN USE RETURNS

Design Spec • Machine • Process • Metrology • Test • Rework • Genealogy • Performance • Reliability • Usage • Warranty

ni.com
 System Architecture
Edge (Factory Floors) Central
Tester Tester Tester
Other Equipment
Fab Assy. A B C
And
Data Data O+ O+ O+ MES PLM, ERP, CRM Portal+ www Rules+
Proxy Action Proxy Proxy Data Sources

Action

Action

Action

Action
Data

Data

Data

Data

Data

Action
Rules
Data

Data
Data
Data O+ Edge Analytics O+ Central Analytics
Action 24x7 rule execution and orchestration 24x7 rule execution Cloud
Control
Room+ Data and orchestration Or
O+ Data Platform On-Prem.
Factory A Action/Rules
(internal or outsourced) Edge repository O+ Data Platform
Central repository
Factory B
Factory C

ni.com Actionable Insights Across All Manufacturing And Test Processes

 Semiconductor Solutions

Quality,
Yield Analysis Supplier
Reliability and Efficiency Time To Market
and Reclamation Transparency
Brand Protection

AI/ML Deployment For Manufacturing

Data Security

Data Lifecycle

ni.com
 Providing Innovative Solutions

• Collect lots of data

• Use it primarily when there is a problem: Bad Yield, RMA (Returns), Etc.
Typical • Find the problem but frequently not the root cause
Methods
• Process is often manual and reactive, not proactive
• Use of many tools, but not an integrated solution

Collect Detect Act

• Lifecycle data • Prescriptive analytics • Automatic
harmonization of any type
• AI / Machine Learning • Distributed
O+ • Product, machine
Solution • 24x7 analytics engine • Controlled
and process data
• Data security • Real-time

ni.com
A unique, automated and proactive integrated solution
 The Value We Bring
Comply with
Quality, Reliability and Minimize Minimize Analyze Protect your
automotive
Brand Protection excursions RMAs root cause brand
standards

Yield Analysis Improve Minimize site-to- Optimize Identify equipment

and Reclamation overall yield site variations re-test policy performance issues

Enable consistent Avoid excessive Identify Ensure efficient

Efficiency tester availability index and pause test time variations retest policies and
and utilization times per tester execution

Optimize balance Facilitate multi- Share learnings

Time To Market Shorten NPI time between time, team from NPI to HVM
cost, and quality collaboration and back

Benchmark Ensure supplier compliance

Supplier Transparency suppliers with flows for every chip

ni.com
 Proxy+

1 Real-Time
Data Collection 2 Real-Time
Control 3 Platform For
Real-Time Action
• Runs on all major semiconductor • Identifies issues as soon • Automated re-binning
test platforms as they occur • Adaptive test time reduction
• Ensures consistent data quality • Alerts operators • Drift detection
and high-speed delivery • Pauses the tester • Data-feed-forward
• Includes a wealth of information
not provided in regular data logs • and much more…
for accurate OEE analysis and
software/hardware validation
• Agnostic to, and supports all test
programs

Proxy+ is an agent running on the tester, enabling real-time data collection, control and action
ni.com
 ni.com

Proxy+ – Optimal+ Ambassador On The ATE

For:
Data collection

Proxy+ Adaptive testing

and tester control

Adaptive Hardware Performance Test

ATE Test Configuration Counters Data

Tester Executive Software

 Rules
Targeting Challenges 24x7
Library of standard rules
accommodate most the
challenges faced by our
industry

Custom rules available for

unique monitors and actions
including support for R and
Python scripts

Deployed at any level of your

supply chain (central vs edge)

Rules engine running 24x7

ni.com
 Rules Turning Challenges Into Actions
Action Categories

Engineering
Equipment Actions Pause
tool alert

Put materials
Process Actions Re-binning
on hold

Re-test Adaptive
Recipe Adjustments
skip/add testing

Virtual
Data Augmentation Feed-forward Feed-backward
operation

Quality Predictive/
Alerts Yield alerts
outlier alerts Anomaly alerts
ni.com
 Data Security Solution

1 End-to-end secure
data lifecycle
2 Key exchange
mechanism
5 Allowing Real-time
and offline rules
without exposing
• Authenticate with tester
sensitive data
3
OS and test program (TP)
Offline mode for
• Secure channel between
offline data logs and
Tester and Local Server

• Secure data at rest on

recovery
6 Compatible with older
TPs not implementing

4
tester and local server
encryption
Sensitive data filtering
• Secure transfer of data logs to
HQ to securely share data
logs with suppliers

ni.com
 Our Marketecture
GO+ Action
Adaptive Test Data Feed-
Shop Floor
Edge ML Edge+ (incl. Smart Re-Binning Auto-Hold Forward and
Control
Ramp) TP API
Real-time
Enablers

GO+
Scratch
Outlier

GO Security
WECO SPL EWMA Escape Adaptive Test
Ai / Ml Detection NPI
Advanced Analytics Detection Prevention Simulation
Central Solution Suite

GO
Portal+ BI

Core Analytics Data Lake

Proxy+ Data Lifecycle Platform
Solution Suite
Edge
Lifecycle Level 1-2 Level 3 Level 4
Data Sources Mfg Machines (Fab, Sort, Assembly, BI, FT, AOI, SLT) MES, QMS, YMS ERP, CRM, PLM,..

ni.com
 ni.com

Solution Examples
 ni.com

Quality, Reliability and

Brand Protection
Solutions
 Quality and Reliability

Minimize excursions
Minimize RMAs
Analyze root cause
Outlier Detection
Escape Prevention
Special quality algorithms –
WECO, EWMA, SPL, Scratch Detection

Protect your brand Auto-hold (via MES)

Re-bin (via MES)

Comply with automotive standards
Data Feed Forward and Test Program API

ni.com
 EXAMPLE

Outlier Detection
NNR (Near Neighbor Residual) Wafer Map – Original Data Table Statistical Widget

Statistical Widget

NNR Outlier

ni.com
 ni.com

Bivariate Outlier Detection

Bivariate outliers
may be related
to pairs of tests
from the same
or different
operations
 ni.com

Multivariate Outlier Detection

Several ML
techniques can
be used to screen
multivariate
outliers

The methodology
here is to use
PCA (Principal
Component
Analysis) to
define the main
PC’s as virtual
virtual tests, and
tests selection
then perform
DPAT on such
tests
 Specialty Algorithm Example
Scratch Detection

ni.com Patent pending

 ni.com

Escape Prevention Example

• Chart shows
Not Enough Tests Performed On Parts that the
number of
Standard PRR Low PRR tests for a
(53 tests) (39 tests) good device is
53 tests

• 5% of the units
on one lot
MISSING
have 39
TESTS
tests

• Automated
rule detects
this in
production and
prevents the
parts from
shipping
 PAT For Packaged Units (FT PAT)
Skip next testing for units marked as ‘outlier’ bins
* Requires ECID

Final Test O+ PAT Final Test

“n” Virtually Bin Outliers “n+1”
Physically Bin Out

Spec Limits Spec Limits

PAT Limits Pat Limits

ni.com
 Defectivity Index (I-PAT) Correlation To Sort
Better screening using both test and defectivity data I-PAT can identify
individual statistical
Applying I-PAT defect outlier recognition
outlier die, and drill
Using G-PAT to detect clusters using combination of test and I-PAT data down to root cause

Smart I-PAT Map Bin Map Bin Map Post Bin Map Post
Standard Outlier Detection Enhanced Outlier Detection

HB99  G-PAT HB998  I-PAT Static PAT

(test only) HB997  G-PAT outliers
ni.com
(test and defectivity)
 Image Processing Flow

Image algorithm
SAM inspection
analyzes each welding
generates board image
location (pin) on the
(jpg)
board

18 parameters are
Parameters are loaded
generated for each pin
into O+ to allow
on the board (instead of
analytics, rules and
just one parameter
correlations to
previously collected
machine/product data
– the welding area)

ni.com
 ni.com

Yield Analysis and

Reclamation Solutions
 Yield

Overall yield Baseline yield and SBL monitoring

Site-to-site yield Test equipment performance
Re-test policy Test and retest policies and execution
Equipment and hardware performance
issues Tests limits validation

Cross-operation correlation

Targets against any measure/KPI

ni.com
 ni.com

Customer Use Case: Operational Yield

Site Issue

Iteration 0 Device:
Network

Problem:
Yield loss

Iteration 1 Issue:
Yield by tester
varies

Standard O+ Rules Found

With no monitoring – Site-Site issue not detected – This case is 16 lots
 ni.com

Yield Improvement Example

Tight Spec Limits

• Current test
limits are too
tight, causing
0.4% yield loss
Existing Spec Limits • Proposed test
Cutting Distribution Tail limits will
reduce yield
loss without
impacting
Proposed limits are product quality
sensitive to
gross outliers
 ni.com

Efficiency
Solutions
 Efficiency

Inconsistent tester availability and Adaptive Testing using Machine Learning

utilization
Test equipment performance
Excessive index and pause times
Test and retest policies and execution
Test time variations
per tester Testers availability and utilization
Inefficient retest policies and execution (OEE analysis)

Classical Test Time Reduction

(TTR analysis, ROA)
Adaptive Test Time Reduction (ATTR)

Cross-operation correlations

Shop Floor Control

ni.com
 ni.com

Test Efficiency Opportunities

Better resolution of time during test

Actual test time maximization (vs. index time)
Retest optimization
Test time consistency
Tester utilization – owned, consigned or paid for
 ni.com

Customer Use Case: Efficiency Problem Device:

Microcontroller
Increasing Test Time with flash

Problem:
Capital avoidance
Issue: Needed 10
more test stations
Problem
Discovered:
Issue with test
program

Fix:

Standard O+ rules found Improved O+ rule

for monitoring for
Testers had different throughputs all future
Test Time Increasing from 120 Sec to 300 Sec testers/devices

Result: Saved 8 test stations = $12M in CapEx and OpEx Savings

 ni.com

Overall Equipment Efficiency

Tester Usage Breakdown By Category
O+ collects
detailed data on
tester operation

Tester usage
statistics allow
to perform
in-depth
productivity
analyses
(e.g. OEE) which
help eliminate
wasted time
 Customer Use Case
Test Time Reduction

HQ Facility A
Proxy/
Publish MES
Server

Simulate Create Rule

Tester/Proxy
Facility B
Proxy/
Publish MES
Server

Tester/Proxy

ni.com Identify tests that can be skipped, create rules and publish to the testers, wherever they are
 ni.com

Adaptive Test Time Reduction

Example Run Showing TTR Element

Sampling
Zero fails
validation
SKIPPED (before
TOUCHDOWNS skipping tests)
in each run
VALIDATION
TTR SAMPLE
TOUCHDOWN

SINGLE RUN
 ni.com

Time-To-Market
Solutions
 Time-To-Market Adaptive test (reduction or augmentation)
and smart ramp

Data loading rules

Shorten NPI time
Optimize balance between time, cost,
and quality
Facilitate multi-team collaboration
Share learnings from NPI to HVM
and back
Load and create conditions
Sandbox to edit metadata
Datasets
• Virtual “workbench”
• Shared analyses and data
augmentation
• Full chain of custody

Limits, Correlation and

GR&R Applications

Report generation
ni.com
 ni.com

NPI Areas Of Focus and Flow

O+ O+ Minimize time
to market
Data Dataset
Sandbox Analysis Reporting Analyze split lots
Collection Management
Determine
production limits
Identify design
• Proxy • OTDF • Data Cleansing • Association PVT Analysis • Static/Scheduled
sensitivities
• Drop • SAF • Mapping • Augmentation Correlation App • Flexible
Box
Limits App
• Validation • Attributes • Intuitive
• STDF GRR App
• Templates • Customized
 ni.com

Customer Use Case: Time-To-Market

Limit Simulation App
Device:
Cell phone
Problem:
Limits not
optimized
Issue:
Would not fail
questionable
measurements
Fix:
Run analysis using
limits application

O+ standard tools found: Limits too wide

Result: Immediate feedback = Faster product launch

 ni.com

Supplier Transparency
Solutions
 Supplier Transparency

Benchmark suppliers Site to site comparison

Ensure supplier compliance with flows Supplier to supplier comparison
for every chip

ni.com
 ni.com

Customer Use Case • Provides

consolidated
Suppliers Benchmark Dashboard For Key KPIs views of
operations
across all
suppliers mfg.
sites
• Enables
objective
benchmarking
of suppliers
• Highlights KPIs
Supplier #1 has lower
GPH than the others
that require
Supplier #1 Avg. attention
touchdown test time
longer than others • Enables
drilldown for
root-cause
analysis
Supplier #1 #2 #3 #4 #5 Supplier #1 #2 #3 #4 #5
 ni.com

Customer Use Case

• Provides
Supplier Transparency Into Consigned Test-Fleet Performance consolidated
view of
~20% of the fleet under
100s-1000s
performing – FPY issues
fleet tools
• Enables real
time and
Test tools wasting 60-80%
consistent
of test time in Pause mode
equipment sets
benchmarking
• Highlights KPIs
require
attention
• Enables
drilldown for
root-cause
analysis
 ni.com

Partner with us to enhance

your big data strategy with our
open platform
Synergetic With Any Data Lake | Cloud and On-Premise | Accessible Optimized Schema |
AI and Machine Learning | Collect and Act Anywhere | Enhance Data Scientist Productivity
 Data Platform Needs
Voice of the Market

“How can I combine, and do more with “Our data retention is at least 10 years
my siloed data systems?” for our automotive products.”

“I know we need to do ML, we just “How can we store old data so it doesn’t
don’t know how to get started.” take so long to reload and use?”

“My teams are proficient in “Can we have programmatic access to O+ data?”

Python or R and I want to leverage this.”

“I want to leverage fab/assembly data (i.e. defect

“We already have a corporate license of
and inspection) to improve my quality.”
Tableau, can we use this to visualize O+?”
ni.com
 Consolidated Challenges

CTO/CIOs
Concerned about enterprise TCO (Total Cost of Ownership)
and IT Professionals

Product, Quality and

Need a solution providing analytics that scale
Yield Engineering Teams

Data Scientists and

Need a collaborative ecosystem
Engineering Teams

ni.com
 Platform Goals

1 Support bi-directional
data integration with any
customer data lake
3 Integrating with
machine learning data
science frameworks,
5 Data security and
encryption

leveraging OptimalPlus
deployed infrastructure

2 Enable easy
consumption of 4 Boost developer's
innovation by leveraging
OptimalPlus data by 3rd OptimalPlus rich API’s,
parties and BI tools algorithms and
infrastructure

ni.com
 Industry Focused Open Platform
Optimal+ Platform Customer Platform
Optimal+ data pipeline
Synergetic with any Portal+ Desktop and Web UI
big data strategy
Analytics Engine and API
Connected to existing
infrastructure Data Pipeline IBM Streams

Open for all kinds of data

Column Store Database
Metadata index +“Hot” cache
Accelerates innovation SQL over Hadoop

Extensible through both

data and algorithms Optimized
“Manufacturing” Data
Customer Data

Customer Data Lake

(public,
private,
ni.com Cloud hybrid)
 The Full Machine
Learning Lifecycle
Learn from data and evaluate
business value

LEARN Deploy and act upon the model

ACT
VALIDATE Monitor data and model performance
ADAPT to identify changes

Understand changes and update

model/process

ni.com
 AI/ML Deployment Challenges
Learn Act Validate Adapt

Getting data Complex “plumbing” Ongoing validation Stale models

Data scientists waste Data scientists waste time Production models need Production changes
time getting and dealing with the “plumbing” to be validated all the inevitably cause models
organizing data associated with getting a time to go stale
model into production
Feature extraction Ongoing data collection Relearning
It is difficult to extract Actionability Data collection becomes Model relearning is often
complex features from Taking action requires an ongoing concern manual
the data set integration with equipment
and systems Technical debt
Freedom of choice Data scientists end up
Data scientists want to Distributed mfg. spending time monitoring
use their favorite tools Issues compounded in “old” projects instead of
and the latest-and- distributed, outsourced investing in new ones
greatest algorithms mfg.

ni.com
 ni.com

Hidden Complexity
The Google View

Data Machine
Verification Resource Monitoring It’s all
Data Collection
Management about the
Configuration Serving infrastructure
ML Infrastructure
Code Analysis Tools

Feature Process
Extraction Management
Tools

Source: Google article from 2014: Hidden Technical Debt in Machine Learning Systems
https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
 Optimal+ Covers The Entire Lifecycle
Central

PLM, ERP, CRM Portal+ www Rules+

Data Machine
Action
Rules
Data
Data

Data

Verification Resource Monitoring

Management
Data Collection
Configuration Serving
ML Infrastructure
O+ Central Analytics Code Analysis Tools
24x7 rule execution Cloud
Process
Data and orchestration Or Feature
On-Prem. Extraction Management
on/Rules Tools
O+ Data Platform
Central repository

Optimal+ covers the full scope all the way through ML deployment
Source: Google article from 2014: Hidden Technical Debt in Machine Learning Systems
ni.com https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf
 Summary
Lifecycle Analytics turning data into actions or immediate ROI
Solutions

Product-Centric for improved quality and reliability and

Approach operational efficiency

support digital transformation in

AI/ML manufacturing

industry focused for seamless integration

Open Platform
with any big data strategy

End-To-End Supplier across operations and industries

Transparency

applying data science to solve industry

Domain Expertise
challenges
ni.com
 Significant Business Impact
Quality, Reliability and 50% case avoidance
Brand Protection

Yield Analysis increase up to 10% NPI 2% HVM

and Reclamation

Efficiency up to 25% test cost savings

Time To Market from weeks to days NPI, TTM, RCA

Supplier Transparency Consistency and compliance

ni.com
 Ask Our Customers

“Escape Prevention enables us to identify “Optimal+ gives us real-time visibility of our test
specific manufacturing and test issues that operations, enabling us to monitor every critical
drive advanced quality screening and parameter to ensure that every product is of the
comprehensive product management.” highest quality and performs as expected.”

Michael Campbell Keith Katcher

SENIOR VP OF ENGINEERING VP OF OPERATIONS ENGINEERING

“Global Ops for Electronics enables us to “We see Optimal+ as a strategic partner. Their
rapidly identify and respond to the source of open architecture enables us to create synergy
any PCB and systems manufacturing issue, across different tools and systems across the
down to an operation, facility, line or station.” globe and accelerate innovation”

Vincent Tong David Reed

SENIOR VP OF GLOBAL OPERATIONS AND QUALITY EXECUTIVE VP OF TECHNOLOGIES AND OPERATIONS

ni.com
 ni.com

Thank You