Lisa - Add new front

page

Process Matching/Installation and Qualification (IQ)

Course ID: 10002315

Revision 3.0 September 2015

Intel Confidential
Pre-Requisites

Prior to starting Module 2 of this course, you will need to have completed the Web based
training (WBT) for the XCCB website.
It is assumed you know how to initiate a PWP from the XCCB website including filling
out the meta data as well as PWP content.

If you have not done so already, please pause this course and complete the WBT for this
course:
xCCB V2 Overview for Authors ( My Learning Course ID: 10001910, Version: 1.0)

Intel Confidential DASM – Data Analytics and Statistical Methods Group 2

Module 2
Designing an IQ/Process Matching Experiment

Intel Confidential
Designing an IQ Experiment

At the end of this lesson you should be able to:

 Understand key concepts related to IQ/Matching experimental design and planning.
 Design an IQ/Matching experiment and document the plan in a PWP.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 4

Standard IQ/Process Matching Experiment

Obtain a reference tool dataset of 8-30 lots.

Design a series of 8-16 “runs” on the new tool that imitates normal production.
 Use the same sampling plan within run as the reference dataset.
 Include the same noise factors that are present in the reference dataset.

Statistically compare the output from the experiment.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 5

Process Change Control
ATTD methodology requires new tool qualifications to be completed under change
control.
 Experimental plans are documented in a Class 3 Preliminary White Paper (PWP).
 Analysis results are documented in a Final White Paper (FWP).

Use the PWP template as a guide for DOE planning.

See spec 20-0053 for PWP/FWP approval procedures
PCCB website URL here

Intel Confidential DASM – Data Analytics and Statistical Methods Group 6

Qualifying Multiple “Like” tools

 A single Class 3 PWP may be used for qualifying like tools/links.

– 1st tool is submitted as Class 3 FWP as the “reference tool”.
– Each site will submit the first tool qual using Class 3 FWP.
– Subsequent tools at sites are qualified via Class 3N FWP process that compares each of the
new tools against the reference tool.
–
– A Reference Tool is the single tool to which all new or future tools are compared.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 7

Reference Tool Selection
When Technology Development is involved, the reference tool is the primary tool at the
development site.
The reference tool changes to the primary tool at the first VF site after transfer
certification completed or in the case of a direct start up.
In HVM environment, the reference tool is chosen from among the fleet of HVM tools
according to the following criteria:
 Closest to CE! configuration.
 Average-to-high volume runner.
 Demonstrated stability and capability.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 8

Experimental Objective

The goal of an IQ experiment is to demonstrate that a new tool produces statistically

equivalent output compared to a reference tool.
 Identify and fix non CE! configurations or “dog” tools sooner rather than later.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 9

Example

You need to write a PWP to bring up a new IHS link in the VF.

 Reference Link: B IHS 05 (TD link)

 New Link: G IHS 03 (VF link)

B IHS 05 G IHS 03

Intel Confidential DASM – Data Analytics and Statistical Methods Group 10

Section 7: Process Factors
Process Factor: Name of the toolset
Present Value: Entity name of the reference tool or link.

Proposed Value: Entity names of the reference tool and

the new tool or link.

IHS Example:

Process Factor Present Value Proposed Value

B IHS 05 (TD link)

IHS Link B IHS 05 (TD link) G IHS 03 (VF link)

Intel Confidential DASM – Data Analytics and Statistical Methods Group 11

Section 10: Quality Characteristics

Include “CE! Compliance” for 5M + E checklist for tool IQ.

List all Level 3 process characteristics affected.
 Default is all KPPs and CPPs for the module (PCS 301 for definition)
 Document any omissions in the concerns and consideration section 9.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 12

Section 10: Quality Characteristics

Use separate rows for multiple chart types (lot/run level data).
 Use (run-to-run) suffix for x-bar charts.
 Use (within-run) suffix for s charts.
 Include all that are applicable:
– Stability, capability, yield/yield loss assessments
– Reliability and PUPH (pure units per hour) studies.
Intel Confidential DASM – Data Analytics and Statistical Methods Group 13
Section 10 – IHS Link Qual Example

There are several other possible QCs in IHS such as Offset in Y Direction, Flux weight, Lid
Tilt, etc.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 14

Section 10 – IHS Link Qual Example

Measurement method is how you will directly measure the quality characteristic.

Ideal target or spec limits represents ideal state of each quality characteristic.

• Ideal target is the ideal state if there is no variability in the process.

• Ideal target for a standard deviation/ within run statistic is zero.

• Ideal target for yield is 100%. Assist rate or defect rate has ideal target is 0%.

• Use Ideal target “as high as possible” for a QC that is ideally infinitely large.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 15

Section 10 – IHS Link Qual Example

Include present mean and standard deviation for each quality characteristic.
Note the source of reference data below the table.
Intel Confidential DASM – Data Analytics and Statistical Methods Group 16
Section 10 – IHS Link Qual Example

Expected change for most IQ and Process Matching experiments is “None”

Intel Confidential DASM – Data Analytics and Statistical Methods Group 17

Reference Dataset
The reference dataset represents the distribution that you want to match.
Characteristics of a good reference dataset.
 Sufficient size
– 8 to 30 lots (16 lots is optimal)

 Stable
– Should not include any process shifts, trends, extreme outliers, or any other special cause variations.

 Includes noise factors that normally occur in production.

 Does not include false replicates or imputed values.

A good reference data set provides an “apples-to-apples”

comparison to the qualification data collected on the new
tool.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 18

Assess the Reference Dataset
Issues with reference data can be detected using the Stability Analysis script in SOS
 For detailed discussion of SOS scripts, please refer to PCS 301.

The SOS script can be used to detect: An acceptable reference dataset can:
• Sample size issues • Be slightly off target
• Lack of stability in the reference data set • Have %OOC > 0%
• False replicates/imputed values • Do not exclude OOC’s if they are
unexplainable
• Show time trends if they are a normal
part of the process

Intel Confidential DASM – Data Analytics and Statistical Methods Group 19

Possible Reference Data Sources

Recent assembly and test technology certification (ATTC) data collection.

Extract of recent production lots.
Specially designed reference dataset.
If a reference tool switches to a VF tool, use the initial qualification data from the VF
tool.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 20

Updating the Reference Dataset

Reference dataset do need to be maintained to ensure they are current.

Update the reference dataset after:
 Process or material change.
 Substantial time has elapsed (1 year or more)

If after several tool quals, the new tool always seems to be much better.
 Redefine reference dataset or reference tool to the best in the fleet.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 21

Knowledge check: IHS Reference Dataset 1

Does this meet the criteria of a good reference dataset?

Intel Confidential DASM – Data Analytics and Statistical Methods Group 22

Knowledge check: IHS Reference Dataset 2

• Does this meet the criteria of a good reference dataset? (Target

= 140)
Intel Confidential DASM – Data Analytics and Statistical Methods Group 23
Knowledge check: Singulation Reference Dataset 3

• Does this meet the criteria of a good reference dataset? (Target = 14)
Intel Confidential DASM – Data Analytics and Statistical Methods Group 24
Explanation: Singulation Reference Dataset 3
• During the singulation process, the blade wears as it cuts.
– Blades need to be replaced periodically.
• If the blade naturally wears on its sides instead of its edge, the size of
the units will grow as more product is processed.
– Blades were replaced twice during the data collection (identified on chart)

• Is this a good reference

dataset?
– Yes, if the sampling plan
for the new tool is very
similar.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 25

Checkpoint 2.1a
Evaluate the appropriateness of each of the proposed reference
datasets.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 26

Checkpoint 2.1b

Note: target = 0

Intel Confidential DASM – Data Analytics and Statistical Methods Group 27

Checkpoint 2.1c
Note: target = 0

Intel Confidential DASM – Data Analytics and Statistical Methods Group 28

Checkpoint 2.1d

Intel Confidential DASM – Data Analytics and Statistical Methods Group 29

Checkpoint 2.2

Open the reference data set found in JMP under Process Matching (IQ) Data sets.

Using your knowledge of JMP and DOE1 learnings, complete the empty cells in the table
on the next page.
Enter your results with three decimal places.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 30

Section 10 with Present Values
Complete the information needed for the three red empty boxes below.

First
Second

Third

Intel Confidential DASM – Data Analytics and Statistical Methods Group 31

Noise Factors
What is a noise factor?

A noise factor is a process parameter that impacts

the process or product parameters in Section 10,
table of quality characteristics.

A noise factor is either controlled or uncontrolled

in the design of experiment.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 32

Section 11: Noise Factors

Include all noise factors that are present in normal production.

Typical noise factors to consider:
 Days
 Shifts
 Operators
 Piece part suppliers
 Supplier SLIs
 Fab lots

Intel Confidential DASM – Data Analytics and Statistical Methods Group 33

 Section 11 Noise Factors – IHS Example

List the noise factor and the range of possible values for each noise factor.
• Example - substrate supplier as a noise factor has Shinko, Ibiden, and UMTC.

For a noise factor that have unlimited ranges, enter “multiple” to document that multiple
levels are possible.
Intel Confidential DASM – Data Analytics and Statistical Methods Group 34
 Section 11 Noise Factors – IHS Example

Record noise factor as scripted, unscripted, or fixed (1 value).

• All important noise factors should be scripted or fixed.

For scripted factors, determine the number of levels. 2-4 levels are recommended.
. Intel Confidential DASM – Data Analytics and Statistical Methods Group 35
Section 11 Noise Factors - Completed

Intel Confidential DASM – Data Analytics and Statistical Methods Group 36

Section 12: Experimental Strategy

Determine the experimental plan, including:

Number of experiments
Number of runs
Treatment of noise factors
 Experimental script

Accept criteria
Experimental flow

Intel Confidential DASM – Data Analytics and Statistical Methods Group 37

Define Experimental Strategy
First experiment should always be “CE! Audit”.
Combine QCs measured on the same experimental material into the same experiment.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 38

Number of Runs – Continuous QCs

Use the convention of 8-16 runs.

 Enter the N Actual. (must be ≥ 8)

 Since IQ experiments are standard designs, you may use “NA” for the Statistical Delta if this standard is
being followed.


Continuous N
Quality Actual (Statistical Delta)
Characteristic Based on N Actual

Pre-heat D1
 8 runs NA
(run to run)

 Pre-heat D1
8 runs NA
(within run)
Note: Sample sizes for binomial QCs should follow binomials
methods for defining detectable delta.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 39

 https://www.canstockphoto.com/kids-running-
Sampling Within Run 24328450.html

The number of samples within run is assumed to be constant.

 Same for reference data set and new dataset.

If sample sizes are varied, it could bias the standard deviation tests.
 Consult a statistician.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 40

Accept Criteria
Accept criteria for each QC is defined based on the type of hypothesis tests being
conducted.
This topic is discussed in depth in both DOE1 Comparative experiments and Statistical
methods for Binomial data, both of which are recommended pre-requisites for this course.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 41

Accept Criteria
Accept criteria for each QC is defined in the last column of Section 12 table.
There is a pull down menu that is activated when filling out the table.
100% CE is the accept criteria for the CE! Audit

Intel Confidential DASM – Data Analytics and Statistical Methods Group 42

Accept Criteria – Continuous QCs with lot structure
Four statistical comparisons are made when working with continuous QCs and lot level structure.

QC Means Test Std Dev Test

SE to Target -or-
Run-Run SEB to POR
SEB to POR

Within Run SEB to POR SEB to POR

Use the options in the template pull down menu:

 Use “SE to Target (Run to Run), SEB to POR (Within Run)” as the default.

 Use “SE to POR (Run to Run), SEB to POR (Within Run)” for non-targeted QCs.
– Note: can conclude SB for non-targeted QCs

Intel Confidential DASM – Data Analytics and Statistical Methods Group 43

Accept Criteria – Binomial QCs
Quality Characteristic SEB Comparison SB comparison
Defect Data (% or DPM) H0: p  p0 H0: p ≥ p0 (SEW)
H1: p > p0 (SW) H1: p < p0

Yield (%) H0: y  y0 H0: y  y0 (SEW)

H1: y < y0 (SW) H1: y > y0

Use SEB to qualify new tool, cost reduction projects or productivity improvement
projects.

Use SB for all process improvement projects like reducing non-wet, decreasing Bin15
fallout rate, etc.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 44

Section 12 Example

CE! Experiments

KPP w/ targets:
SE to Target/SEB to POR
For run-run/within run

Intel Confidential DASM – Data Analytics and Statistical Methods Group 45

Section 12 Example Continued…

KPP w/o targets:

SEB to POR for run-
run/within run

Binomial comparisons

Intel Confidential DASM – Data Analytics and Statistical Methods Group 46

Section 13 - Experimental Script
Create an experimental script for all scripted noise factors in Section 11 and include in
the “Execution/DOE Details” section under the Experiment Details tab (section 13).
If the script is not clearly written, it may lead to poor execution and difficulty in
analyzing and interpreting the results.
One row for each run.
Always include time factors (days, shifts)
 Recommend 4 days of 2-4 runs per day, at least 4 hours apart.
Include other noise factors as part of a designed experiment. (refer to DOE course)
 Setups, Raw material suppliers and supplier lots, PMs, Operators

Intel Confidential DASM – Data Analytics and Statistical Methods Group 47

Example Script
Run / SubstrateS Lid SealantS
Setup Day Shift upplier Lid SLI Supplier LI
1 1 4 IB S1 SKO A1
2 1 5 SKO R2 HEM B1
3 2 4 SKO S2 HEM B1
4 2 5 IB R1 SKO A1
5 3 4 IB R1 HEM B2
6 3 5 SKO S2 SKO A2
7 4 4 IB R2 HEM A2
8 4 5 SKO S1 SKO B2

Wait at least 4 hours between runs.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 48

Execution/DOE Details
Describe all of the steps required to complete the experiment successfully.

An example of DOE details that might be needed:

Follow standard assembly flow with scripted noise factors for lid and preforms as shown on
scripted table:
 After dummy tray run on the tool:
– Take dummy tray and measure on HPR tool and transfer data to station controller.
– Take dummy tray and measure data using scale and send to scale station controller.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 49

Experiment 2 Details

Intel Confidential DASM – Data Analytics and Statistical Methods Group 50

Experiment 2 Details Cont’d

Intel Confidential DASM – Data Analytics and Statistical Methods Group 51

Experiment 2 Details Cont’d

Intel Confidential DASM – Data Analytics and Statistical Methods Group 52

Checkpoint 2.2a
Critique each PWP element. Make corrections where necessary.

Process Factor Present Value Proposed Value

IHS Parameters B IHS 05 G IHS 03

Answer for Lisa: Proposed value should include both present and proposed values.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 53

Checkpoint 2.2b
Critique each PWP element. Make corrections where necessary.

Quality Measurement Ideal Present Mean Present Expected

Characteristic Method Target Std. Dev. Change

Preform Offset X MARS Database 0 -0.005 0.002 None

TIM Weight Scale 48mg 48.90 TBD None

Answer for Lisa:

MARS database is not a measurement method.

Present stdev for TIM weight should have a number based on reference tool data.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 54

Checkpoint 2.2c
Critique each PWP element. Make corrections where necessary.

Noise Factor Range of Possible Select Scripted or Range

Values/Scope Unscripted Evaluated
Links 7 Unscripted 1
Operator Many Unscripted ≥3
Preform Supplier Many Scripted 1
Preform SLI Many Scripted Many
Lid Supplier Many Scripted 2
Lid SLI Many Scripted Many

Ans for lisa:

Scripting preform supplier means more than 1 in the range evaluated.
Preform SLI and Lid SLI have Many levels but are scripted. Scripting means define the specifics of the noise factor

Intel Confidential DASM – Data Analytics and Statistical Methods Group 55

Checkpoint 2.2d
Critique each PWP element. Make corrections where necessary.

Lisa Ans: Description of experiment is CE! Audit. Therefore only CE! Compliance should be checked, there is no quantity of
material and accept criteria is 100% CE!

Intel Confidential DASM – Data Analytics and Statistical Methods Group 56

Checkpoint 2.2e
Critique each PWP element. Make corrections where necessary.

Ans for Lisa:

Link matching to TD reference link is either concurrent or non

concurrent. One sample isn’t possible if matching to POR.
Need to include more details for executing.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 57

Checkpoint 2.2e

Ans for lisa: these are continuous QCs and are listed as binomial. Need to move
them up to continuous QC section. Statistical Delta should be filled out as N/A since
it falls within the 8-16 lot BKM.

Intel Confidential DASM – Data Analytics and Statistical Methods Group 58

Quiz

1. For a lot level quality characteristics that has a reachable target, what is the accept
criteria for each test. (Possible options: SE to POR, SE to Target, SB to Target, SEB
to POR, SB to POR)
1. Run to Run (Mean) Test – SE to Target
2. Run to Run (Standard Deviation) Test – SEB to POR
3. Within Run (Mean) Test – SEB to POR
4. Within Run (Standard Deviation) Test – SEB to POR

2. Matching experiments with lot level parameters should ideally have at least __ lots (8
is the answer)

Intel Confidential DASM – Data Analytics and Statistical Methods Group 59

 13. Detailed Description of each Experiment in the Experimental Strategy
Purpose: To document the details of each experiment or testing procedure and to provide sufficient detail to conduct the experiment table

Experiment ID:
Objective of the experiment/procedure:

Ensure
Ensure IHS
IHS tool
tool in
in VF
VF is
is matched
matched to
to TD
TD reference
reference tool
tool on
on critical
critical and
and key
key process
process parameters
parameters

Experimental Design

One Sample
Concurrent (including matched pair)
Non-Concurrent
MCA
Other

Details

Execution/DOE Details

7. Process Factors

Process Factor Present Value Proposed Value

IHS Link B IHS 05 ??

Intel Confidential DASM – Data Analytics and Statistical Methods Group

 Continuous Quality Characteristics N Actual 𝛿
(Statistical
Delta)

X
X Placement
Placement Offset
Offset (Run
(Run to
to Run)
Run) ?
X
X Placement
Placement Offset
Offset (Within
(Within Run)
Run) ?
Lid
Lid Pull
Pull Force
Force 120
120 Units
Units

Intel Confidential DASM – Data Analytics and Statistical Methods Group

 Data Analytics and Statistical Methods Group

Intel Confidential