1.10 BB wk1 CI - SS - Introduction To DOE

Intrroduction to Design
of Experiments
Black Belt Training
1 | Week 1 BB Training | V1.0 |

DMAIC Blueprint
Define Measure Analyze Improve Control
Define & scope Define ‘as is’ Identify potential Generate potential Create control &
problem process causes (x’s) solutions monitoring plan
Determine project Validate

Investigate Select & test Implement full
objective & measurement
significance of x’s solution scale solution
benefits system for outputs
Develop
Create project Quantify process ID sig. Causes to
implementation Finalize transition
charter performance focus on y=f(x’s) plan
Tollgate Tollgate Tollgate Tollgate Close Project
| Introduction to Control | Business Use Only
2 | Week 1 BB Training | V1.0 | 2

Objectives
• Introduce you to the concept of designed

experiments
• Introduce some key terminology in
experimental design
3 | Week 1 GB
BB Training
Training || V1.0
V1.0 || Introduction
History of Design of Experiments
• 1920s/1930s: Sir Ronald A. Fisher

– Experiments in Agriculture
– Importance of Replication, Randomization, Blocking
• 1950s: George Box
– Experimentation in Chemical and Process Industry (mainly R&D)
– Response Surface Models
• 1970s: Genichi Taguchi
– Experiments in Product and Process Design
– Robust Designs (Signal to Noise Ratios)
– Highly Fractional Designs
• 1990s
– Alternative Robust Design Methodologies
– Steepest Ascent Methods
– Mixture Designs
4 | Week 1 GB
BB Training
Training || V1.0
What is an Experimental Design?
• A systematic series of tests in which various Input

Variables (X’s)are directly manipulated and the effects on
the Output Variables (Y’s) are observed.
• To determine which X’s most affect Y’s
– Influential X’s to center the Y on the target
– Influential X’s to minimize the variability of Y
– Influential X’s to minimize the presence of Noise
Variables
• A well designed experiment eliminates all possible
causes except the one that you are testing
• If an effect occurs on the Key Process Output Variable,
then it can be tied directly to the KPIV’s you have directly
manipulated
5 | Week 1 GB
BB Training
Training || V1.0
Uses of Experiments
• Characterizing a Process
–Determining which X’s most affect the Y’s
–Includes controllable and uncontrollable X’s
–Identifies critical process and noise
variables
–Identifies those variables that need to be
carefully controlled in the Process
–Provides direction for controlling Inputs
rather than control charting the Outputs
• Optimizing a Process
–Determining where the critical Inputs 6 | Gold Training Week 3 | V2.0 | 02 Intro to
6 | Week 1 GB
BB Training
Training || V1.0
V1.0 || Introduction DOE | Business Use Only
Types of Experimentation
• Trial and Error

• One-Factor-at-a-Time (OFAT)
• Fractional Factorials
• Full Factorials
• Response Surface Methods
• Others
7 | Gold Training Week 3 | V2.0 | 02 Intro to

7 | Week 1 GB
BB Training
Training || V1.0
Trial and Error
• Problem: Current Car gas mileage is 20 mpg. Would like to get 30

mpg.
• We might try:
– Change brand of gas
– Change octane rating
– Drive Slower
– Tune-up Car
– Wash and wax car
– Buy new tires
– Change Tire Pressure
• What if it works?
• What if it doesn’t?

8 | Week 1 GB
BB Training
Training || V1.0
One-Factor-at-a-Time
• Problem: Gas mileage is 20 mpg
–How many more runs would you need to

figure out the best configuration of
variables?
–How can you explain the above results?
9 | Week 1 GB
BB Training
Training || V1.0
Full Factorial Experiment
• Problem: Gas Mileage is 20 mpg
OFAT Runs
10 | Gold Training Week 3 | V2.0 | 02 Intro

10 | Week 1 GB
BB Training
Training || V1.0
V1.0 || Introduction to DOE | Business Use Only
Factorials Compared to OFAT Studies
• Suppose we looked at one factor at a time

• In the case below, we run one trial on Temperature (1 vs. 2) at
Pressure 1
– Because of sampling (experimental) error, we conduct two runs at
each level
• Finding Temperature 1 the best, we conduct another trial holding
Temperature constant at 1 and vary the Pressure

11 | Week 1 GB
BB Training
Training || V1.0
OFAT and Interactions
Run One
Run Two
• Assume we did an OFAT study in the above situation

• If we held Temperature constant at level-1 and varied Pressure, we
would conclude that Pressure at level-2 is best
• Then holding Pressure constant at level-2, we vary temperature
and find Temperature 1 is best
While we would have made significant improvement, we would

have missed the optimum point!
12 | Week 1 GB
BB Training
Training || V1.0
Planning for Design of Experiments
13 | Week 1 BB Training | V1.0 |

Planning
• Team Involvement
• Maximize Prior Knowledge
• Plan the execution of all phases
• Rigorous sample size determination
• Allocate sufficient resources for data
collection and analysis
• Ensure you have a budget!!!!

14 | Week 1 GB
BB Training
Training || V1.0
Definition - Factor
Controllable factors
c
1
c2
c p
Inputs Output
Process
• Factors: A factorzis onez of the controlled orzuncontrolled

1 2 q
Uncontrollable factors
variables being studied in the experiment
– A factor may be quantitative, e.g., temperature in degrees, time in
seconds
– A factor may also be qualitative, e.g., different machines, different
operator, clean or no clean
In most cases, you’ll use the Critical X’s from your Scope Phase
15 | Week 1 GB
BB Training
Training || V1.0
Definition - Factor Levels
Pressure: HHHH LLLL HHHH LLLL HHHH LLLL
51
01
52
02
5
Temp: HHLL HHLL HHLL HHLL HHLL H HL L
130
Factors 120 Levels

Yield
110
100
In d e x
• Level: The “levels” of a factorTest

areSequence
the values of the factor being
examined in the experiment
– For quantitative factors, each chosen value becomes a level, e.g., if the
experiment is to be conducted at two different temperatures, then the factor “
temperature has two “levels”
– In a qualitative factor, clean or no clean has two “levels”

16 | Week 1 GB
BB Training
Training || V1.0
Factor Level Selection
• Too Close/Too Wide issues

• Objective: Determine vital few Inputs from a large number of
variables (Screening)
– Set “Bold” levels to include current setup
– Goal: if we vary the Input to extremes we will be ensured of seeing an
effect on the output if there is one
– May exaggerate variation
• Examples of Bold settings:
– Qualitative:
• Method A Vs B
• Reactor 1 Vs Reactor 2
– Quantitative
• 5 minutes Vs. 15 minutes
• 30 psi Vs. 60 psi

17 | Week 1 GB
BB Training
Training || V1.0
Factor Level Selection
• Objective: To better understand factor

interactions
–Once Critical Inputs are identified, reduced
spacing of the levels is used to identify
interactions among Inputs
–This approach is usually leads to a series of
sequential experiments
• Objective: To identify the operating window
of a set of input variables
–Close settings are again used
–Sequential experimentation is also used
18 | Week 1 GB
BB Training
Training || V1.0
Select Experimental Design
• Screening
• Full Factorial
• Fractional Factorial
• Response Surface
• Sample Plan: What to randomize
• How are we controlling Noise Variables that
we know about?

19 | Week 1 GB
BB Training
Training || V1.0
High Level Map of Experiments
Screening Designs
Fractional Factorials
Characterization Full Factorials

Studies
Optimization
Studies Response Surface Methods

20 | Week 1 BB Training | V1.0 | to DOE | Business Use Only
• Screening Designs:
– Uses: To isolate the “vital few” from the trivial many
– Investigates a relatively large number of factors with a small number
of experimental runs
• Characterization Designs:
– Uses: To identify the key leverage variables
– Investigates more complex relationships among a small number of
factors (2-6)
– Generally full factorial experiments or higher resolution fractional
factorial designs
• Optimization Designs:
– Uses: To define the optimal operating windows for key leverage
variables
– Full factorial and response surface designs

21 | Week 1 GB
BB Training
Training || V1.0
• Design (Layout): Complete specification of experimental test runs

including blocking, randomization, replications, repetitions, and
assignment of factor-level combinations to experimental units
Run Temperature Pressure Design Layout
1 Hi Hi
2 Hi Hi
3 Lo Hi
4 Lo Hi 2x2 Factorial:
Two Inputs
5 Hi Lo with two
6 Hi Lo Levels each
7 Lo Lo
8 Lo Lo

22 | Week 1 GB
BB Training
Training || V1.0
• k1 x k2 x k3 ... Factorial: Description of the basic design. A 2 x 3 x 3

design indicates three Input variables, one with two levels and two with
three levels. The number of treatment combinations is the product. In
this case we have 18 treatment combinations.
Run Temperature Pressure
1 Hi Hi 2x2 Factorial:
2 Hi Hi Two Inputs
with two
3 Lo Hi
Levels each
4 Lo Hi
5 Hi Lo
6 Hi Lo
7 Lo Lo
8 Lo Lo
23 | Week 1 GB
BB Training
Training || V1.0
• Treatment Combination: An experimental run using a set-up with

specific levels of each Input variable. Also known as a Cell or test
run.
Treatment
Combination 1 Hi Hi
or Test Run 2 Hi Hi
3 Lo Hi
4 Lo Hi
5 Hi Lo
6 Hi Lo
7 Lo Lo
8 Lo Lo

24 | Week 1 GB
BB Training
Training || V1.0
• Balanced Design: A design with an equal number of

experimental units in each treatment combination or
experimental run Run Temperature Pressure
1 Hi Hi
This is a balanced design
because we have 2 repetitions of 2 Hi Hi
each treatment combination
3 Lo Hi
4 Lo Hi
5 Hi Lo
6 Hi Lo
7 Lo Lo
8 Lo Lo

25 | Week 1 GB
BB Training
Training || V1.0
• Unbalanced Design: A design with an unequal number of units in each

treatment combination. DOE procedures automatically handle this. For
full-factorial experiments, you would use General Linear Model
procedure.
This is an unbalanced design
2 1 Hi Hi
because we have 2 repetitions
of each treatment combination
except for the last one
2 Hi Hi
2 3 Lo Hi
4 Lo Hi
2 5 Hi Lo
6 Hi Lo
1 7 Lo Lo

26 | Week 1 GB
BB Training
Training || V1.0
Terms
• Fixed Factor: If Factor levels are specifically assigned, the

Factor is said to be fixed. Inferences generalize to only those
Factors. Effects are of interest.
• Random Factors: If Factor levels are selected randomly from a
population of values, the Factor is said to be Random. Variance
Components are of interest.
• K-way Interaction: Interaction between K Input variables
• Effect: The change in the average response over two levels of a
factor or between experimental conditions
• Experimental Region (Factor Space): All possible factor-level
combinations for which experimentation is possible

27 | Week 1 GB
BB Training
Training || V1.0
Terms
• Test Run: A single combination of factor

levels that yields one or more observations of
the output variable
• Main Effect: The Average change from one
level to another for a single Factor
• Interaction: The combined effect of two
factors that is over and above the singular
effect of each factor

28 | Week 1 GB
BB Training
Training || V1.0
Calculation of Effects
• The Effect of a factor is defined as the average change in the Output

variable produced by a change in the levels of a factor. Consider the
table below where we have two factors, Temperature and Pressure. The
Output variable is yield.
• The change in yield across Pressures (the Main Effect for Temperature) is
defined as:
• As Temperature increases from level 1 to level 2, yield increases by 21

points

29 | Week 1 GB
BB Training
Training || V1.0
Calculation of Effects
Pressure 1 Pressure 2
Temp 1
20 30
Temp 2
40 52
• Likewise, the Effect for Pressure is defined as:
• Yield increases by 11 points as the Pressure is moved from level-1 to

level-2
• In some experiments we find that the effect between the levels of one
factor is not the same for different levels of the other factors

30 | Week 1 GB
BB Training
Training || V1.0
Example of a Factorial Experiment
• An engineer wants to test the effects of

Temperature and Pressure on chemical Yield
Run Temperature Pressure Main Effect: Mean
Test Run 1 Hi Hi (Hi) vs Mean (Lo)
Temperature
2 Hi Hi
3 Lo Hi
4 Lo Hi
5 Hi Lo
6 Hi Lo
7 Lo Lo
8 Lo Lo

31 | Week 1 GB
BB Training
Training || V1.0
Dealing with Noise Factors
• These must be controlled, but it is difficult

since we don’t necessarily know all the
factors
• Ways of controlling
–Use randomization
–Try to keep the variables constant
–Blocking: Make the factor part of the
experiment
–Repetition and replication
–Analysis of covariance
32 | Week 1 GB
BB Training
Training || V1.0
Inference Space
• Area within which you can draw your conclusions

• Two classifications: Broad and Narrow
• Narrow Inference
– Experiment focused on specific subset of overall operation
– Examples: Only one shift, one operator, one machine, one batch, etc.
– Narrow inference studies are not as affected by Noise variables
• Broad Inference
– Usually addresses entire process (all machines, all shifts, all
operators, etc..)
– Generally, more data must be taken over a longer period of time
– Broad Inference studies are affected by Noise variables

33 | Week 1 GB
BB Training
Training || V1.0
Inference Space
 Click to edit Master text  Click to edit Master text

styles styles
• Second level Narrow • Second level Broad
-Third level -Third level
• Fourth level
• Fourth level
» Fifth level
» Fifth level
Generally, Narrow inference studies are done first to control Noise variables. Broad
Inference Studies are used to verify results of the Narrow Inference studies.

CI_SS01_Green Belt| Overview
34 | Week 1 BB Training | V1.0 | | June 2019 to DOE | Business Use Only
Keep Noise Variables Constant
• One Machine
• Run study in a single day or shift
• Use the same operators throughout
How does this technique affect your inferences?

35 | Week 1 GB
BB Training
Training || V1.0
Blocking
Make the Noise Factor part of the experiment
• A Block is similar to the rationale subgroups

addressed in Capability Analysis and SPC
• Variability between blocks should be greater
than variability within blocks
• Add day, shift or batch as a factor to the
experiment
How does this technique affect your inferences?

36 | Week 1 GB
BB Training
Training || V1.0
Repetition and Replication
• Repetition and replication provide estimates

of the natural variability in the experimental
system
• Repetition: Running several samples during
one experimental setup run (Short-term
variability)
• Replication: Replicating the entire experiment
in a time sequence (Long-term variability)
• You can use both in the same experiment
What are the differences in your inferences when comparing repetition and replication?
• Both directly relate to sample size in the
experiment
37 | Week 1 GB
BB Training
Training || V1.0
Repetition
Pressure : HHHH LLLL HHHH LLLL HHHH LLLL
51
01
52
02
5
Temp: HHLL HHLL HHLL HHLL HHLL HHLL
130
120
Repetition
Yield
110
100
In d e x
Test Sequence

Replication
Pressure : HHHH LLLL HHHH LLLL HHHH LLLL
51
01
52
02
5
Temp: HHLL HHLL HHLL HHLL HHLL HHLL
130
1st
Replication
120
2nd
Replication
Yield
110
100
In d e x
Test Sequence

Experimental Validity
• Internal Validity:
–Did the Input Variables really affect the
Output Variables or did a Noise Variable
cause the results?
–Associated with Narrow Inference studies
• External Validity:
–How well will the results of the experiment
relate to similar processes, production
lines, process, days, etc.?
–Associated with Broad Inference studies
• Statistical Conclusion Validity:
40 | Week 1 GB
BB Training
Training || V1.0
Ensuring Internal Validity
• Randomization of experimental runs “spreads” the noise across

the experiment
• Blocking ensures Noise is part of the experiment and can be
directly studied
• Holding Noise Variables constant eliminates the effect of that
variable but limits Broad Inferences
• Statistically control Noise Variables (Analysis of Covariance) - Not
used often in manufacturing
• External Validity
– Include representative samples from possible Noise Variables
– Examples:
• Ensure experimental units represent supplier variability
• Do experiment across shifts and and days
• Include different product families

41 | Week 1 GB
BB Training
Training || V1.0
Randomization Example
• Assume we can see sixteen time points into the future. This is
the natural process and the two inputs Temperature and Pressure
are not having an effect.
100
90
Output
80
70
• Now we will backup in time and run a 2x2 factorial across values
0 5 10 15
of Temp and Press that do NOTRun
impact the Output.

42 | Week 1 GB
BB Training
Training || V1.0
• This is a 2x2 factorial run in nonrandom order. How many

repetitions, how many replications?
Press: L L L L H H H H L L L L H H H H
Temp: L L H H L L H H L L H H L L H H
100
90
Output
80
• What are our70conclusions about Pressure? Temperature? Use

the following Worksheet.
0 5 10 15
Run

43 | Week 1 GB
BB Training
Training || V1.0
• Objective: to verify the effectiveness of randomization

• Procedure:
– Estimate the values for each level of Temperature and Pressure
– Estimate the average effect of Temperature
– Estimate the average effect of Pressure
• Be prepared to report the results
Data Table
High (8 scores) Low (8 Scores) Effect(H-L)
89.1 70.4 18.7

Pressure:
78.2 81.4 -3.2

Temp:

44 | Week 1 GB
BB Training
Training || V1.0
• Now we run the same experiment in with the runs in random order
Press: L L H H L H L H L L L H H H L H
Temp: L L H H L L H H H L H L H L H L
100
90
Output
80
70
• What are our conclusions about Pressure? Temperature? Use
the Chart on the0 next page.
5 10 15
Run

45 | Week 1 GB
BB Training
Training || V1.0
• Objective: to verify the effectiveness of randomization

• Procedure:
– Estimate the values for each level of Temperature and Pressure
– Estimate the average effect of Temperature
– Estimate the average effect of Pressure
• Be prepared to report the results
Data Table
High (8 scores) Low (8 Scores) Effect(H-L)
Pressure: 81.4 78.2 3.2
Temp: 78.7 80.8 -2.1

46 | Week 1 GB
BB Training
Training || V1.0
What can go Wrong in Experiments?
• Factor levels can be too close together or too

far apart
• Nonrandom experiments can produce
spurious results
• Sample size may be too small
• Measurement systems may not be adequate
• No pilot run done: Experimental runs get
screwed up because of lack of discipline
• Confirmation run not done to verify results
• Data and/or experimental units lost
• Inadequate Test Method 47 | Gold Training Week 3 | V2.0 | 02 Intro
47 | Week 1 GB
BB Training
Training || V1.0

1.10 BB wk1 CI - SS - Introduction To DOE

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

1.10 BB wk1 CI - SS - Introduction To DOE

Uploaded by

Copyright:

Available Formats

Intrroduction to Design

1 | Week 1 BB Training | V1.0 |

Define Measure Analyze Improve Control

Determine project Validate

Tollgate Tollgate Tollgate Tollgate Close Project

| Introduction to Control | Business Use Only

2 | Week 1 BB Training | V1.0 | 2

• Introduce you to the concept of designed

• 1920s/1930s: Sir Ronald A. Fisher

• A systematic series of tests in which various Input

• Trial and Error

7 | Gold Training Week 3 | V2.0 | 02 Intro to

• Problem: Current Car gas mileage is 20 mpg. Would like to get 30

8 | Gold Training Week 3 | V2.0 | 02 Intro to

• Problem: Gas mileage is 20 mpg

–How many more runs would you need to

• Problem: Gas Mileage is 20 mpg

10 | Gold Training Week 3 | V2.0 | 02 Intro

• Suppose we looked at one factor at a time

11 | Gold Training Week 3 | V2.0 | 02 Intro to

• Assume we did an OFAT study in the above situation

While we would have made significant improvement, we would

13 | Week 1 BB Training | V1.0 |

14 | Gold Training Week 3 | V2.0 | 02 Intro

• Factors: A factorzis onez of the controlled orzuncontrolled

Pressure: HHHH LLLL HHHH LLLL HHHH LLLL

Factors 120 Levels

• Level: The “levels” of a factorTest

16 | Gold Training Week 3 | V2.0 | 02 Intro

• Too Close/Too Wide issues

17 | Gold Training Week 3 | V2.0 | 02 Intro

• Objective: To better understand factor

19 | Gold Training Week 3 | V2.0 | 02 Intro

Characterization Full Factorials

20 | Gold Training Week 3 | V2.0 | 02 Intro

21 | Gold Training Week 3 | V2.0 | 02 Intro

• Design (Layout): Complete specification of experimental test runs

22 | Gold Training Week 3 | V2.0 | 02 Intro

• k1 x k2 x k3 ... Factorial: Description of the basic design. A 2 x 3 x 3

• Treatment Combination: An experimental run using a set-up with

24 | Gold Training Week 3 | V2.0 | 02 Intro

• Balanced Design: A design with an equal number of

25 | Gold Training Week 3 | V2.0 | 02 Intro

• Unbalanced Design: A design with an unequal number of units in each

26 | Gold Training Week 3 | V2.0 | 02 Intro

• Fixed Factor: If Factor levels are specifically assigned, the

27 | Gold Training Week 3 | V2.0 | 02 Intro

• Test Run: A single combination of factor

28 | Gold Training Week 3 | V2.0 | 02 Intro

• The Effect of a factor is defined as the average change in the Output

• As Temperature increases from level 1 to level 2, yield increases by 21

29 | Gold Training Week 3 | V2.0 | 02 Intro

• Yield increases by 11 points as the Pressure is moved from level-1 to

30 | Gold Training Week 3 | V2.0 | 02 Intro

• An engineer wants to test the effects of

31 | Gold Training Week 3 | V2.0 | 02 Intro

• These must be controlled, but it is difficult

• Area within which you can draw your conclusions

33 | Gold Training Week 3 | V2.0 | 02 Intro

 Click to edit Master text  Click to edit Master text

34 | Gold Training Week 3 | V2.0 | 02 Intro

How does this technique affect your inferences?

35 | Gold Training Week 3 | V2.0 | 02 Intro