1

An Overview of
Designed Experiments
By: Larry A. Scott
Principal: Process Technologies
Welcome to:
DOE Simplified:
Agenda
Intro to DOE:
Full Two-Level Factorials
Design: Eye-Hand Exercise
Analysis:
Interactions The Hidden Gold
Example: Engine Performance
Overall Strategy Of DOE
2
Process Process
Controllable Factors (X) Controllable Factors (X)
Responses (Y) Responses (Y)
Uncontrollable Variables (Z) Uncontrollable Variables (Z)
DOE Works on DOE Works on Any Any Process Process
DOE is:
A series of tests,
in which purposeful changes
are made to input factors,
so that you may identify causes
for significant changes
in the output responses.
Design Terminology
OFAT vs. DOE (factorials)
Factor Effects
Main Effects
Interactions (effects)
Balanced Test Matrix
Array Codes
Prediction Equations (Y = mX + b)
Y = b
0
+ b
1
X
1
+ b
2
X
2
+ b
12
X
1
X
2
3
Full TwoLevel Factorial Design
Run high/low combos of two or more factors
Use statistics to identify the critical few
Main effects
Interactions (the hidden gold!)
What could be simpler?
Exercise: Eye-Hand Coordination
Your mission: In a 10 second span, mark
as many dots in two circles as you can.
You must alternate between circles and
use only one hand.
Response: Number of dots in both circles.
1 inch
2inch
+
Eye-Hand Coordination
Factors:
A. Which hand holds the pen:
Non-Dominant hand (ND)
+ Dominant (D)
B. Size of circle:
Small
+ Large
Eye-Hand Coordination
Factor Space and Test Matrix
A
Hand
B
Size
AB
1 + ___
2 + ___
3 + ___
4 + + + ___
Y
3 4
1 2
A
B
+
-
-
+
ND D
Large
Small
5
Eye-Hand Coordination
Templates
1. Small, Dominant: ___
3. Small, Non-Dominant: ___
2. Large, Dominant: ___
4. Large, Non-Dominant: ___
Eye-Hand Coordination
Procedure for Doing Exercise
1. Write the numbers 1 through 4 on four
slips of paper. Put these in one hand
and blindly pull out at random.
2. Following procedure in your notes,
perform tests in random run order.
Record data (Y) in blank at bottom of
templates. Chart on following graph.
6
Eye-Hand Exercise Outline
1. Randomly perform the + eye-hand tests.
2. Count and record responses (cycles).
3. Plot responses on interaction graph.
Reference:
DOE Simplified Practical Tools for Effective Experimentation
by Mark J. Anderson and Patrick J. Whitcomb, Productivity, Inc., Portland, OR
(2000).*
Eye-Hand Coordination
Interaction Graph
20
15
10
5
Non-Dominant
Hand
Dominant
The Y-axis shows
the number of
dots (cycles).
Plot the data for
D vs. ND for the
small circles,
connect with a
line and label.
Then do same for
the large circles.
7
Eye-Hand Coordination
Interaction Graph
Using statistical software, lets see how
well the Stat-Ease programming staff did
on this exercise and compare results.
Eye-Hand Coordination
Interaction Graph (published*)
B: Circle Size
B
u
l
l
s
e
y
e
s
A: Hand
Non-dominant Dominant
15
20
25
30
35
40
45
Small
Large
In this case, there
was a significant
interaction: the
effect of switching
hands became
more pronounced
with large circles.
*Gunter, et al, Linking High School Nath and Science Through Statistical DOE," 1995, Nacomb, N!, School District.
8
Real-World Example: OFAT Misses
Breakthrough Interaction!
L
i
f
e

130
B
A+ A
90
50
10
B+
Eye-Hand Coordination
Factor Space and Test Matrix
3 4
1 2
A
B
+
-
-
+
ND D
Large
Small
A
Hand
B
Size
AB
1 + ___
2 + ___
3 + ___
4 + + + ___
Y
9
How many tests did we perform? ______________
How many tests using left hand? ______________
How many tests using right hand? ______________
How many tests using large circle? ______________
How many tests using small circle? ______________
What was our relative efficiency? ______________
Relative Efficiency of Factorial vs
One Factor at a Time (OFAT)
Advantages of DOE vs. OFAT
- Efficiency
- Simplicity + Effectiveness
- Confidence (from stats)
- !nteraction Data
- Predictive Capability within Design Space
- Extrapolation Capability outside Design Space
- variation Data
- Cost !nfo
10
Relative Efficiency of Factorial vs.
One Factor at a Time (OFAT)
Relative efficiency = Relative efficiency = 16 16/ /8 = 2 8 = 2
A- A+
B-
B+
C-
C+
17
19
26
16
25
21
85
128
Case Study: 2
3
Factorial Design
Background: An engineer wants to study the effect
of three factors on mileage performance of an engine.
30 10 Spark Angle (deg)
52 8 Engine Torque (lb/ft)
5000 2000 Engine Speed (rpm)
High (+) Low () Factor
How is miIeage performance How is miIeage performance How is miIeage performance How is miIeage performance
infIuenced by the factors studied? infIuenced by the factors studied? infIuenced by the factors studied? infIuenced by the factors studied?
1. D. Grove and T. Davis, Engineering QuaIity & ExperimentaI Design (1992),
Longman Scientific & TechnicaI with John WiIey & Sons, Inc.
11
2
3
Full Factorial Array
1010 25.8 30 32 5000 8
324 9.1 30 32 2000 7
219 12.5 30 8 5000 6
27 3.7 30 8 2000 5
528 29.0 10 32 5000 4
201 9.8 10 32 2000 3
77 15.5 10 8 5000 2
13 4.5 10 8 2000 1
Y
2
Y
1
C B A
Std
Order
Responses:
Y
1
= fuel flow (lb/hr)
Y
2
= NO
x
(g/hr)
Fuel Flow:
All Effects Calculated
13.73 -2.13 -0.03 -1.18 4.02 -1.92 9.38 Effect
25.8 + + + + + + + 8

+
+

+
AB

+
AC
+

+
+
BC

+
+

ABC
9.1 + + 7
29 + + 4
4.5 1
12.5 + + 6
3.7 + 5
9.8 + 3
15.5 + 2
Y
1
C B A Std
Fill in effect of A. Which, if any, effects are significant?
12
Fuel Flow (Y
1
):
Calculating Effect of A (Engine Speed)
B
A
C
Effect
Y
n
Y
n
= == =
+ ++ +
+ ++ +



General formula:
A
effect
= == =
+ ++ + + ++ + + ++ +

+ ++ + + ++ + + ++ +
= == =
15.5 29 12.5 25.8
4
4.5 9.8 3.7 9.1
4
13.4
9.1 25.8
29 9.8
4.5 15.5
3.7
12.5
Engine speed
+

Half-Normal Plot:
Identify the Big Effects
A: Engine speed
B: Engine torque
C: Spark angle
Effect
0.00 3.48 6.96 10.44 13.93
0
20
40
60
70
80
85
90
95
97
99
A
B
AB
Effect
0.00 0.24 0.47 0.71 0.94
0
20
40
70
80
85
90
97
99
A
B
C
AB
Fuel flow
NOx
60
13
Sparsity of Effects Principle
Trivial Many: the remainder that result from random variation.
These effects will be centered on zero.
Since they are based on averages,
you can assume normality
by the Central Limit Theorem*.
Two types of effects:
Vital Few: the big ones we want to catch
20 % of ME's and 2FI's will be significant.
Half-Normal Probability Paper
Significant effects
(the vital few) fall
abnormally high (to
the right) on the
absolute effect
scale. These are
the keepers.
What do you do with
the little ones?
7.14
21.43
35.71
50.00
64.29
78.57
92.86
P
i
0
|Effect|
A
B
AB
3 6 9
12
15
1+
Interactions The Hidden Gold
The effect of speed
depends on the level
of torque. This is an
interaction.
Can you explain
whats happening to
the fuel flow?
B - B +
30
24
6
0
Fuel flow
torque
L
o
w

s
p
e
e
d
H
i
g
h

S
p
e
e
d
12
18
Analysis of Variance Report
(ANOVA)
Sum of Mean F
Source Squares DF Square Value Prob > F
Nodel 595.99 3 198.66 78.0
Residual 10.19 + 2.55
Total 606.18 7
Factor Coefficient
!ntercept 13.7+
A-Engine Speed 6.69
B-Engine Torque +.69
AB 2.01
0.0005
15
New!
Minimum-Run Resolution IV Designs*
The minimum number of runs for
resolution IV design is only two times the
number of factors (runs = 2k). This can offer
quite a savings when compared to a regular
resolution IV 2
k-p
fraction.
*(Invented by Whitcomb and Oehlert. Details posted to www.StatEase.com
- proceedings of talk at Annual Quality Congress, Toronto, 5/04)
16
Minimum-Run Resolution IV Designs
50 64 25 30 32 15
48 64 24 28 32 14
46 64 23 26 32 13
44 64 22 24 32 12
42 64 21 22 32 11
40 64 20 20 32 10
38 64 19 18 32 9
36 64 18 16* 16 8
34 64 17 14 16 7
32* 32 16 12 16 6
Min 2
k-p
k Min 2
k-p
k
50 64 25 30 32 15
48 64 24 28 32 14
46 64 23 26 32 13
44 64 22 24 32 12
42 64 21 22 32 11
40 64 20 20 32 10
38 64 19 18 32 9
36 64 18 16* 16 8
34 64 17 14 16 7
32* 32 16 12 16 6
Min 2
k-p
k Min 2
k-p
k
No savings for 8 or 16 factors.
RSM: When to Apply It
Region of Operability
Region of Interest
Use factorial
design to get close
to the peak. Then
RSM to climb it.
17
RSM vs OFAT OFAT
-2 -1 0 1 2
30
45
60
75
90
Factor A
R
e
s
p
o
n
s
e
-2 -1 0 1 2
60
65
70
75
80
85
90
Factor B
Response
R
e
s
p
o
n
s
e
65
73
80
88
95
Response
-4
-2
0
2
4
-4
-2
0
2
4
Factor A
Factor B
R
e
s
p
o
n
s
e
Advantages of DOE vs. OFAT
- Efficiency
- Simplicity + Effectiveness
- Confidence (from stats)
- !nteraction Data
- Predictive Capability within Design Space
- Extrapolation Capability outside Design Space
- variation Data
- Cost !nfo