Design of Experiments

Alvin Villegas
February 27, 2009

Design of Experiments An Introduction

Design of Experiment (DOE) is a structured, organized method for determining the relationship between factors (Xs) affecting a process and the output of that process (Y). Other Definitions: 1. Conducting and analyzing controlled tests to evaluate the factors that control the value of a parameter or group of parameters. 2. "Design of Experiments" (DoE) refers to experimental methods used to quantify measurements of factors and interactions between factors statistically through observance of forced changes made methodically as directed by mathematically systematic tables.

- From isixsigma.com Library of Six Sigma Terms

Why DOE?
I want to Project Objective: Improve Appearance Objective improve my Physical Project physical Personal Care Sleeping Habit appearance

Diet

Exercise

Why DOE?
FACTORS OPTION 1 OPTION 2

Personal Care

Astringent

Facial Treatment

Diet

Fruit/Veggies

Meat

Sleeping Duration

4 Hours

8Hours

Exercise

Walking

Full Weights Training

Why DOE?
Run No. 1 Personal Care Astringent Diet Meat Sleeping Duration 4 hrs Exercise Walking Results

Astringent

Fruits/Veggies

8 hrs

Weights

Astringent

Meat

8 hrs

Walking

Facial

Fruits/Veggies

4 hrs

Walking

The Optimal Combination

12 Facial Meat 8 hrs Weights

Why DOE?
Project Result
BEFORE AFTER

SO WHY DOE???

It Produces Great Results!!!

Design of Experiments An Introduction

The Fathers of Experimental Design

Sir R.A. Fisher - Experimental Design as applied to Agriculture George Box Devised the Response to Surface Methodology Genichi Taguchi Robust Parameter Design/Taguchi Method for Quality Engineering gained popularity in the Manufacturing Sector

Design of Experiments An Introduction

Some Applications of DOE - Reduction of Design Cost Speeding up Design Process - Reduction of Product, Material and Labor Complexities - Reduction of delivery Cost by Minimizing Process Variation

- Improvement of Overall Customer Satisfaction through the Determination of the Optimal Process/System Design
- Improvement of Turn Around Time of any Transactional Process (e.g. Payroll Process) - Reduction of Rework of any Transactional Process (e.g. Invoice Processing)

IMPORTANT DOE TERMINOLOGIES

RANDOMIZATION - Allocation of units to treatments by means of some random mechanism such as tables of random numbers, or the use of randomization devices such playing cards or dice. - The risks associated with random allocation (such as failing to obtain a representative sample in a survey, or having a serious imbalance in a key characteristic between a treatment group and a control group) are calculable and hence can be managed down to an acceptable level. - Random does not mean haphazard, and great care must be taken that appropriate random methods are used. REPLICATION - Where measurement is made of a phenomenon that is subject to variation it is important to carry out repeat measurements, so that the variability associated with the phenomenon can be estimated.

BLOCKING
- Is the arrangement of experimental units into groups (blocks) that are similar to one another. Blocking reduces known but irrelevant sources of variation between units and thus allows greater precision in the estimation of the source of variation under study.

IMPORTANT DOE TERMINOLOGIES

FULL FACTORIAL DESIGN - Each factor is tested at each level in every possible combination with the other factors and their levels. FRACTIONAL FACTORIAL DESIGN

- Is used to reduce the number of runs by evaluating only a subset of all possible combinations of the factors.
TWO-LEVEL FULL FACTORIAL DESIGN - Is used to simultaneously analyze the effects of multiple factors, each at two levels, on a numerical response.

2k Factorial Designs
Factors Levels Low - Response

A
High +

Low -
B High +

2k Factorial Designs
2k factorial designs are experimental designs for which there are k factors (or driver variables) and each of these factors will be investigated at 2 levels, high and low or, symbolically, + and -. All possible combinations of factor levels are used in the investigation. That is, if there are k= three driver variables, then the data that would be collected would be represented as follows:

2k Factorial Designs
The full factorial model when k = 3 is given by:
Y= 0 + 1X1 + 2X2 + 3X3 + 12X1X2 + 13X1X3 + 23X2X3 + 123X1X2X3 +

A replicated 2k design (r2k) would gather r observations under each of the 2k (factorial) combinations previously listed.

2k Factorial Designs
NUMBER OF FACTORS NUMBER OF LEVELS PER FACTOR NUMBER OF RUNS (FULL FACTORIAL)

2 2 3 3 4 4 5 5

2 3 2 3 2 3 2 3

4 9 8 27 16 81 32 243

6 6
7 7 8 8

2 3
2 3 2 3

64 729
128 2187 256 6561

23 Factorial Designs
X1 X2 X3 Y

+ + + +

+ +
+ +

+ +
+ +

Y1 Y2 Y3 Y4
Y5 Y6 Y7 Y8

DOE in a Call Center Operations

Company: Mortgage and Leasing Company

Problem Statement: The company implemented a new training and scheduling method in its two facilities in order to improve over-all service level. But after six months, it is observed that the average call waits are still higher than the industry standard. The system at each call center operates with 2 queues (channels), each consisting of 12 agents and 1 SME. A call distribution software automatically directs the calls to the agents and SME within a queue and agents can also route calls to the SME if a caller requires a more detailed answer.

DOE in a Call Center Operations

The Experimental Design and Analysis : The team conducted a brainstorming session and came up with the following factors: of Queues (One/Two): It was argued that a single open network queue would be more flexible and allow better agent utilization. 2.Type of Headset (Mono-Aural/Bi-Aural): The agents involved in the brainstorming session felt that the mono-aural headsets currently in use failed to cut out background noise, causing agents to take longer to understand and resolve customer issues.
1.Number

DOE in a Call Center Operations

The Experimental Design and Analysis : 3. Extra Specialists (No/Yes): The call hold times tended to be longer at the specialist end of the process, hence it was felt that adding an extra specialist would take the burden off the existing.

DOE in a Call Center Operations

The Experimental Design and Analysis : Since there are only 3 factors, the team decided to execute a full factorial experiment with the following considerations: One randomly chosen run applied per week to balance out variability in call volumes during the week (8 Weeks). Each combination run will be executed for a week and average call wait time per hour recorded. A single replicate of the experiment will be conducted at both centers, giving two replicates overall. The centers were designated as blocks to account for any between-center variability.

DOE in a Call Center Operations

The Experimental Design and Analysis : The design layout is shown below in standard order with the actual factor levels and response data:
Factors Std Run Order A: Queues 1 2 3 4 5 6 7 8 One Two One Two One Two One Two B: Headsets Mono Mono Bi Bi Mono Mono Bi Bi C: Specialists No No No No Yes Yes Yes Yes Center 1 23 42 10 30 20 24 10 14 Center 2 16 35 5 31 22 21 6 15 Response: Wait Time

DOE in a Call Center Operations

1. Using Minitab, choose Stat>DOE>Create Factorial Design.

Click 2-level Factorial, then in number of Factors Choose 3 since we have 3 factors understudy.

DOE in a Call Center Operations

2. Name Factors and Set Factor Levels. Choose Stat>DOE>Create Factorial Design>Factors.
In the Low/High Columns, youll initially see -1 (denoting the low setting/parameter) and +1 (denoting the high setting/parameter). Replace these with the actual setting of the factors.

Enter Factor Name and Type (Numeric or Text). In our example the type of data that were using are all text data.

DOE in a Call Center Operations

3. Randomize experimental run. Choose Stat>DOE>Create Factorial Design>Options.

Entering a base for the random data generator allows you to control the randomization so that you obtain the same pattern every time. This way you will get the same design order that is used in this sample session.

DOE in a Call Center Operations

4. View the Design and use this to run the actual experiment.

Experimental Run order based on Yates Order Experimental Run order based on Random Order Combination of the Different Factors

Enter Response of the Experimental Run

DOE in a Call Center Operations

5. Screen the Design. Choose Stat>DOE>Analyze Factorial Design.
Click, Graph then select Normal and Pareto. The significance level is set at 0.05, which is our acceptance criteria.

Select Wait time, this will automatically appear in the responses field

DOE in a Call Center Operations

6. Screen the design and Identify Important Effects.

Based on the analysis of factors, those main effects and interaction effects with P value < 0.05 are considered to have significant effect on the response, these are: Queues, Headset, Specialist, and, the interaction of Queues and Specialist.

DOE in a Call Center Operations

6. Screen the design and Identify Important Effects. (Cont.)
Pareto Chart of the Standardized Effects
(response is Wait Time, Alpha = .05) 2.36 A B
Percent
F actor A B C Name Q ueues Headset Specialist

Normal Probability Plot of the Standardized Effects

(response is Wait Time, Alpha = .05)
99 Effect Type Not Significant Significant A
F actor A B C Name Q ueues Headset Specialist

95 90 80

AC
Term

70 60 50 40 30 20 10 5 B AC

C AB BC ABC 0 2 4 6 Standardized Effect 8 10

-10

-5

0 Standardized Effect

10

The Pareto chart and Normal Plot use the same significance level of 0.05 to determine the significance of the effects. It clearly shows that factors Queues, Headset, Specialist, and, the interaction of Queues and Specialist are significantly active.

DOE in a Call Center Operations

7. Fit a reduced model. Choose Stat>DOE>Analyze Factorial Design.
In the include terms in the model up through order field, choose 2

Click terms to reduce the number of factors to the critical ones only as a result of the previous analyses of factors.

If there are non critical factors in the selected terms field, highlight these factors and click the arrow pointing to the left to transfer these to available terms field.

Ensure that only the critical factors are in the selected terms field.

DOE in a Call Center Operations

7. Fit a reduced model. Choose Stat>DOE>Analyze Factorial Design>Graphs. (Cont.)

Click graphs to determine fit of the new model

Uncheck Normal and Pareto Effects Plots. Select Four in one.

DOE in a Call Center Operations

8. Evaluate the reduced model.
99 90 50 10 1 -5.0 -2.5 0.0 Residual 2.5 5.0
Residual Percent

Residual Plots for Wait Time

Normal Probability Plot of the Residuals
4 2 0 -2 -4

Residuals Versus the Fitted Values

10

20 30 Fitted Value

40

Histogram of the Residuals

8
Frequency

Residuals Versus the Order of the Data

4 2 0 -2 -4

4 2 0 -3 -2 -1 0 1 Residual 2 3 4

Residual

1 2 3 4 5 6 7

8 9 10 11 12 13 14 15 16

Observation Order

Based on the analysis of factors of the reduced model, it is apparent that the following factors are indeed significant since the P values are less than the 0.05 alpha value.

The data plots for wait time adequately fits the model above, so it is safe to assume that we can find the optimum combination using the remaining factors that could potentially reduce call wait times.

DOE in a Call Center Operations

9. Draw Conclusions by displaying factorial plots. Stat > DOE > Factorial > Factorial Plots.
Click Main Effects Plot then Set-up. In the responses field enter wait time. In the selected field, ensure that main effect factors are selected. If the factors are initially contained in the available field, transfer these by highlighting the factors and selecting the arrow pointing to the right.

DOE in a Call Center Operations

9. Draw Conclusions by displaying factorial plots. Stat > DOE > Factorial > Factorial Plots. (Cont.)
Click Interaction Plot then Set-up. In the responses field enter wait time. In the selected field, ensure that those factors with interaction are selected. If these are initially contained in the available field, transfer these by highlighting the factors and selecting the arrow pointing to the right.

DOE in a Call Center Operations

9. Draw Conclusions by displaying factorial plots. Stat > DOE > Factorial > Factorial Plots. (Cont.)
Click Cube Plot then Set-up. In the responses field enter wait time. In the selected field, ensure that main effect factors are selected. If these are initially contained in the available field, transfer these by highlighting the factors and selecting the arrow pointing to the right.

DOE in a Call Center Operations

9. Draw Conclusions by displaying factorial plots. (Cont.)
Main Effects Plot (data means) for Wait Time
Queues 25.0 22.5 20.0 Headset

Mean of Wait Time

17.5 15.0 One Specialist 25.0 22.5 20.0 17.5 15.0 No Yes Two Mono-Aural Bi-Aural

The main effects plot presents the potential combination of factors that will lead to the reduction of call wait times. Possible combination of factors is One Queue, Bi-aural Headset, and Additional Specialist.

Interaction Plot (data means) for Wait Time

35
Queues One Two

30

Further analysis of the interaction plot shows that adding specialists in a two queue channel will reduce call wait time but in a single channel the effect is incremental

Mean

25

20

15 No Specialist Yes

DOE in a Call Center Operations

9. Draw Conclusions by displaying factorial plots. (Cont.)
Cube Plot (data means) for Wait Time
8.0 14.5

7.5 Bi-Aural

30.5

Headset

21.0

22.5 Yes Specialist No Queues Two

The cube plot reinforces the findings: the best setting for reduced wait times should be the SINGLE QUEUE AND BI-AURAL HEADSETS SETTING. An extra specialist doesnt add value to this setting, so we can reduce our cost by sticking to the existing number of specialists.

19.5 Mono-Aural One

38.5

DOE 101: What are the elements pertinent to your experiments analysis?
1.

2.

3.

4.

Design of the experimental run. You need to know the levels and factors of the experiment, number of trials to execute and how to randomly run the experiment. The analysis of the estimated effects and coefficient of the response. Factors with significant effect would have P value less than the alpha value which normally is pegged at 0.05. The pareto chart and normal probability plot. If in the pareto chart the standardized effects of the factors go beyond the line of the computed standardized effect (represented by a red line), it means that these are candidate to have influence on the response. The normal probability plot will show outright that these factors are dispersed against the diagonal line. Both plots utilize the acceptance level of 0.05. Evaluating the reduced terms. Analyze the estimated effects and coefficient of the screened factors, if the P values are less than the acceptance level of 0.05 then accept these factors.

DOE 101: What are the elements pertinent to your experiments analyses?
Evaluate the residuals of the model using the following: - Normal Probability Plot: residuals should be within the diagonal line. This explains that the difference of the actual response against the predicted response of the different factors are normally distributed. - Histogram: plot of the residuals should reflect a bell-shape or close to bell-shape distribution. - Residuals versus the Fitted Values Plot: Data plots should show a random scatter and have no pattern. - Residuals versus the Order of Data: Data plots should have no trend/pattern. If the factors meet the acceptance threshold and its data fits the residuals model, then the remaining factors are assumed to have an effect on the response.

DOE 101: What are the elements pertinent to your experiments analyses?
5. Decide on the optimum setting/combination of factors. Select factors and its corresponding levels that suffice the objective of the experiment. This can be done using the following plots: - Main Effects Plot: Presents (individual) factors that have significant effect to the response. Furthermore, it will provide the responses of the factors on varying levels (+, -). - Interaction Plot: Parallel line in an interaction plot means that there are no interaction between the factors understudy. The greater the degree of departure of the lines from the parallel state, the higher the degree of interaction.

- Cube Plot: Presents the means of the raw response variable data for each factor level combination. From here we can easily infer the optimum setting of the experiment.