Six Sigma

Experiment Design Tools
Experiment Design Tools
Most companies collect data to monitor the health of their processes, but most of this data is not acted on in any way because it does not contain meaningful information. Often, when these compa-nies decide to perform an experiment in an effort to produce more meaningful data on a specific CT characteristic, they design a onefactor-at-a-time test, but progress is slow and expensive using this technique. On the other hand, companies who learn to use Design of Experiments (DOE) are able to leverage their efforts by conducting well thought out experiments where several factors are altered simultaneously in one efficient series of tests. DOE provides a wealth of information and in a shorter period of time. The role of a statistically designed experiment is to identify the most influential factors (vital few) associated with a particular CT characteristic and to define their relationships using analytical quantities. Since interactions between various factors are also defined, a single DOE can yield many revealing facts allowing the experimenters to quickly improve their process. It is most effective to conduct DOE's early in the design process, experimenting with the effect of tolerancing and failure modes, so that when the product or service is introduced to the market place, customer satisfaction will be high and warranty claims minimal. The same experimental techniques can then be used throughout the life cycle to continuously improve, taking into account new information as it becomes available. There are two components to a DOE: Experimental Design and Analytical Procedure. It is best to think these out carefully before conducting the DOE in order to produce the best result. Depending on the level of understanding available about a process, there are various types of DOE that can be employed. Screening designs are typically used early in the process where many factors are involved, Characterization designs narrow the number of factors down to only a few, and Optimization designs focus on only one or two factors, but in much more depth. To ensure that a DOE is designed correctly, it is necessary to consider such topics as Yates standard order, factor levels using the –1, 1 notation, column contrast, balance, orthogonality, confounding and interactions between factors. Also, there are design factors that influence the time and cost aspects of the DOE, such as replications and the basic design type. The two-level two-factor design is relatively common in practice and makes a good starting point for most problems. When replications are taken into account, very meaningful results can be achieved with as few as 8 to 16 runs. When you don't know which factor has a substantial effect on a response variable, and you want to verify several factors, then the fractional factorial design is the preferred analytical tool. It is common practice to do a fractional factorial experiment when we have five factors or more. In this case, you only conduct runs which represent a portion of the matrix, because, for example, a 6-factor two-level experiment with no replications would involve 64 runs. The advantage, therefore, of a fractional factorial experiment is that fewer tests are required. When it comes to analysis of the data from a DOE, we must first evaluate the statistical significance. This is done by computing the One-Way ANOVA or the N-Way ANOVA (for more than one factor). The practical significance can be evaluated through the study of sums of squares, pie charts, Pareto diagrams, main effects plots, and normal probability plots. Note that there are situations where factors are statistically significant, but not practically significant. In any analysis, it is important to analyze the residuals prior to any conclusions being drawn. A step-by-step approach to designing and conducting any DOE must be adopted. The use of cross-

functional teams in the selection of factors to study, the design of the experiment, conducting the experiment, analyses of the data, recommendations, and implementation of DOE results all require careful planning and attention to detail to achieve success.

Key Questions
• • • • • • What is a "statistically designed experiment" and what roles do they play in Six Sigma? What are the guiding principles surrounding statistically designed experiments? How can the practical and statistical influence of a single CTP be exposed? How can the practical and statistical influences of two or more CTP's be exposed? How can conflicts between competing performance objectives be resolved? What is the process and guidelines for planning statistically designed experiments?

Key Questions A DOE is a systematic method where a number of factors are changed simultaneously, following a predetermined pattern to investigate their effect on the response. The role of a statistically designed experiment is to determine the most influential factors associated with a particular CT characteristic (vital few) and to understand their relationships using analytical quantities. The fundamental guiding principles surrounding statistically designed experiments are: Process Relationships, Replication, Randomization, and Experimental Control. The statistical influence of a single CTP can be exposed through hypothesis testing using tools such as OneWay ANOVA, and the practical influence can be exposed through the study of sums of squares presenting the results in pie charts or Pareto diagrams. The statistical influence of two or more CTP's can be exposed through hypothesis testing using tools such as NWay ANOVA, and the practical significance can be exposed with the use of graphs such as main effects and normal probability plots. Conflicts between competing performance objectives can be resolved by the use of interaction effects between factors in order to find the best compromise settings. The process and guidelines for planning statistically designed experiments are: Define the problem, set the objective, select response variables, select factor levels, conduct experiment(s), analyze results, and ensure results are well understood and communicated.

Why/When to Experiment
High Focus on defect detection and correction


Cost to deal with poor quality Focus on defect prevention

= f(X1, X2, …, Xn)  CTPs  Factors Improve

 CTQs or CTCs or CTDs


Low Product Design
Experiment Design Tools

Process Design



Why/When to Experiment At a planning session, Six Sigma team members are evaluating whether or not they should conduct an experiment on their process. They ask the Black Belt to outline a criteria for this. The Black Belt explains why and when to use experiments: • • • • • • • Experimentation can be used to improve the performance, quality and cost associated with key customer requirements. We use them to confirm and quantify the important inputs (CTPs) to our processes. Determine the "vital few" variables when there are no analytical equations to guide us. Experimentation can significantly reduce new product (or service) development cycle times and it is conceivable that manufacturing tolerances can be relaxed. A robust design can be created if we test it prior to design freeze. A product or service that is robust to variations in input factors and operating environment will cost society less over the life of the product, resulting in its customer loyalty. The need for product inspection and performance testing can be significantly reduced with extensive experimentation. We should always experiment as early in the cycle as possible, so as to improve quality, thus reducing rework, warranty claims and dissatisfied customers.

The Black Belt draws the above chart on the blackboard, summarizing his point that we should begin to test key business opportunities with a series of well planned experiments as early in the development cycle as possible in order to reduce the cost of poor quality, and in addition, throughout the product life cycle to achieve continuous improvement.

Fundamental Principles of Experimentation 1. In this example. using the example of the classical apple falling on the head of Isaac Newton. Replication allows a measure of experimental error. These variables will probably have small effects on the results." "Replication is the second principle of the experimentation process. etc. two to five replications are conducted for each set of conditions in order to ensure that the "white noise" is captured. The Instructor outlines the four basic principles. the energy imparted by the falling apple is the process of interest to Isaac Newton. He set out to design a series of experiments to determine the relationships between the mass of the apple and the fall distance as they relate to the energy imparted. wind velocity." . Usually. Understand Process Relationships • • • • • • • • Among process variables Among response variables 2. which will result in experimental error." "The fourth fundamental principle of experimentation is what is known as experimental control. and probably the most important principle". This simply means that you deliberately change some variables. a new Black Belt asks the Instructor to review the fundamentals of Experimentation. Randomization Select experimental units randomly Randomize run sequence 4. Experimental Control Active variation of process variables Control other sources of variation Fundamental Principles of Experimentation During Black Belt training." "The third characteristic of an experiment is to ensure that the order of conducting a series of runs is randomized in order to avoid false conclusions due to factors which may vary over the course of the experiment. Replication Increased precision of effect estimates Provide measure of experimental error 3. such as temperature. "is to fully understand the process. while carefully controlling all other variables as much as possible. says the Instructor. noting the effect. "The first.

He explains that a statistically designed experiment is a test or a series of tests in which purposeful changes are made to the input variables of a process or system so that we may observe and identify reasons for changes in the output response. that is. i.: white noise. guided by our judgement. The test results may lead us to select new factors which were dismissed or overlooked initially. The Master Black Belt concludes by saying that unless we experiment with our processes. we will never identify ways to improve those processes. The examples above imply that each factor can have independent effects on the result. Factor C affects the mean and the standard deviation C1 1_05_01_004 4. which we cannot control.the manipulation of controllable factors ( indepen dent variables ) at different levels to see their effect on some response (dependent variable). We would then design a new experiment. 1. We strive to find factors which have independent effects like this. Factor D has no effect Y Experiment Design Tools Y Role of Experimentation The Master Black Belt is asked to define the role of experimentation by the Champion. but this is not always possible. improved quality or some form of improvement of our CT characteristic. Factor B affects the standard deviation B1 A2 Y A1 B2 Y 3. Improve 5 C2 D1 D2 . Experimentation provides us with a sound.e. Factor A affects the mean 2. The objective in many cases may be to develop a robust process. we can then decide how to change the settings or levels of some factors in order to produce better yield. until we achieve the desired results. Once we have obtained the results of these tests. a process affected minimally by the many sources of variability. effective and economical method for determining the best way to change.Role of Experimentation Experimentation . and so on.

Also. He suggests that they review the FMEA's that had been completed recently by the team. Improve 6 . since these contain an excellent checklist of what is important to this particular process and act as a summary of the collective knowledge about the process from the brainstorming sessions. a team member asks the Black Belt if he could help the team select factors for an experiment on their process. each factor needs to be independently controllable. The team now felt that they knew how to go about selecting factors for their first experiment. The Black Belt points out that factors can come from a wide range of sources as shown in the list above and that it is impossible to select the most influential factors unless you fully understand the process.) or can be discrete (like new/old. pressure. etc. machine 1/machine 2. etc. The Black Belt also says that they need to be aware that factors can be continuous in nature (like volts.).Sources of Experimental Factors Input Factors (Xs) People Material Equipment Policies Procedures Methods Environment Output (Ys) Service PROCESS CTC A blending of inputs to achieve the desired outputs Product CTQ CTD Task 1_05_01_005 Experiment Design Tools Sources of Experimental Factors At a Six Sigma team meeting.

"The first step". he reminds the team that a review of the FMEA for their process may provide clues as to how to categorize the various factors. 2 to 5) so that the experiment can focus on only these. and which ones you can move to the controllable (but hold constant) category. The Master Black Belt draws the above chart on the board and describes a process he uses to ensure that all factors are considered when designing a DOE. and elect to hold constant for the experiment OUTPUT (Y) INPUTS (X’s) Factors that we wish to vary in a DOE • Experimental Error – Measurement – Other • Environment UNCONTROLLABLE INPUTS 1_05_01_006 White Noise PROCESS UNDER STUDY Response variable has variation due to: • Effects: – Factors – Interactions Factors that we cannot control • Environmental • Unknown Experiment Design Tools Types of Experimental Factors The team realizes that there are different types of factors involved in their process – some controllable and others uncontrollable. Of all the possible inputs that could be used in a DOE.e. Before beginning. by the time this exercise is finished. resulting in as few runs as possible.Types of Experimental Factors CONTROLLABLE INPUTS Factors that we can control. They ask the Master Black Belt for advice in how to handle these. the experimenters will already be more knowledgeable about their process and be ready to begin the design of their DOE. He pointed out that. What is "Design of Experiments" (DOE) Definition Improve 7 . or recording of data that would otherwise have been ignored or overlooked. especially ones that could be varied." The Master Black Belt goes on to say that the input factors remaining are likely to be few (i. the factors that cannot be controlled are important to identify because there may be ways that their effect can be reduced by careful design of the experiment. or just because it doesn't make sense. but are not likely to be allowed to vary for certification reasons. says the Master Black Belt. "is to divide all known factors into one of the three categories as shown in the chart. Also. you need to decide which ones you wish to vary in the experiment. It may be time well spent to put as many factors into this category as possible.

He explains that a DOE is a way to generate information about a process which can then be used to improve it. to investigate their effect on the response or output. active variation of a process to identify better settings or procedures to improve the product quality.General The controlled. The Black Belt outlines other features of a DOE: • • • • • It is a way of separating the vital few from the trivial many factors. carefully recorded. he uses the above definition. It is a systematic scientific approach which allows decisions to be based on facts. What is "Design of Experiments" (DOE) When the Black Belt is requested to define the concept of a DOE to the Champion. It is a structured way to economically improve any process. this definition applies equally to transactional. following a predetermined pattern. As can be seen. I am interested to see if I can apply this tool in the Customer Services area". and a procedure for analyzing the data once it has been collected. the DOE process consists of a series of experiments where competing settings of each factor are combined in order to efficiently reach valid and relevant conclusions. engineering design. testing many factors (and interactions) in a short time. Specific A DOE is a systematic method where a number of factors are changed simultaneously. or to manufacturing processes. It is dynamic. In conclusion. The Black Belt explains that a DOE must have two basic elements – a design where the team decides what to focus on. . It is a planned set of trials (runs) with specific measures. The Champion says: "Tell me more.

you will have optimal water temperature which represents the response of the process. raw material. meaning that four combinations are possible to get results". "We now can prepare the design matrix with factor Flow Rate of Cold Water and Flow Rate of Hot Water. parts. let's suppose that the knobs can be set only at high or low levels. The process is quite simple. Improve 9 . "This is a very simple example that we've done on the board and it represents a part of the planning of a DOE. "Then we 2 have a two-level. etc. why don't we use this tool more often?" asks a Champion. By adjusting hot and cold knobs." "In most industries. we will use the shower example. conducting an experiment requires human and material resources. In this case. Who wants to volunteer?" asks the Master Black Belt. The two knobs represent the two variables or factors of the experiment. X2) Water Temperature = f (flow of hot water. money.e." "It seems to be very easy. two-factor matrix. 2 . Is it obvious to determine which are the best settings? The experiment itself still needs to be done. flow of cold water) Experiment Design Tools The Shower Experiment During a refresher session on Design of Experiment (DOE). time. "Of course. i.The Shower Experiment Flow Rate of Run 1 2 3 4 Cold Water Low High Low High Flow Rate of Hot Water Low Low High High 1_05_01_008 Y = f (X1. "Remember. a Champion asks a Master Black Belt: "Do you have a simple example to show us?". each DOE should be carefully planned with all the required resources" concludes the Master Black Belt. In order to get worthwhile results. we use two levels to do an experiment.

Experimental Design: A means to structure the variables for experimental manipulation. • • • • • • • • • Fixed Effects The experimental levels are selected by the team. Mixed Effects A combination of fixed and random models. 2. Analytical Procedure: A means to extract relevant information from the experiment. This can be broken down into three types of effects. He summarizes the questions that one needs to ask as shown above and explains the two components as follows: 1. Conclusions can be generalized to the population. Analytical Procedure • ANOVA • Regression • Etc. Conclusions apply only to selected factor levels. Improve 10 How sophisticated does the analysis have to be? Experiment Design Tools Components of a DOE In his presentation describing the various features of a DOE.Components of a DOE 1. • Analysis of variance (ANOVA) • Regression • Find a relationship between variables. Experimental Design • Fixed Effects • Random Effects • Mixed Effects Questions: What result do we expect to get? What are we going to measure? How will we collect the data? How will we present the data? Questions: How will we analyze the data? How will we find relationships? How will we find interactions? 1_05_01_009 2. . the Master Black Belt feels he should go into more detail. Random Effects Levels are randomly selected from a population. Most experiment designs are of this type.

The Black Belt illustrates this with a two-factor example. A much better approach is what is termed a "full factorial design" where a few factors are varied at the same time. One may conclude that this process defines the optimum settings. He explains that by varying the temperature (factor A). this becomes more pronounced when more variables are involved. in the majority of cases. the pressure (factor B) is varied to determine its best response (2 above). As one can imagine. we use "Fixed Effects" designs and ANOVA for the analysis procedure.In conclusion. This approach includes interaction effects which can be very important in some cases. and would be missed completely in the "one-factor-at-a-time" approach. Why Not One-Factor-at-a-Time? The autoclave operator insists that he should not vary more than one parameter at a time. Benefits of Statistical Experiments Change One Evaluation criteria Time to get results Cost Chance of detecting optimum Confidence in results Ability to identify independent interactions and main Expert Observe Very Long Very High Very Low Very Low Long High Medium Medium Long High Overall High Overall Suboptimum likely High Guess Opinion Everything Factor Stat at once At-a-time Long Short Rel Low Very Good High DOE Low Low Moderate Moderate Moderate Very Low . He also points out how focusing on "onefactor-at-a-time" will probably miss the optimum combination of factors. fearing that the results would be impossible to interpret. explains that since there are so many factors (potential CTPs) involved in even simple processes. a zone of maximum response is determined (1 above). trying to convince him otherwise. in random order. it would be very time consuming and expensive to do a series of experiments "one-factor-at-a-time". but if we overlay this picture onto the real situation (3 above). the Master Black Belt states that. to yield a matrix of effects. the so-called optimum is far from defining the maximum potential value (4 above). Then by keeping temperature constant. They are anxious to try a DOE because they feel that there are significant improvements that can be achieved. The Black Belt sits down with the autoclave operator and the Six Sigma team to design such an experiment. The Six Sigma Black Belt.

low cost. The above table is the result of his efforts. . for all criteria selected. Using a subjective scale. The cost of a DOE is much lower than the cost of poor quality (long-term). In addition. which may not be possible otherwise. they do cost money and disrupt the process. not replace it. He decides to map the various techniques and to identify their strengths and weaknesses compared to statistical experiments. so they must be properly planned. the statistical experiment is far superior than any of the others. one can see that. in the short-term. Properly designed experiments will permit simple interpretation of results. Proper experimental design is considered to be more important than sophisticated statistical analysis. However. statistical experiments are the only technique to provide the data for "main effects" and "interactions" between various factors. After creating the table.effects None None None None None Yes Conclusion: Statistical Experiments are Rich With Information!! Benefits of Statistical Experiments A Black Belt observes over the course of the past year that there is a lot of folklore about the benefits of observation. judgement and "one-factor-at-a-time" as techniques for deciding how to make a change to a process. We use experiments to supplement our judgement. optimal results with a high degree of confidence. the Black Belt makes some notes in the way of conclusions: • • • • • • Statistical experiments provide fast.

"Good question. the students are not familiar with these. The Black Belt in training asks: "Does this mean that we will no longer use the traditional types of experiments?"." replied the Instructor.Types of Experiments Very Informal • Trial and Error Methods – Introduce a Change and See What Happens – Produced Under Controlled Conditions • Pilot Runs – Set up to Produce a Desired Effect • One-factor-at-a-Time Experiments – Vary One Factor and Keep All Other Factors Constant • Planned Comparisons of Two Methods – Background Variables Considered in Plan – Study Separate Effects and Interactions • Experiment With 5 to 20 Factors – Screening Studies 1_05_01_012 – Modeling. etc. He said that once everyone had some experience with the Six Sigma approach. a Black Belt in training asks how the traditional approach that he had been familiar with compares to the Six Sigma approach. Improve 13 Very Formal • Comprehensive Experimental Plan With Many Phases Six Sigma Approach • Experiment Planned With Two to Four Factors Traditional Approach • Running Special Lots or Batches . Optimization Experiment Design Tools Types of Experiments After having some exposure to the basic concept of a DOE. Multiple Factor Levels. At this point in the course. The Instructor replies that they will still be used on occasion – a special lot or a pilot run could still be used on occasion if you were wanting to check the data collection process for calibration purposes. "let me write out what I'll call the "hierarchy" of experiment types. they would readily see that it gives much better information than the traditional approach and that the new tools would be easier to use." He writes down the first four headings and everyone agrees that they know what these involve. but they are amazed to think that up to 20 factors can be dealt with within one experiment. He went on to write the headings that describe the Six Sigma approach.

Usually. Such designs are usually based on full factorial models using two or three levels. The Black Belt points out that the above class descriptions are very general in nature and should not be used as the sole benchmark when designing an experiment. we work with screening designs. putting the "trivial many" aside. When a team member asks the Master Black Belt to give an overview of how the various classes of DOE are applied.Classes of DOE Screening Characterization Optimization A Factors C F I E B K J D F B K J F K H G Time 1_05_01_013 Experiment Design Tools Classes of DOE There are several classes of DOE that can be applied to improve a process. Improve 14 . he draws the above chart and elaborates on each as follows: • • • Screening designs are generally used to identify the "vital few" process factors. and rarely with optimization designs. Characterization designs are generally used to study simple effects involving a small number of factors (3 or 4). Optimization designs (also referred to a modeling designs) are used to study complicated effects and interactions involving one or two factors. He also points out that. In general. In practice. such designs are employed to yield a precise mathematical model of the process under investigation and may involve several levels (to capture nonlinearities). being tailor-designed for the particular factor and levels being investigated. each experiment is different from the ones before. such designs involve a large number of factors and are limited to only two levels. more than one experiment is generally required to obtain a meaningful model. Of course. occasionally with characterization designs. the results of one being used to design the next. most often.

new.Terms Used in DOE Process Factor Variable A B C Coded Value +1 -1 +1 -1 Setting Pressure High (40psi) Low (20 psi) Machine Shift New Old Number 1 Number 2 Y = f (A.3 8. and that later on we will see how to arrange this data so the experiment can be conducted. old. Low Level – Low setting (e. High Level – High setting (e.4 6. each team member has a clear understanding of the design structure of a DOE and begins to plan an experiment that applies to their process. B.7 5. shift 2. Data – The result – i.g. the Black Belt presents the above chart which represents the design anatomy of a typical experiment.). or "an invitation for an information event to occur". CTD.7 9.: 20 psi. the magnitude of the response variable.).2 3. C) Factors Treatment or Combination Process Variables B C -1 -1 +1 +1 -1 -1 +1 +1 -1 -1 -1 -1 +1 +1 +1 +1 Response Variable 5. Treatment Combination – Combination of levels for this element of the experiment. shift 1.1 7. etc. Factor – Process variable that will be purposely altered during the experiment. CTC). which he does as follows: • • • • • • • • • DOE – Design of Experiment. Level – Factor setting. Order – Sequence order or run order for the experiment. From this brief description. etc. He is asked to review the primary terms. Response Variable – The subject of the experiment (CTQ.3 10.e.5 +1 -1 Sequence Order 1 2 3 4 5 6 A -1 Order +1 -1 +1 -1 +1 -1 +1 Data 1_05_01_014 Levels 7 8 Experiment Design Tools Terms Used in DOE As a means of introducing the topic of DOE to the Six Sigma team. The Black Belt points out that the matrix shown is in the form convenient for the design of the experiment.g. Improve 15 .: 40 psi.

7).0 Y 5 3 ∆=2 AB+ Condition A+ 12 ∆=7 Main Effect 1_05_01_015 AExperiment Design Tools A+ Main Effects and Interactions Before beginning to design their experiment. the team leader asks the Black Belt to explain what is meant by "main effect" and "interaction". the yield (Y) of the process is over three times better than when B is low. i. Likewise. The Black Belt uses the above example to describe the two terms. The interaction effects can be used to achieve the desired result.Condition Run 1 2 3 4 A -1 1 -1 1 #1 B -1 -1 1 1 #2 A*B 1 -1 -1 1 Interaction Effect Y 3.e.: when B is high. Main effect describes the direct effect of factor A on the response variable (Y).0 12.0 5.Main Effects and Interactions Y B. Interaction effect describes the effect of the product A x B on the response variable (Y) when two factors have a combined effect on the response variable. Improve 16 5 . B is another main effect. The two charts on the right hand side above show the interaction effect. The Black Belt points out that one of the powerful advantages of a well designed DOE is that it has the benefit of being able to identify interactions which would be totally missed in a one-factor-at-a-time experiment.0 5. Factors A and B are generally assumed to be totally independent of one another (2 vs.

It is just a special way of making a comparison. -1) called coefficients in a design matrix. the contrast of one factor or interaction is the result of the sum of the coefficients multiplied by the response of each run. The coefficients of a factor or interaction are used to calculate its contrast. that the response variable is greater when the factor is set at its low level (-1)." "As you can see on the visual.0 B -1 -1 1 1 AB 1 -1 -1 1 Y1 1.e.Definition Contrast of factor F = ΣΥ F+ . actually contrast is an intermediate calculation used in statistical analysis of a DOE.5 4. Then contrast compares two things or groups of things by comparing their difference.5 * 1.5 4.5 13. a Black Belt asks: "You mentioned the word CONTRAST in your explanation. which we will learn later.5 = -4. A negative result means the inverse. we can say that a column contrast is the linear sum of vectored responses. etc.5 = 4.5 13. a positive contrast means that the response variable is greater when the factor is set at its high level (+1). are replaced by coded values (+1.5 * 4. To define a contrast." "Is it then accurate to state that the term contrast is used for more than one thing?" asks a Black Belt." "Thus. Improve 17 . for a main effect.5 * 13.5 4. The Master Black Belt responds: "Actual factor settings for an experiment. compared to the low level (-1).Contrast .0 Example Computation Run Order 1 2 3 4 1_05_01_016 A -1 1 -1 1 12.5 = 13.ΣΥFVectored Vectored Column Column Response Response A -1 1 -1 1 Y DESIGN MATRIX Run Order 1 2 3 4 (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y 1. 80 psi. and it also defines a set of coefficients used in a design matrix" concludes the Master Black Belt. a set of coefficients have to conform to conditions known as "balanced" and "orthogonality". High. but I don't understand what it means". such as Low.5 = -1. i. "Yes. compared to the high level (+1).5 12.5 4.5 (1) a b ab Contrast Column Contrast Sum of Vectored Responses Experiment Design Tools Contrast – Definition During a training session on Design of Experiment (DOE). 120 psi. In other words.5 * 4.

This feature ensures the effects are independent. When the sum of each column equals zero. The Master Black Belt says: "Well. and the sum of the coefficients of AB contrast is also equals to zero. this condition means that for one column. In a case of a twolevel design. To be considered as a contrast. but do you know what these conditions or properties are"? A Black Belt responds: "Balanced and orthogonality. Actually.Contrast . we see that the AB interaction is created by multiplying the coefficients of factor A by the coefficients of factor B. but I don't have a clue as to what these mean". For every pair of contrasts. Improve 18 . Minitab software will automatically provide a balanced and orthogonal designs" concludes the Master Black Belt. the sum of a set of coefficients must equals zero. small effects will not be washed out or masked by large effects. We have factors and interactions between factors.Properties DESIGN MATRIX Run 1 2 3 4 A B AB 1 -1 -1 1 -1 -1 1 -1 -1 1 1 1 Balanced ΣX i = 0 for each factor sum 1_05_01_017 Orthogonal Experiment Design Tools Σ Xi X j = 0 for all dot product pairs Contrast – Properties During a training session on Design of Experiment (DOE). Because all effects can be independently estimated. a Master Black Belt says: "We know that a column of a design matrix is considered as a contrast under certain conditions. then the design matrix is said to be orthogonal. take a design matrix for example. when the sum of the product equals zero." "When you closely look at the picture. there is the same number of High (+1) and the Low (-1) levels. This is the basic condition to have contrast. you don't have to build orthogonal contrast. the experiment is said to be balanced.

the list of runs would be increased by 8 and the whole list would then be randomized. If we wanted to have three replications instead of two. if some unknown factor is changing continuously over the period of the experiment.Run Order for a DOE Randomized RunOrder 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1_05_01_018 Design Order 16 15 10 11 3 5 13 8 12 11 4 6 2 9 7 4 Factor A 1 -1 1 -1 -1 -1 -1 1 1 -1 1 1 1 -1 -1 1 Factor B 1 1 -1 1 1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 Factor C 1 1 -1 -1 -1 1 1 1 -1 -1 1 1 -1 -1 1 -1 15 16 Experiment Design Tools Run Order for a DOE Once the factors and levels to be investigated are identified. The original design order is given in the second column. The use of Minitab makes it easy to lay out how to actually run the experiment." Improve 19 ." A team member asks: "Why is it necessary to randomize the run sequence?". its influence will be minimized by this technique. with two replications. In other words. two-level experiment. The Black Belt responds by saying: "Running the experimental trials in a random order protects against the effects of unknown nonexperimental variables (like environment) that might bias the results. For a three-factor. the above table shows how the run sequence would appear. The Black Belt states: "You should note that the original order from the design matrix is now randomized. you must then decide how many replications you wish to do.

"can you help us with a criteria to do this?". Improve 20 . Select step sizes to ensure not missing the optimum." said the Black Belt. use ± 3 standard deviations to set levels. "but remember." The Black Belt outlines the important aspects as follows: • • • • • • • • • Draw on all past experience to provide knowledge of your process.high 400 Y True Effect True Effect 500 A-high 300 30 B-low 20 10 0 A-low 200 B difference A difference 100 600 Factor Settings Y 1_05_01_019 Experimental Effect Lo (-) Hi (+) Lo (-) Hi (+) Experimental Effect Factor Settings Factor Settings Experiment Design Tools How to Select Factor Levels A Black Belt makes the statement that selecting factor levels is a critical step in determining the success of a DOE." said the Champion.: no one dominant). and safe. A good rule of thumb is 20% beyond input spec limits. reducing to narrower steps in subsequent experiments. If no spec limits exist. Levels for each factor should be balanced in relation to each other (i. well beyond white noise or measurement resolution. Initial levels should be bold or wide. The intent is to capture a large meaningful result. "If that's true. it is much better to plan a series of experiments from the beginning than it is to try to do it in only one. Levels need to be logical. rational. You should plan to spend only 25% of your allotted time and budget on your first experiment.How to Select Factor Levels 80 Response Variable (Y) 70 60 50 40 B.e. practical. "Yes. Use a pilot run to verify sensitivity to level selections.

we will be able to draw meaningful conclusion from the collected data" concludes the Black Belt. the Black Belt proposes to conduct a Design of Experiment (DOE). the effect due to "quality of paper" will be confounded or mixed in with the printer in use. After a rigorous study. During a coaching session with a Master Black Belt." The Black Belt asks the Master Black Belt for advice. Establish the Objective Improve 21 . Define the Problem 2. we will be able to differentiate the contribution of each paper type in the response variable. we did the design matrix A initially. assuming that the influence of all other background variables have been adequately controlled or otherwise. the factor "quality of paper" will be analyzed. but in this case it shows all low settings for printer 1 and high settings for printer 2. I see that by doing this. neutralized" says the Master Black Belt. "Yes. "If we design a new matrix as shown in B with both types of paper in both printers. The Black Belt says "We want to know the effects of the "quality of paper" on printing quality. In other words. Strategy of Experimentation 1. we hypothesized that an increase in "quality of paper" increases the quality of printing. "What is the objective of the experiment?" asks the Master Black Belt. In this case. I fear that the differences in the response variable due to "quality of paper" won't be separated from the unique effects of the printer.Confounding Effects MATRIX A Printer 1 Paper Type Low Paper Type Low Paper Type Low Paper Type Low ST AN DA RD Printer2 Paper Type High Paper Type High Paper Type High Paper Type High Coded values -1 1 Printer #1 #2 Paper Low Quality High Quality MATRIX B Run 1 2 1_05_01_020 Printer 1 -1 1 -1 1 Paper -1 -1 1 1 3 4 Experiment Design Tools Confounding Effects A Black Belt is working on a project to minimize the number of rejected reports due to bad printing quality that occur with the two main printers located in the computer room. To test this theory.

we select the independent variables that we want to test. Select the Response Variable(s) 4. Achieve the Objective "The purpose of an experiment is to better understand the real world.3. Draw Conclusions 10. and when properly planned or designed. we should define and therefore state the problem. Fractional Factorial. He describes all the advantages and steps to conduct a valuable experiment. Select the Experimental Design 7." William Diamond IBM – Retired Statistician Strategy of Experimentation A Champion asks the Master Black Belt to list the general steps involved in conducting a Design of Experiment (DOE). but what are the difficulties or obstacles that we should be aware of before using it?" "This is a good question". The objective is to reduce the PPM defective to avoid rework." "When do we start collecting data?" asks a Champion. Barriers to Effective Experimentation At the Annual Six Sigma Symposium. Analyze the Data 9. and as result we are producing a large amount of rework. replies the guest speaker. Choose the factor levels 6. Ask why there are negative perceptions in your organization? You are deploying efforts but do you adequately communicate your plan and progress to your organization? Think about it". the X1. I would like to assist you in conducting one experiment. At this phase. which may not be possible otherwise. During the question period." "All of these preceding steps constitute the planning part of an experiment. we are ready to select appropriate levels of each factor knowing what would make sense for our process. after the Measure and Analyze phases. so I will learn more about this tool and be able to provide more assistance to the Black Belts"." says the Champion. "let me show you a list of the main barriers to effective experimentation. As we can see. "The selection of the experiment design. "you seem to be very comfortable with this subject. X2. eliminating some of these barriers is within your reach. the mean of the process under study is not on target or there is too much variation. . it will permit relatively simple interpretation of the results. "Well. Select the independent factor(s) 5. For example. An experiment design is the deliberate arrangement of competing settings among one or more independent variables. XN." "I'm impressed. a guest speaker presents the powerful tool "Design of Experiment" (DOE). …. not to understand the experimental data. Now. We then assign these levels -1 and +1 notation.e. Why should we conduct a DOE if the problem definition or the objectives are not clear? Some of these barriers may be attributable to a lack of training and information. If you don't mind. Collect the Data 8. "One more step is required before collecting data" replies the Master Black Belt. etc. or the structure of the experiment i. Full Factorial. an Black Belt asks: "I see that there are many advantages to using this tool.

C. If. Factor A."As in any organization. says the guest speaker. we can initiate change even in an environment of excessive bureaucracy.C. the type of gasoline had been more significant. • • • • This one relatively simple experiment produced a wealth of knowledge. 1_05_01_023 Best Factor Level 100 Km/hr Windows Closed/A. one needs to include the extra cost for super grade to determine if it is still an advantage. With perseverance. What probably really matters is minimizing cost. A good plan. Off 30psi Off High Octane Closed Cruise Speed Tire Pressure Windows Air Conditioner Type of Gasoline DE. Windows * A. because of a lack of time and because of other priorities. I insist on the importance of carefully planning your DOE. The actual order of importance of the various factors is contrary to his original thoughts (order in the first box above). He makes the following points as a way of illustrating the various aspects that need to be considered in any DOE. During a training session. for instance. D. he presents the results to a class of Black Belts in training. E. Tire Pressure D. Cruise Speed C. Windows Level 1 (-) Regular 100 Km/hr 20psi Open Off Level 2 (+) High Octane 130 Km/hr 30psi Closed On Measure : Litres/100 Km. The Car Mileage Experiment Experiment Design: Objective: How to improve fuel economy on the highway.C. so that they will start to learn. which is adequately communicated may prevent negative perceptions and promote management's active involvement. You have to involve them in the realization of a DOE. not maximizing fuel economy. An interaction between the windows and the air conditioner proves to be the second largest effect (not even imagined). But once again. My final message to you is DON'T GIVE UP". E. Type of Gasoline B. A. management may not always be able to provide you with adequate support and coaching. understand and promote the use of these powerful tools. Experiment Design Tools The Car Mileage Experiment A Master Black Belt conducts a DOE on his automobile to try to understand which factors are important contributors to fuel economy. Improve 24 . Air Conditioner Results: Factor (in Order Of Importance) B.

If the factor is discrete. Acme. of course.87233 1.20943 ab 1.12497 1.70582 1. for instance. is significant. To make this evaluation he randomly selects 15 printed wiring boards and randomly assigns them to the three paste types.17466 0.99265 1.70582 1.27879 .e.88709 0.06742 0.00341 Level 3 Twecho 1. Our factor. is the paste.99265 Fa ctor 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 it0. Improve 25 Level 1 Wilsons 0. After curing he subjects each board to a tensile test to quantify the adhesion of the different pastes. At a team meeting the Engineer gave the team some general information about the single factor analysis which he was conducting. one needs to ensure that the ability to measure fuel mileage is accurate enough to differentiate such small effects (i.88709 Level 2 Acme 0. It must be measured on an interval or ratio scale.79582 1. The independent variable is commonly known as a factor.27879 Factor Response 1.27382 1. Following an analysis of the problem a cross-functional team concludes that the primary failure is due to low adhesion of the circuitry.27382 1.00341 1. After careful consideration of the evidence the team hypothesized that the problem is related to the type of solder paste used during production. Continuous factors must be classified into subgroups or levels. One Factor Analysis Re sponse 0.79582 1. Three types of paste are available.70682 1.70682 1. "The dependent variable is called the response.63187 1. An Engineer on the team is tasked with evaluating the pastes to determine which one is the best.63187 1.• • Before concluding that the type of gasoline. in our case the measurements will be in pounds. and Twecho.12497 1.06742 1_05_01_024 Experiment Design Tools One Factor Analysis A printed circuit board is related to high field failure rates. Wilsons. how will you analyze this data?" one team member asked. as in our case. There are generally many other factors involved.61013 1.: is measurement error causing a false conclusion).61013 1. In all cases the factor must be treated as a categorical variable." "Now that you have defined the factor levels and have measured each response.17466 M 0.20943 1. further classification may not be required as the subgroups naturally exist. These must be kept constant as much as possible in order to obtain valid results.87233 in 1.

01 0. between and total. These values expose the statistical influence of the different levels. This report also generates a P value which also indicates that there is a statistically significant difference (P < 0.33). five for each of the different types. the Engineer calculated the sum of squares within (SSW). He calculated a ratio of the SST compared to SSB and the SSW and found that the SSB amounted to 75% of the variation and SSW was attributable to 25% of the total variation." Improve 26 ct ra P i l ca 0.64 + 0.63187 1.5064 Error 12 0.000 a St ti al ic st 1_05_01_025 Experiment Design Tools Sum of Squares as Related to One-Way ANOVA An Engineer was evaluating three brands of soldering paste used on printed circuit boards. He reviewed his findings with the Black Belt.0278 Total 14 1.126 1. Based on the SS values a mean square (MS) ratio." The Black Belt entered the data into Minitab. To do this we will enter the data in Minitab.12497 1. He concluded that the difference in the levels was an appreciable influence of variation.143 Level 3 Twecho 1.79582 1.016 0.24 P 0. By combining the response data for all of the levels he calculated SST. The SSB represented the source of variation or the variation attributable to the factor under study.036 0.0127 0.87233 1.126) he established the SSW (0. Fifteen tensile strength measurements were taken. which is the F value. The practical can then be displayed using charts such as a pie chart.35 0."We will look at the influence of the different levels both practically and statistically.70682 1.27879 F 18.88709 0.99265 1. The practical influence of different levels can also be exposed using Minitab by taking the SS and calculating a ratio as we did in Excel.00341 0.05) in at least two of the means differ from the rest. By adding the SS for each of the levels (0.064 75% 25% Level 2 Acme 0." Sum of Squares As Related To One-Way ANOVA Analysis of Variance Source DF SS MS Factor 2 1. and total (SST). "The one-way ANOVA also calculates the SS within. To better understand the implications of the differences you have uncovered we need to look at the data statistically. is derived. analyze the sum of squares. conduct a one-way ANOVA. To estimate the practical significance. Next he subtracted the SSW from SST which resulted in the SSB. After reviewing the Engineer's analysis the Black Belt remarked. "Your analysis has revealed the practical influence of the factor under study.17466 0.70582 1. stack it.032 Variance Column SS Column SS Between SS Within SS Total (Source) (Error) (Total) .3331 0.27382 1.143 + 0.06742 0.61013 1.3458 Level 1 Wilsons 0. between (SSB). this represented the error.33 1. and do a residual analysis. completed a one-way analysis of variance (one-way ANOVA) and reviewed the results with the Engineer.20943 1. create a main effects plot.

One-Way ANOVA Table An Engineer is working on a project to maximize the tensile strength of the bond of circuitry on a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. The Engineer was unsure how to analyze the data and asked the Black Belt for help. After reviewing the data, the Black Belt suggested two possible analysis tools, the t test and the one-way ANOVA. "In Minitab we can run both of these tests. However, ANOVA has the advantage of comparing various levels, while the t test makes paired comparisons. It appears that the Twecho paste is producing a significantly higher tensile strength." "How do you know this?" the Engineer asked. "The one-way ANOVA output in Minitab produces a graphical representation of the mean confidence intervals and an F statistic and a P value. We can see that the confidence interval around the mean of the Twecho paste is completely outside the range of the confidence interval of the other two paste. The use of ANOVA has some underlying assumptions of the data; the population variances of the response are equal across all levels for a given factor, and the residuals are constant and normally distributed with a mean of zero and a constant, but finite variance." "How do I analyze the residuals?" the Engineer asked. "This can also be done in Minitab with the residual analysis tool."

One-Way Main Effects Plot

Level 1 Wilsons 0.99265 1.06742 0.87233 1.17466 0.88709

Level 2 Acme 0.79582 1.12497 1.27382 1.20943 1.00341

Level 3 Twecho 1.70682 1.63187 1.70582 1.61013 1.27879

Main Effects Plot - Means for Response











Experiment Design Tools





One-Way Main Effects Plot An Engineer is working on a project to maximize the tensile strength of the bond of the circuitry to a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. To analyze the data the engineer created a one-way main effects plot. On the chart he depicted the average response for the three different levels, that is, the three different types of paste. Having completed the analysis the Engineer presented the results to the project Black Belt. "As you can see by this main effects plot the paste at level three provides us with the maximum tensile strength of the three types; this is the type that will be specified from now on." "This looks very good and seems to support the selection that you have made. However, do you have statistical evidence to support this selection?" The Black Belt asked. "Statistical evidence?" The Engineer asked. "The main effect plot is a very good graphical tool to help substantiate your conclusions; however, it should be used to compliment other tools. With this tool alone I can't be sure that the variation presented in the plot is significant. I need to know with a certain degree of confidence if the differences between the averages could have happened as a result of chance random sampling variation or not. To do this we need to first use tools such as one-way ANOVA."

Residual Model Diagnostics
Normal Plot of Residuals
4 3 2 1 0 -1 -2 -3 -4 -5 -2 -1 0 1 2

I Chart of Residuals






-10 0 5 10 15

Normal Score

Observation Number

Histogram of Residuals
4 3 2 1 0 -5 -4 -3 -2 -1 0 1 2 3 4 4 3 2 1 0 -1 -2 -3 -4 -5 10

Residuals vs. Fits









Residual Analysis An Engineer was performing a one-way ANOVA comparing three different types of soldering paste. The

Engineer remembered from his training that one of the underlying assumptions of the ANOVA is that the residuals are random and normally distributed. However, he did not know how to verify this. He asked the project Black Belt for help. The Black Belt explained. "When you are doing your one-way ANOVA in Minitab you need to store both the data's fits and residuals. This data is then used to construct a residuals plot." Together they generated a Residual Model Diagnostics plot. The Black Belt reviewed the plot and proclaimed, "Very good, it appears that your assumptions have been meet." "How can you tell?" the Engineer asked. "The two left charts, the Normal Plot of Residuals and the Histogram of Residuals, both give an indication of the normality of the residuals; both look normal. The I Chart for Residuals gives us an indication of change in the residuals through time. This chart implies that the points are randomly distributed through time; that is, the points are evenly distributed above and below the central line, most points fall near the central line, some points approach the control limits, and no points are beyond the control limits. The last chart, Residuals vs. Fits, gives us evidence of changes in residual variation at the different levels. Each group appears to be similar in variance."

The Tw o-Level Tw o-Factor (2 D esign - 4 R epresentations
G eom etric


Low Interior Space

A High


Low B

(1) b

a ab





A lgebraic A
High Run O rder



1 2 3 4



Experim ent D esign Tools

The Two-Level Two-Factor (22) Design – 4 Representations "A two-factor, two-level design can be represented in at least four different ways" says a Master Black Belt to a Champion. In the "Geometric" representation, the corners of the square represent the levels of factors A and B, from "lowlow", to "high-high". The "Standard", or Yates designation (in the upper right image), shows the notation of all treatments taken from the treatment combination. "a" indicates "A high" and "B low", "b" indicates "B high" and "A low". "ab" indicates that both factors (A and B) are high. The notation used in the Yates designation refers to the factor set at the



-1 +1 -1 +1

-1 -1 +1 +1

we always identify individual trials with –1 or 1 . called "Algebraic" form is used to represent the settings used for each run (from 1 to 4). which refers to both factors (A and B) being set at the low level. a factorial experiment with k factors. the 2 factorial designs. i. Thus if there are two factors at two levels. has widespread industrial applicability. labeled A and Improve 31 . I have just read a book where the author is using another designation. Is there any difference and advantages in using one compared to the others?" k The Master Black Belt replies: "Remember. different notations are used. and is the most common representation of full factorial experiments". One convention is to label each factor by a letter (or numeral) and then to denote the two levels (versions) of each factor by a plus (+) and a minus (-) sign.e. Other Factorial Design Notation Matrix (22) Two-Level Two-factor Design Sign Notation Run No. To identify the individual trials. k each at two levels. The "Tree". Commonly the minus sign refers to lower level. The Champion recognizes that the "Algebraic" and "Yates Standard Notation" are directly related to the runs of the experiment. with their respective levels (low and high). one trial at each combination of levels of the factors. lubricated bearing).g. with a plus (+1) or minus (-1) sign to denote if the factors are set to their high or low settings. The experiment consists of 2 trials. The Master Black Belt adds: "the "Algebraic" notation respects Yates Standard Order. The fourth image. the standard condition. illustrates the experiment design for factors A and B.high level. Alternative Notation Japanese Notation Classical Notation A + + B + + A 0 1 0 1 B 0 0 1 1 A 1 3 1 2 B 1 1 2 2 A (1) a b ab B 1 2 3 4 1_05_01_030 Experiment Design Tools Other Factorial Design Notation Matrix (22) A Black Belt asks her Master Black Belt: "In our DOE. also called "Taxonomy" or "Characterization Chart". or absence of the factor (e. except for the case of (1). not lubricated vs.

respectively. and as +1 when it is set at "high".2 B. The classical convention is to denote the two versions of each factor by the presence and absence of its corresponding lowercase letter. here the trial in which all factors are at their "low" level is denoted by (1). says the Master Black Belt." The Master Black Belt adds: "The convention we have adopted is to use the -1. and Type of bath set at -1 (low. or water).)." "…You are familiar with the –1. Three factors are identified: Quenching speed (A). Aging time at -1 (low. which is similar to the sign notation. 1 and 2 for the two version of each factor. or. which makes it possible to identify." Yates Standard Order (2 2) Quenching Speed = A Low: 1 m/s = High: 2 m/s = m/s = meter per second Aging Time = B Low: High: Bath = C Low: High: 4min = 8min = Water = Glycol = Factors Run Order -1 +1 Yates Standard Order Table (2 Factors Run Order Treatment combination 3) -1 +1 -1 +1 A -1 +1 -1 +1 -1 +1 -1 +1 B -1 -1 +1 +1 -1 -1 +1 +1 C -1 -1 -1 -1 +1 +1 +1 +1 A Quenching Speed B Aging Time C Bath 1_05_01_031 Settings for a two-level two-factor design Experiment Design Tools Yates Standard Order (22) A Six Sigma team conducts an experiment in the heat treatment process. This run is Improve 32 1 2 3 4 5 6 7 8 1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s 4 min 4 min 8 min 8 min 4 min 4 min 8 min 8 min Water Water Water Water Glycol Glycol Glycol Glycol 1 2 3 4 5 6 7 8 (1) a b ab c ac bc abc . Alternative notations are to employ 0 and 1. We used a two-level. three-factor design. For each run. and Type of bath (C). Aging time (B). the Yates Standard order is the one most often used". or 4 min. the corresponding factor level is identified as -1 when the factor is set at "low". the four trials comprising 2 factorial design can be represented as shown in this table. The levels of each factor are set as follows: • • • Quenching speed (factor A): Aging time(factor B): Bath type (factor C): Low: 1 m/s Low: 4 min Low: water (-1) (-1) (-1) High: 2 m/s High: 8 min High: Glycol (+1) (+1) (+1) "When designing the experiment. factor levels and contrast coefficients used to calculate factor and interaction contrasts when performing DOE analysis. 1 notation used in Minitab. following the Japanese tradition earlier established by Taguchi. or 1 m/s). The first combination shown in the Yates Standard Order will be a treatment with Quenching speed set at -1 (low. 1 notation.

5 4.0 ? and Blow = 0. The Master Black Belt explains that it is important to randomize the Yates Standard Order prior any experiment in order to average out the noise caused by nuisance variables if they exist.0 9. All other runs are set in the same fashion.0 A ? 50%.g. The experiment is designed according to Yates Standard Order for one replicate only.5A.5 Main Effect Plot for B Run Order 1 2 3 4 (1) a b ab Contrast Effect Avg + 1_05_01_032 A -1 1 -1 1 12. The team defines the factor settings at Alow = 3.0 AB 1 -1 -1 1 6. Computing Contrasts and Effects (22) Ohm’s Law Voltage = Resistance x Amperage Specifications Uv Y = RΩ * I A = A*B Response Y: V Maximize Factor A: 6.0Ω ±50% Factor B: 1.0 6. -1.& AB +) . Since the "B effect" is positive.0 ? ? 50%. and the output (voltage) is recorded for each run (e. the voltage (output) increases by an average of 6 volts when the amperage (factor B) increases from low to high. The team recognizes that using this Yates Standard Order can help them avoid errors in preparing an experiment. +1.5 A and Bhigh = 1. two-factor design.0 3. Resistance (factor A).0 7. 12 for factor A). Two factors are considered (Resistance and Amperage).0 Avg ∆ Experiment Design Tools Computing Contrasts and Effects (22) A Six Sigma team conducts an experiment. Ahigh = 9. Improve 33 A and AB effects are neutralized Factor B -1 +1 B B+ (A. the voltage (output) will vary in the same direction as the direction of change of factor B.g. -1.0 3.0 ?.0 6.0A ±50% Y Factor Low High -1 +1 A 3.5 3. +1.0 9.& AB +) (AB .th represented in the first row by the numbers: -1.0 B -1 -1 1 1 12. is specified at 6. The Master Black Belt explains that the same table can be used to plan an experiment for a two-level.& A +) (A. and Amperage (factor B) at 1.5 9. up to the 8 run where the settings are +1.0 for both factor A and B).0 3.5 4.5 V for a Resistance of 3.5 13.0 3. The Black Belt computes the contrasts for each factor (e.0 ? and an Amperage of 1.0 9.5 1.5 4.0 Y1 1. and the objective is to maximize output (voltage). The Black Belt explains that since the effect of B is equal to +6. Similarly. Run #3: Voltage equals 4. he calculates the effect for the interaction of the two factors (3.0 for the interaction AB).0 6. and the corresponding effects (6.0 B 0.5 A).& A +) (AB .0.0 6.

Amperage (factor B) must be kept at the high level. In our two-level. the Black Belt reminds the team of the characteristics of the contrasts: a) Balanced. "This chart. In our experiment we see that contrast A is positive. along with their interaction AB allow us to compute the Sums of Squares. asks another team member. and b) Orthogonal. Finally. each contrast (A. shows the practical significance of the sources of variation" replies the Black Belt. SSfactor B for 38% and SSinteraction A*B for 10% of the SStotal (i. In other words. The team recognizes the advantage of using a pie chart to graphically represent the practical significance of the effects exerted by the factors. represents the effect of only one particular factor or interaction since the other two are neutralized by the combination of + and – signs. Finally. the total change in the response variable. "In our experiment. created from the Sums-of Squares in the ANOVA table. They make it possible to compute the effects and the Sums of Squares. and note that the remaining 10% represent the interaction effect (factors AB). the contrasts for factors A and B. the contrast of A represents variation due only to factor A since B and AB are neutralized. the Black Belt indicates that a Pareto Diagram can also be used instead of a Pie Chart." "What is the purpose of the pie chart?". The Black Belt highlights the fact that with this full factorial design. . to ensure independence of effects.e. We can also conclude that 90% of the variation of the response variable (CT Characteristic) is explained when factors A and B vary between low and high settings. Computing the Sums of Squares and Practical Significance (22) "Why do we calculate the contrasts"? Ask a team member. The Black Belt recalls that before considering the practical significance we must evaluate the statistical significance of the different effects. "Contrasts are part of the foundation of Analysis of Variance".The team concludes that to maximize voltage (output). two-factor design. B or AB). SSfactor A accounts for 52%. meaning that the output is higher when we set factor A at the high level compared to when it is set at the low level. replies the Black Belt.

0 can only be attributed to the difference between the levels of factor A. and the interaction AB by the combination of their – and + at each level of A. Another team member adds "Maximum output is 15. "This is the correct and complete answer" says the Black Belt. Improve 35 . Finally.5 6.Plotting the Main Effects (2 2) EffectA = ∆y = y A + − yA − replicates taken at the treatment Effect A = 1 { contrast A } combination 2n _ _ n=Number of or Y 10. we cannot only consider factor A. We must also consider factor B. "…however. 7. the average equals only 3. and when it's low (-1).0 +1 4.5. the average output equals 10.5". This is because the settings neutralize both the effects due to factor B. We perform an analytical procedure by calculating and plotting the Main Effects.5. they acknowledge the advantage of plotting the main effects to depict the results of the experiments. the output average is 10. This is the result of contrasts that are balanced and orthogonal in well-designed experiments. The Black Belt then asks "Which settings maximize output.0 Average Y at Low condition of factor A Y Main Effect Plot for B 10. says the Black Belt. the team asks the Black Belt to explain the next steps.0 2.5 5. The change in the output i. The team calculates the effect of factor A and then they conclude that factors A and B have the different effects.e.5 Main Effect Plot for A Average Y at High condition of factor A 3.0 Average Y at High condition of factor B Average Y at Low condition of factor B B:+1 -1 Factor A ∆Y 10. and to visualize the difference in the response output between low and high settings for each factor.0 B:+1 B:-1 1_05_01_034 3.0 when factor A is high and factor B is also high. and what will be the output level for such settings?" A team member answers "When factor A is set at high.5 Analysis of Factor A Y 15. In our experiments we can see that when factor A is high (+1).0 B:-1 A:-1 A:+1 -1 Factor B +1 Experiment Design Tools Plotting the Main Effects (22) After conducting an experiment and recording the responses. "You are partially correct".

0 units (from 2. output changes by an average of 4. we extract relevant information" explains the Black Belt. and two levels were considered: "Airlines" and "Manufacturers". The Black Belt adds: "two factor interactions are common. "I remember a project where we analyzed invoices that were overdue.0 Experiment Design Tools Plotting the Interaction Effect (22) "During the analytical procedure of the experiment. the output changes by an average of 10.0 B:+1 Analysis of Interaction AB Y 15.0 B:-1 A:-1 A:+1 2.0 B:+1 B:-1 A:-1 A:+1 1_05_01_035 2.0 Both lines are not parallel thus indicating an interaction effect 6. if the lines are not parallel. Conversely. three factor and higher are rarely seen in practice in our type of industry. "A major part of this is to determine if there is an interaction effect between factors". Note that graphs should be used to visualize the result of the ANOVA table.Plotting the Interaction Effect (2 _ _ n=Number of replicates taken at the treatment combination 2) Effect or AB AB = ∆y = y AB + − y AB − = 1 { contrast 2n AB Effect } Interaction Plot for AB Y 15. Improve 36 B:-1 .0 to 15. replies the Black Belt. For example.5 5.0).0 5.0 5.0 to 6. We were surprised to discover that there was a important interaction effect between the two factors (Invoice value and Customer). they have a combine effect on the output. an interaction exists between the factors. This is an indication of interaction between factors A and B. then there is no interaction between the factors.0 B:+1 B:+1 B:-1 ∆Y 8.0 units (from 5. The second factor was "Invoice value". when factor A changes from its low to high level.0). "Yes".e. When B is kept at its high level and factor A varies from its low to high level. i." "If the lines in the interaction plot are parallel. and factor B is kept at its low level. "Interaction occurs when the effect of a factor is a function of the level of another factor. However.5 6. Graphs are useful to visualize significant interactions between factors but they should not be the only means of interpreting data." "Do interactions apply to transactional processes"? ask a team member. The first factor was "Customer".

The Black Belt states that the engineer correctly conducted a two way ANOVA using Minitab.94 p-Value 0.5 172.Statistical Significance and Practical Significance (2 2) Output Run 1 2 3 4 Standard Order (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y1 40.6 Y2 38.2 182.48 21.92 Mean Squares 3.0197 0. Three replicates per setting were done.1 85. two factor experiment (factors A and B).961.1 86.0000 0.9 Percentage % 89% 5% 2% 4% 100% Practically Significant .1247 Significant *** *** A*B 2% Residual Error 4% Source A B A*B Residual Error Total Experiment Design Tools Statistical Significance and Practical Significance (22) An engineer consults with a Black Belt after conducting a two-level.20 182. "This is called Practical Significance and in this case changes in factor A represent 89.52 63.0 Y3 38.7 74. The Black Belt also highlights the importance of verifying the practical significance of the results using the Sums of Squares.52 63.4 64.4 42.4 41. and that the condition can be reproduced. This means that changes in factors A and B significantly affect the output of the process.20 182.12 8. "The F test indicates p values for factors A and B are smaller than 5%.543.961.59 F-value 164.7 3.45 2. Improve 37 1_05_01_036 Sums-ofSquares 3.4% of the total variation. and now he evaluates the statistical significance of the main effects for each factor and the effect of interaction between factors A and B. It is a clear indication that the change is not due to chance.543.2 44." The engineer recognizes that although factor B is Statistically Significant it does not account for more than 5 % in practical terms. thus with little interest for him at this stage.48 172.5 63.543. It is what we call Statistically Significant".6 68.71 3.1 72.6 St Si atis gn ti ifi ca ca lly nt B 5% A 89% Source A B A*B Residual Error Total Degree Of Freedom 1 1 1 8 11 Sums-ofSquares 3.

Improve 38 . The fourth graph shows no particular pattern and the dots are randomly distributed. thus we conclude that the residuals are normally distributed. we check the random distribution of the residuals versus the settings. we must verify that residuals are normally distributed. a Black Belt notices that in Minitab he can graph the residuals. which is assumed constant for all levels of the factors." The Black Belt remembers the importance to analyze residuals prior to any conclusions about experiment result. the Black Belt consults with a Master Black Belt. no data out of the control limits and no particular pattern. "When conducting a DOE. The models of the errors are assumed to be independent random variables that are 2 normally distributed with mean equal to zero and variance ? .909 0 5 10 Observation Number Histogram of Residuals 4 Frequency 3 2 1 0 -3 -2 -1 0 1 2 Residual 1_05_01_037 Residuals vs.0SL=3." "Second." "Finally. Fits 3 2 Residual 1 0 -1 -2 -3 40 50 60 Fit 70 80 90 Histogram of Residuals normally distributed around 0 Experiment Design Tools Variable does not change across values of the Output Residual Analysis (22) While analyzing his first 2 x 2 factorial design. The histogram of the residuals shows this normal distribution and the centering around zero. The dots are randomly distributed.000 -3. The normal plot shows dots aligned on a straight line indicating that residuals are indeed normally distributed.0SL=-3." The Black Belt notices: "First. and to estimate them. thus we may conclude that the residuals are randomly distributed regarding the experiments run order. we consider the Individual Chart of Residuals. we verify that residuals are normally distributed center around zero. our objective is to test the appropriate hypotheses regarding the treatment of effects." "Third.Residual Analysis (2 2) Normal Plot shows no non-normality Shows no residuals out of control beyond the control limits Residual Model Diagnostics Normal Plot of Residuals 3 2 1 0 -1 -2 -3 -2 -1 0 1 Normal Score 2 5 4 3 2 1 0 -1 -2 -3 -4 -5 I Chart of Residuals 3.909 Residual Residual X=0. To refresh the concept of residuals.

300 StDevCoef 1.017 + 17.341 1.81 2.3.017 + 17.72 63.341 1.94 P 0.9*(-1) = 71.92 Adj SS 3725.3.2 2 Estimated Effects and Coefficients for yield Term Constant A B A*B Effect 34. He then continues.71 T 0.000 0.183*1 . With a similar calculation." "The average response output Yield when Temperature (A) and Pressure (B) are set to their High level is computed using the model: Y = 58.183*1 + 3.183 * A +3.Modeling .000 0. then our process will have an average yield of 58.020 0.800 4. We can also see that the level of Factor A (Temperature) is key in controlling the yield of the process as compared to Factor B (Pressure). Improve 39 ˆ Y =58.341 1. "are we limited two these factor settings.48 21. we can determine that when A is High and B is Low.71 172.017 +17.71 3961.900 2.125 Experiment Design Tools Modeling – 22 "The results of the DOE quantify the effect that the two factors and the interaction have on the process response" says the Champion. "To confirm this linearity.91 1. the average yield will be Y = 58. there are other things to understand.125 P Analysis of Variance for yield Source Main Effects 2-Way Interactions Residual Error Pure Error Total DF 2 1 8 8 11 Seq SS 3725.59 F 86.25 12.1.9*1 = 79." "We usually assume a linear response. or is it possible to know more about the process behavior?" "It is correct to say that the DOE analysis gives us a good knowledge about the factor effects and their interaction" answers the Black Belt.48 172. says the Black Belt.017 17.341 43. in such a case this mathematical model is valid".71 172.86 63.01% .183 3.367 7.71 Adj MS 1862. From the mathematical model we can see that if no change is made. we use other techniques such as center points or multi-level factorial design" The Champion recognizes the advantage to represent the process behavior using a mathematical model which makes it possible to estimate process output for factor levels other than the pre-established settings.9 * B 1_05_01_038 .59 21. "However.48 172.000 0.600 Coef 58.29 2.72 63.

"leverage variables". 64 runs and. I have calculated that with these 5 factors at two levels." . So. We would like to do an experiment with at least 5 factors. a full factorial experiment will involve a large number of tests. but this would be too large an experiment to do a full factorial. I will need 32 runs. What do you recommend I do?" The Master Black Belt replies. "During the team's brainstorming session. In fact. even when only two levels of each factor are being investigated. when you don't know which factor has a substantial effect on a response variable. it is common practice to do a fractional factorial experiment when we have 5 factors or more. we have identified several factors that can influence our CTQ characteristic. 7 factors will require 128 runs and I have not included any replications yet. and you want to verify several factors. the angularity of the parts formed on the Fluid Cell Press. then the fractional factorial design is the preferred analytical tool. With 6 factors. We had identified these factors a long time ago but we don't know yet which ones have the biggest impact. So far.Multi-Factor Experiment On Lo Off In Out Hi Front Back Left Right Up Down 2 × 2 × 2 × 2 × 2 × 2 × 2 Improve 40 1_05_01_039 5 factors : 2 5 = 32 combinations 6 factors : 2 6 = 64 combinations 7 factors : 2 7 = 128 combinations Experiment Design Tools Multi-Factor Experiment A Black Belt from a manufacturing area is discussing her project with a Master Black Belt. "if there are many factors in the experiment.

However. BC. AD. it is considered reasonable to assume that third order and higher order interactions (interaction between three factors or more) are statistically negligible. BD. Statisticians have shown that for 5 factors or more.Fractional Factorial Designs Full Factorial Fractional Factorial Example with 4 Factors Main Effects A B C D 1_05_01_040 Interactions AB. but the impact is minimal. How will you be able to test all the possible factor combinations if you do not use a full factorial experiment?". ABD. reducing the required time and resources. we will do a half fraction of it. AD. a certain amount of information will be lost. The trade-off is mainly that we lose information regarding the interactions between three factors or more. BC. for most situations in industry. BD. When compared to a full factorial design. AC. "A fractional factorial design is a carefully prescribed subset of all possible combinations of a full factorial design. CD ABC. ACD. and yet we can still assess many factors at the same time. which consists of 8 runs." . you can obtain nearly the same information as a full factorial with a well-designed fractional factorial experiment. AC." "I don't understand how you can do that." "The benefits of conducting a fractional factorial experiment are that fewer tests are required compared to the full factorial design. We want to study four factors at two levels. asks the Champion. BCD ABCD Main Effects A B C D Interactions AB. CD Improve 41 Experiment Design Tools Fractional Factorial Designs A Black Belt is presenting her Design of Experiment plan to a Champion: "For my project on the Fluid Cell Press. Instead of doing a full factorial 4 design. which represents 16 runs (2 ) and is very time consuming. I will use a fractional factorial design.

and C. the design generator can be ABC=-1 or ABC=1. First. Therefore.Forming a Half-Fraction of a Full Three Factor Design Run A 1 2 3 4 5 6 7 8 -1 1 -1 1 -1 1 -1 1 B -1 -1 1 1 -1 -1 1 1 C -1 -1 -1 -1 1 1 1 1 AB 1 -1 -1 1 1 -1 -1 1 AC 1 -1 1 -1 -1 1 -1 1 BC ABC 1 1 -1 -1 -1 -1 1 1 -1 1 1 -1 1 -1 -1 1 Generator: ABC=1 Same Contrast Run A Confounding : A & BC or alias B & AC structure C & AB B -1 1 -1 1 C -1 -1 1 1 AB -1 -1 1 1 AC -1 1 -1 1 BC ABC 1 -1 -1 1 1 1 1 1 Experiment Design Tools Forming a Half Fraction of a Full Three Factor Design "I know that Minitab creates fractional factorial designs. The other columns constitute a repetition of these three. we have to sort the entire matrix by the highest order interaction. the ABC effect is lost since it was used as the generator to form the half fraction of the full factorial matrix. In this instance. 5 and 8. For three factors with this design. The same thing happens for B with AC. For most situations in our industry. Second. but for my own understanding I am interested in knowing how to construct one. we will take a simple example with a three factor design." "The sign of the ABC effect constitutes a "block". the main effects are confounded with the two factor interactions. Look what happens to the matrix. This is called confounding or alias structure. In our case. we must sort on the ABC column. we will keep only the runs 2. First. if we choose the generator ABC=1. Note that I could have chosen ABC=-1 as the design generator and arrived at the same conclusions. B. The fractional design is formed by selecting all those treatment combinations where the ABC effect is either "1" or "-1". asks a Black Belt to a Six Sigma Instructor. "OK. so we will use the ABC interaction to create the fraction. and C with AB. Could you explain this to me?"." "The design becomes these 4 runs and we only need the three columns for A. 3." Improve 42 1_05_01_041 2 3 5 8 1 -1 -1 1 . it is considered reasonable to assume third order interactions to be statistically negligible. the unique effect of "A" can not be separated form the unique effect of the BC interaction because both columns have the same sign at each experimental run.

sure! Depending on the number of factors and runs. could you explain this to me?". different types of designs are available. says the Black Belt. "However." "With resolution IV and V designs. I am not sure that I understand the terminology associated to the designs. …. I have noticed that for a specific number of factors that we want to study. Such designs are also employed to obtain a mathematical model describing the relationship between the CT characteristic and the factors (Y = f(X1. Resolution v and higher designs are called Optimization Designs and are most often used to study more complicated effects and relationships involving two or three factors. (Optimization Designs) V Experiment Design Tools Design Resolution "In Minitab. Improve 43 1_05_01_042 . For resolution V and higher: to study more complicated effects involving 2 or 3 factors and obtain amathema tical model. in more detail. "Well. Another experiment is then needed to study.Design Resolution Resolution III IV Main Confounding Pattern Use Main effects are confounded To screen many factors with second order interactions (Screening Designs) Second order interactions are To study uncomplicated confounded with other second effects and interactions order effects interactions (Characterization Designs) Second order interactions are confounded with third order interactions Same as resolution IV. the resolution defines the alias or confounding structure of a design. XN)). main effects are not confounded with interactions but interactions are confounded together". asks a Black Belt to a Master Black Belt. "Yes. if I use the function "Create Factorial Design" (under Stat > DOE). replies the Master Black Belt." "That design doesn't seem to be very useful". the quantity of information is limited. the important factors that have been found with the resolution III design and their interactions effects. as their name implies. "We consider them as Characterization Designs and they are generally used to study relatively uncomplicated effects and interactions from a smaller number of factors compared to resolution III designs. continues the Master Black Belt. so these designs are generally used to initially screen a large number of factors. a resolution III design means that the main effects are confounded with interactions between factors. these designs are called Screening Designs and a small number of runs are required to use them". For example.

a resolution VI with 32 runs and a full factorial with 64 runs.Choosing a Factorial Design Number of Runs* 2 4 8 16 32 64 128 full 3 III full IV full III V full III IV VI full III IV IV VII full IV IV V VIII Number of factors 4 5 6 7 8 2 6 −2 IV Improve 44 1_05_01_043 *Note: the number of runs is for 1 replication. we have identified two levels (high and low) that can be used in an experiment. "k" represents the number of factors and "p" what fraction of a full factorial the design is. p=2 means a ¼ fraction and so on. but we don't really know much about them. For each of them. From the Analysis Phase certain factors appeared to have an influence on our CTQ. Don't forget that you will have to include a few replications which will significantly increase the amount of runs. it is my first experiment on this machine. a resolution IV design with 16 runs." "In this situation. p=1 it means a ½ fraction of the full factorial. What design do you suggest I use?" asks a Black Belt to a Master Black Belt. Interactions between factors are possible but we don't know if they exist. Experiment Design Tools Choosing a Factorial Design "For my project on the automatic riveting machine. "This refers to the notation that is used to describe k-p factorial design. In your case. The design you select will depend on what you need to know." . replies the Master Black Belt. In general we use 2 where "2" indicates that the factors are studied at two levels." The Black Belt replies. six factors have been selected as potential Critical to Process characteristics. I suggest you use the resolution IV design with 16 runs. your level of knowledge of the process and resource availability. "well. "For six factors. there are four types of factorial designs that are available: a resolution III design which consist of 8 runs. Then." 6-2 "Why is this design called a 2 ?" asks the Black Belt. p=2 means that your design is ¼ fraction of a full factorial for 6 factors at two levels. you don't have a lot of knowledge about the factors and you are definitely not at an optimization stage.

This CTQ will be measured on two typical parts that we produce: "curved" and "straight" parts. People. Can you please clarify the problem with the CTQ?". The result of the Gage R&R study detected a problem with the measuring system." "The Black Belt continues. we have identified more than 60 causes that can affect the angularity of the parts. "The objective of the Design of Experiment is to identify the factors that affect the quality of the parts formed in the Fluid Cell Press. but it has been corrected. We used a fishbone diagram to summarize the causes in the following categories: Machine. we have organized a location where all the parts will be measured. "Sure".Objective. Two separate statistical analyses will be done. "with the information gathered since the beginning of the project and a recent brainstorming session." "Do you have a good measuring system to obtain accurate data for your DOE?"." "We regularly rework the parts produced in this process. replies the Black Belt. "The parts produced have an angle that do not meet the target of 90 degrees. asks a Champion. one for each type of part. Methods. "Yes." Improve 45 1_05_01_044 . An operation of check and straighten has to be done practically on every part to bring the CTQ on target. Tools. Materials. The CTQ characteristic that we want to improve is the angularity of the parts. Response Variables and Factors CTQ target: 90 O CTQ: Angularity Machine Tools Materials Angularity of the parts People Experiment Design Tools Methods Environment Objective. and Environment. Response Variables And Factors A Black Belt presents her Design of Experiment plan to a group of Champions. and a Gage R&R study has been conducted with the team members who will measure the parts. asks the Champion from the Quality department.

says the Methods' Champion. Finally. However most industrial experiments are done with 8 factors or less. In general. the 7 factors were identified. noise and held constant.Select Factors and Levels Factors A: Loading of tray B: Height of blocks C: Grain direction D: Covering type E: Pressure F: Forming Speed G: Spring-back Levels Low (-1) High (1) 20% 50% 1/4” 1 1/4” 45º 90º Soft Hard 8.000 PSI 15." "This process is so complex. "Well. asks another Champion. "Some factors such as forming speed only have two options". but not so much that they extend beyond the operational range of the factor to have a better chance of detecting the effects that are present. "With the information we gathered since the beginning of the project on the Fluid Cell Press. replies the Black Belt." "Is it a reasonable number of factors for an experiment?". more than 60 causes were identified but we classified these causes to narrow down the number of factors to study in the experiment".000 PSI Normal Slow 2º 4º Y True Effect In general. the levels should be set "wide". Response Experimental Effect Lo (-1) Hi Factor Settings (+1) Experiment Design Tools Select Factors And Levels During a presentation of a DOE to a group of Champions. the levels should be set wide but not so much that they extend beyond the operational range of the factor." "How did you select the levels?". The controllable factors are studied in the experiment because we can control and change them during an experiment. we separated the causes into three different categories: controllable. How did you select these seven factors? I am sure that there are more than seven factors". Other levels come from the operators' experience and by pushing the factors to reasonable extremes. we used a Nominal Group Technique where each team member selected and rank the factors of his choice by order of importance. replies the Black Belt. we have identified seven factors that can influence the CTQ characteristic (Angularity of the parts). generally the number of factors studied in an experiment is less than 15. a Black Belt explains the selection of factors and levels. First. replies the Black Belt. and a recent brainstorming session. From the group of the controllable factors. "Initially." Improve 46 1_05_01_045 . asks a Champion.

"Sure. but I am not sure which function to use in Minitab to analyze them. we cannot separate the effect. A*F. we can input in Minitab the following terms for the factors of the model: A. we know that no main effects are confounded with any other main effect or second order interaction." "Since it is a resolution IV. we use the first interaction as a label because when we analyze the data. the same contrast is used for these three interactions. asks a Black Belt. what type of experiment did you conduct?". Can you please help me?". asks the Master Black Belt. A*G and B*C. we have to consider that the label A*B represents in fact A*B+D*G+E*F. A*B. D. because in the design matrix. G. A*D. But second order interactions are confounded with other second order interactions." "Note that since the interactions are confounded with other interactions. For most situations in industry. 2 with 2 replications. The main effects are confounded with third order interactions. "A fractional factorial experiment with seven factors at two levels each. F. we verify the alias or confounding structure using the DOE function of Minitab." "This function confirms that we have a resolution IV design. I used a 7-3 resolution IV design. so a total of 32 runs were done. However. First. E. To perform a statistical analysis such as ANOVA. For example. it can be either A*B. if the A*B effect is significant." . A*E. C. D*G or E*F that is important. it is considered valid to assume that third order and higher interactions are negligible. A*C.Alias Structure Alias Structure for a 2 A B C D E F G A*B A*C A*D A*E A*F A*G B*C 1_05_01_046 7-3 iv design + B*D*G + B*E*F + C*D*F + C*E*G + A*D*G + A*E*F + C*D*E + C*F*G + A*D*F + A*E*G + B*D*E + B*F*G + A*B*G + A*C*F + B*C*E + E*F*G + A*B*F + A*C*G + B*C*D + D*F*G + A*B*E + A*C*D + B*C*G + D*E*G + A*B*D + A*C*E + B*C*F + D*E*F + D*G + E*F + D*F + E*G + B*G + C*F + B*F + C*G + B*E + C*D + B*D + C*E + D*E + F*G Improve 47 A*B*C + A*D*E + A*F*G + B*D*F + B*E*G + C*D*G + C*E*F Note: The terms presented here are only up to the third order interactions Experiment Design Tools Alias Structure "I have the results of my DOE on the Fluid Cell Press. B. and second order interactions are confounded with other second order interactions.

027 0.000 0. asks the Black Belt. it is more likely that the D*G interaction is present here.889 375.188 0. How do I know which model to use in Minitab?.889 375.43 55. the function "General Linear Model" can be used". It seems that D.722 10.000 0. "In order to obtain the sums-of-squares of each term in a model. "The sums-of-squares are the basic measure of variation associated with each term of the model.188 4.000 0.60 5. G and A*B have the largest sums-of-squares compared to the total. "Is it really important to calculate all the sums-of-squares?".753 6.000 0.133 3.722 10.147 3.000 0. Since D and G are both important as main effects.586 Adj SS 9.445 3.038 1158.001 0. "The ANOVA table shows that all the effects are statistically significant at an alpha level of 0.000 0.389 583.679 134." Improve 48 1_05_01_047 .133 3. we see that the A*B interaction is confounded with D*G and E*F. they are not all practically significant as demonstrated when we consider the magnitude of each sum-of-squares compared to the total sum-of-squares.29 14.070 13.147 3.170 1. we select the main effect and the two factors interaction".50 16.001 0.679 134.038 Adj MS 9.897 5.445 3.070 13.238 F 41.188 4. Although they are all statistically significant.230 1. replies the Master Black Belt.897 5.84 1582." "Don't forget that A*B is only a label here.008 0.30 45. for a resolution IV design from the alias structure. asks the Black Belt.170 1.14 5.06 28.85 2457.389 583.836 2.05.039 Experiment Design Tools ANOVA Table A Master Black Belt explains to a Black Belt how to analyze the results of a DOE in Minitab.230 1.26 9. The next step is to graph the residuals to verify the assumptions associated with ANOVA. "Yes".070 13.389 583.ANOVA Table Analysis of Variance for Angularity Source A B C D E F G A*B A*C A*D A*E A*F A*G B*C Error Total DF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 17 31 Seq SS 9.002 0.230 1.445 3. replies the Master Black Belt.753 6.836 2.133 3.000 0.679 134.27 564.000 0.836 2. and to detect the presence of outliers that can contaminate the results." 7-3 "My design is a fractional factorial 2 of resolution IV.40 21. If we look at the Alias structure associated with your design.753 6.170 1.00 P 0. "Generally.889 375. They are also used to evaluate the practical significance of the effects.67 13.897 5.722 10.147 3.

"From the alias structure associated with the design.836 32.4% of the total variation of the response variable (CT characteristic). It seems that G and D are the vital few variables since they account for more than 80 % of the variation in the response variable. asks a Black Belt.4 A*C 13. What does it mean taking into account the fact that the 7-3 design is a fractional factorial 2 of resolution IV?".6 % of the total variation. D*G and E*F are confounded with A*B." Percent Count .070 11." "From the Pareto chart we see that the main effect G represents 50.868 4.5 100.8 A*B 134. and that D represents 32. To estimate the practical significance.Practical Significance Pareto Chart for Source 1200 1000 800 100 80 60 40 20 0 600 400 200 0 Source Count Percent Cum % G 583.1 95." "The A*B effect represents 11.4 D 375. asks the Black Belt.0 Experiment Design Tools Practical Significance "How can I identify the effects that are really important? The ANOVA table shows that almost all the effects are statistically significant". In this case.4 50. and since both D and G are important main effects.4 %.4 82.5 ers Oth Improve 49 1_05_01_048 51. "This situation happens especially when the experimental error is small".6 94.133 1. replies the Master Black Belt. there is a good chance that the interaction D*G is exerting the most influence here.679 50. we know that other interactions are confounded with the A*B interaction. we use the sums-of-squares from an ANOVA table to create a Pareto or a Pie chart.

"The main difference is that instead of using the sums-of-squares. if some contrasts represent real differences rather than random variation. If all of the contrasts represent nothing but random variation (i. if there are no real differences between the averages at -1 compared to 1).e. says a Master Black Belt to a group of Black Belts. and A*B is a significant interaction effect. However. the Normal Probability Plot represents the estimated contrast effects but both graphs are used to identify the important effects"." "We can clearly see in our plot that G. "What is the difference between this graph and a Pareto chart of the sums-ofsquares?".Normal Probability Plot of the Effects Normal Probability Plot of the Standardized Effects (response is A ngulari. we know that D*G and E*F are confounded with A*B. D and A*B represent real differences. "Well. Therefore. In other words. asks a Black Belt. then these contrasts are plotted on the lower left or upper right of the straight line formed by the other contrasts. G and D are the vital few variables. A lpha = ." Improve 50 1_05_01_049 -40 -30 -20 -10 0 10 20 30 40 50 . From the alias structure associated with our design. replies the Master Black Belt. and since both D and G are important main effects there is a good chance that the interaction D*G is exerting the most influence here. then the contrast effects should form a reasonably straight line on the Normal Probability Plot. the "outliers" represent significant effects. asks a Black Belt. "How do I interpret this plot?".10) G A : B : C : D : E : F: G : A B C D E F G 1 0 -1 D A C C A E A D E F AC B B C A G A F B A A B N al S orm core Standardized Effect Experiment Design Tools Normal Probability Plot of the Effects "A graph used to analyze the results of a DOE is the Normal Probability Plot of the effects.

. replies the Black Belt. It helps us to visualize and confirm the important main effects. "If there is an interaction between these two factors. we should set factor D at its high level (1) and factor G at its low level (-1)". Is there a tool that creates a visual representation of the results?". "The Main Effects Plot is a very useful tool". and the design is a fractional factorial 2 of resolution IV with two replications. says the Master Black Belt. "The CT 7-3 characteristic is the angularity of the parts. asks the Master Black Belt. "What is the CT characteristic and the type of design you used?". "The data is the result of a fractional factorial design related to our project". says the Black Belt. "The mean of the response variable is plotted for the levels of each factor. it is preferable to do an interaction plot to find the best setting for both factors". Since our objective is to minimize the response variable (CT characteristic). asks the Master Black Belt. " "Where does this data come from?". asks a Black Belt. replies the Master Black Belt. says the Master Black Belt." "The Main Effects Plot shows a large difference in the mean of the response variable between the low (-1) and high (1) levels of factors D and G". "Those are the exact same results I found using the sums-of-squares in the ANOVA table".Main Effects Plot M Effects for Angularity ain -1 1 4 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 Angularity 1 2 1 0 8 6 A 1_05_01_050 B C D E F G Improve 51 FACTORS Experiment Design Tools Main Effects Plot "I find that the ANOVA table is not user friendly because it displays many numbers arranged in a tabular form. replies the Black Belt.

factor D has a large impact on the mean of the response variable. replies the Master Black Belt. When combined with factor G." "Considering that we want to minimize the response variable. "We use it when the results of an ANOVA table make us suspect the presence of an interaction. I create an Interaction Plot. and factor G at its low (-1) level. asks a Black Belt." "To interpret this plot we must consider our objective. the response variable is smaller compared to when G is set at its high level (1). we conclude from this graph that factor D must be set at its high (1) level. Do you need to maximize or minimize the response variable?". asks the Master Black Belt. Since the lines are not parallel. We can also see that when factor D is set at its high level (1). the difference in the response variable between "G low" and "G high" is smaller than when D is set at the low level (-1). The Interaction Plot helps us to visualize and confirm this interaction"." Improve 52 .Interaction Plot Interaction Plot for Angularity D -1 1 -1 1 D Low (-1) 20 15 M ean 10 D High (1) 5 1_05_01_051 -1 1 G Experiment Design Tools Interaction Plot "When do we use the Interaction Plot in a DOE?". replies the Black Belt. "I need to minimize it". "Since I believe that there can be an interaction between factors D and G in our DOE. "We then can see that when factor G is set at its low level (-1). we confirm that there is an interaction between factors D and G.

the experimental runs are sub-divided into groups in such a way that the runs within a group are relatively homogeneous. shifts. the experimenter can test the significance of the blocking variable. etc. since it makes it difficult to identify factor effects. and the run within each block is performed in a random order. "to reduce the effect of noise variables such as batches of material." "Blocking variables allow us to remove the noise from the error term by creating a separate term. the results will be "contaminated". we use a "randomized block design" where the homogeneous units are formed into blocks. "To remove the effect of the added noise. we must not forget that generally a Blocking variable is confounded with higher order interactions". However. when creating a design matrix.. we use "Blocking Variables"." Improve 53 . suppliers.Blocking Variable Non-homogeneous units Formed into blocks Batch 1 Batch 2 StdOrder 7 6 8 5 3 4 1 2 RunOrder 1 2 3 4 5 6 7 8 Block 1 1 1 1 2 2 2 2 A -1 -1 1 1 1 -1 -1 1 B -1 1 1 -1 -1 1 -1 1 C 1 -1 1 -1 1 1 -1 -1 1_05_01_052 Experiment Design Tools Blocking Variable A Black Belt planning a DOE related to a process in the composites material department says: "I am afraid that by using different batches of material during our DOE. and second. operators. Thus. After all. "When using Blocking Variables. it reduces the experimental error which makes it easier to detect effects. A Master Black Belt replies. The reason for grouping units or runs into blocks is to reduce the amount of experimental error or noise. blocking provides two benefits: First. every batch of material is different from the others".

22 1.56 1105.56 855. I followed your recommendation and used a blocking variable.98 28. "When conducting a DOE on a chemical process.75 0.71 36.06 1314.56 B*D 1 0. the block effect is not statistically significant." The Master Black Belt replies.05. redo the model without considering the blocking variable.06 39. you just have to put the blocking as a term in the model used.Analyze an Experiment With a Blocking Variable Y: Filtration Rate Blocking Variable: Days A: Temperature B: Pressure C: Fomaldehyde Concentration D: Stirring Rate Analysis of Variance for Filtrate Source DF SS MS Block 1 7.56 B*C 1 22.003 0.06 A*C 1 1314. for an alpha level of 0.318 0.703 C AD D AC -5 0 5 Standardized Effect Analyze an experiment with a Blocking Variable During a coaching session. The same conclusion is obtained when we use the Normal Probability Plot of effects.56 1870.435 0. If the block effect is significant.46 0.966 0.02 0.06 Total 15 5730. "Well. then we can proceed with the analysis.06 C 1 390.56 7. C and D.56 22. main effects plots. Two days can have a large effect on any chemical process and I don't want to increase the amount of experimental error.06 0. we can continue the analysis without considering the blocking variable.00 43.56 B 1 39.898 0.56 0.94 Experiment Design Tools A Norm Score al 1 0 -1 F 0. as well as the interaction effects A*C and A*D are statistically significant. However.25 62. we continue our analysis with the prescribed steps (Residual analysis. the variable "days" was the blocking variable.56 A*B 1 0.023 0. The practical significance of these factors and interactions is confirmed when we compare their sums-of-squares with the total sums-of-squares. and interaction plots).17 P 0.06 D 1 855.004 0. and that factors A.30 12. After completing these steps." "From the ANOVA table we see that.06 Error 4 120.06 A*D 1 1105.78 0." Improve 54 1_05_01_053 .06 5.642 0. and if necessary.006 0.56 A 1 1870. Statistical significance means that we took an accurate decision in using such blocking variable.56 C*D 1 5. and analyze it with the other factors contained in a detailed ANOVA table". Since it took two days to complete the experiment. I don't know how to analyze the data. thus making it more difficult to identify the factor effects.25 30.001 0. "If the block effect is not statistically significant. knowing that the results are not contaminated. a Six Sigma Black Belt says.06 390.

From previous screening studies. the team assigned to this project identifies two variables which influence the process: • • Curing Pressure (factor A). Curing Temperature (factor B). The Six-Sigma team adopts a measurement scale for the output (Y) that ranges from 0 to 100. To code each setting for factors A and B. the Black Belt creates this data table in Minitab showing the 18 runs. she chooses 1 as the label to denote a low setting.100 where 100 is perfect part) Minitab Entry Shown in Standard Order Factor A: Curing Pressure Factor B: Curing Temperature Data Collection Sheet A 1 1 1 2 2 2 3 3 1_05_01_054 B 1 2 3 1 2 3 1 2 3 Y1 40 50 60 50 70 50 30 20 70 Y2 20 30 50 10 60 60 40 10 80 3 . the team is ready to analyze the data using Minitab's statistical functions. where 100 is a perfect composite part. . Current settings create problems related to shape and other CTQ characteristics of a particular group of parts. . After conducting the experiment and recording the responses. Improve 55 A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 .Two-Factor Multi-Level Full Factorial CTQ Y: Composite part (Measurement Scale: 0 . To collect the data in an organized way. 2 for an intermediate setting and 3 for a high setting. The team prepares to conduct a two-factor three-level DOE with two replicates. So what calculations must be made ? Experiment Design Tools Multi-Level Two-Factor Design The Champion responsible for the manufacturing of composite parts believes that the autoclave settings are not optimal.

The team conducted two replicates for this two-factor threelevel experiment. for an ? risk of 5%.Statistically Significant? Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 The Anova Table Experiment Design Tools Statistically Significant A Six-Sigma team conducts a DOE related to the curing process of composite parts. The Black Belt explains to the team the interpretation of the results shown in the ANOVA table. this allows us to see if the interaction between factors A and B has an impact on the output". Two factors are considered: Pressure (factor A). Improve 56 1_05_01_055 . and Temperature (factor B).05. it is highly improbable that the change in output is due to chance alone.6% probability that the change in the output is due to chance. After recording the responses. factor B (Temperature) has a great influence on the output. as shown by a p value smaller than 0. it affects the output through its interaction with factor B.008. which one do we choose?". replies the Black Belt." Team members recognize that even if the main effect of factor A is not statistically significant. asks a team member. "From this table we see that factor A (Pressure) is not statistically significant. The Interaction between factors A and B (A*B) has a significant effect on the CTQ. meaning that there is a 46. We now can say that the change in the output is not random in nature. With a p value of 0. and thus is repeatable. the Black Belt generates the ANOVA table in Minitab to determine if there is a statistical difference between the output at different levels of the factors (Pressure and Temperature). "In this case we choose "Balanced ANOVA". "There are various ANOVA functions in the Minitab menu.466. On the other hand. The p value is 0.

000 *** 0.6 .05. both factors.80 25.2%) .1%) Error (1500 . Improve 57 DF 2 2 4 27 35 SS 555.0 41. we see that. Can you clarify this apparent contradiction.4.1% of the Total Sum-of-Squares (practical significance).00 51. for an ? risk of 0. In fact.9 MS 277.8 1500.6 5577.8 1394.1 43." The Champion recognizes that in order to maximize the process output Y. "You just told me that factor A (Type of coolant) is statistically significant.6 5577. In order to take advantage of this interaction.6 F 5.7%) Experiment Design Tools Practically Significant "There is something I do not understand.Practically Significant ? Multi-Level Full Factorial Analytical Significance Analysis of Variance for Y Source A B A*B Error Total DF 2 2 4 27 35 SS 555.41.8 .8 2877.6 ." "Does this means that factor A is not to be considered?" asks the Champion. we see that this factor accounts for only 4.6 5755.014. We conducted an experiment with two factors: A) Type of coolant and B) Type of end-mill. "We must not decide based only on main effects. we must work on both factors A and B.2 100. As we can see.0 A (55. thus "type of coolant" is statistically significant.0 13388.43%) A*B (5577.9 % 4." says the Champion.4 55.8 1500. we must first take advantage of the effect caused by factor B.0 13388. the p value for factor A is 0. from the analysis of the Sum-of-Squares.000 *** Pie Chart of Source Practical Significance Source A B A*B Error Total 1_05_01_056 Sums-of-Squares B (5755. "No" says the Black Belt. the interaction A*B is statistically significant and also practically significant. but now you say that this factor is not significant. In the ANOVA table.6 5755. Each factor was set at three levels. but not practically significant.10 P Significant 0.7 11. A and B must be considered.014 *** 0." The Black Belt replies "There are situations where certain factors are statistically significant.11.0 %). However. because this factor accounts for the largest percentage of the total Sum-of-Squares (43.

We can see that the residuals are randomly distributed and that no point lies outside of the control limits." Using Minitab. Does this mean that we have finished the DOE data analysis?" asks a team member. the fourth graph reveals if the residuals are randomly distributed around the fitted values. Improve 58 . "There is a very important statistical analysis left to do. which is what we expect to see in this graph.0SL=27.0SL=-27. It indicates that the residuals are well grouped around zero. Fits 0 -20 -15 -10 -5 0 5 10 15 20 Residual 10 20 30 40 Fit 50 60 70 80 Experiment Design Tools Residual Analysis "We have conducted the DOE. the Black Belt performs the residual analysis and prints out the corresponding four graphs. interpreted the ANOVA table and identified the factors that have statistical and practical significance.38 2 6 Frequency 5 4 3 2 1 1_05_01_057 10 Observ ation Number 20 Histogram of Residuals 20 10 Residual 0 -10 -20 4 Residuals vs.38 X=0. Finally. We must analyze the residuals.Residual Analysis Multi-Level Full Factorial Residual Analysis for Composite Part 1 20 10 Residual 0 -10 -20 -2 -1 0 1 Normal Score 2 Residual Normal Plot of Residuals 3 30 20 10 0 -10 -20 -30 0 I Chart of Residuals 3.000 -3. The second is the histogram. The third is the chart of the residuals function of sequence run order. The team members conclude the residuals are normally distributed around zero and that they are not affected by special causes. "Not yet" replies the Black Belt. as it should be if nothing special is affecting the residuals." • • • • The first shows that the residuals are normally distributed. The residuals on this graph look alright.

and helps us to interpret the effects of the factors on the output. The Black Belt plots the Main effects. asks a design engineer. and explains the interpretation to the team." "Factor B ( temperature) however.Main Effect Plot Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 Main Effect plot ." "Does this mean that in order to obtain the highest average output we should specify the autoclave's temperature to the value corresponding to the level 3.Means for Composite Part 63 55 47 39 31 A B Y 1_05_01_058 The factor pressure (A) has relatively little effect on the response Y. "We should study the interaction plot to find the optimal setting for both factors. and 2 replicates are conducted for each setting of this full factorial design. has a great effect on the output. Experiment Design Tools Main Effect Plot An across functional team conducts a DOE during the design of an improved composite part. "The graph confirms the results of the ANOVA table." Improve 59 . and not worry about the pressure?". 2 Each factor was set at three levels (1. The factor Temperature (B) has a great effect on the output. We can see that average output is the highest when factor B is set at level 3. The team studies the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine." replies the Black Belt. Factor A (pressure) has relatively little effect since the average output remains almost constant at each of its three levels. 2 and 3) for a 3 experiment. "It is too early to make that statement.

2 and 3) for A 3 experiment. and take advantage. There is no output difference for A at level 1 and 2 when B is set at level 3 but a difference exists for factor A at level 3 for this setting of B. "We see three curves. That is. After discussing the main effect plot. we would not be able to discover. and 2 replicates are conducted for each setting of this full factorial design. we would have missed the non linearity of the response average corresponding to level 2 setting of factor B. and factor B (temperature) is set at level 2". "When factor A is set at level 1. Notice 2 that if we had conducted only a two-level design (i. The lowest output average is obtained when factor A (pressure) is set at level 3.Interaction Plot Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 Interaction Plot .Means for Composite Part A 1 2 3 1 2 3 Curvature on the CTQ output average of Composite Part for B (Temperature) average when A (Pressure) is at level 3 75 65 A2 A1 A3 M ean 55 45 35 25 15 1 2 3 B The smallest output average correspond to factor A at level 3 and factor B at level 2 Experiment Design Tools Interaction Plot An across functional team conducts a DOE during the design of an improved composite part. 2 experiment). each one corresponds to a particular level of factor A. they should consider the Main Effect and Interaction plots in order to identify the optimal settings. The setting 2 for B is interesting since it gives the largest output difference between the three settings of factor A." The design engineers recognize that before specifying any setting. Improve 60 1_05_01_059 The largest output average correspond to factor A at level 3 and factor B at level 3 . the output or response is almost linear for the different levels of factor B. 2 Each factor was set at three levels (1. The highest average output is obtained when both factors A and B (pressure and temperature) are at their respective level 3. The team studies the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine. the Black Belt creates and interprets the "Interaction Plot".e." "Another interesting point to note is that the response average is independent of factor A when factor B is Low. of the maximum or the minimum of the curves relative to factor A settings.

Steps 1 and 2 are to define the problem.Center points Sometimes it is desirable to study quantitative factors at more than two levels. "How do you add center point to your design?" asks the Champion. should be stated in such a way that it provides guidance to those designing the experiment. Examples of situations that would lead to three or more levels include: • • • Checking for quadratic effects (curvature). or confirming the results of past studies under new conditions." The Champion recognizes that. The objective statement should clarify whether the experiment involves a screening experiment. which is used to study a large number of variables. to find which has the most important influence. In this case. The objective of the cycle. . preferably with a statement of planned action. but the center point gives an indication of whether it is appropriate or not to interpolate between the factorial points. Strategy of Good Experimentation Step 1 – Define the Problem Step 2 – Establish the Objective Step 3 – Select the Response Variable(s) Step 4 – Select the Independent Variable(s) Step 5 – Choose the Variable Levels Step 6 – Select the Experiment Design Step 7 – Collect the Data Step 8 – Analyze the Data Step 9 – Draw Conclusions Step 10 – Achieve the Objective Strategy of Good Experimentation "I want to conduct an experiment on my process". A well-constructed problem statement. we may address these concerns without a significant increase of the number of experiments." "Note that the magnitude of the curvature effect can be directly compared to the factor effects for relative importance if the number of center points is equal to one-half the number of factorial points. 2 needing only 6 experiments (i. says an Black Belt to a Master Black Belt. since its p value is less than 5%. the ANOVA table shows that it exists a significant effect due to the curvature at the center point. combined with a good objective statement for the planned experiment. and to establish the objective for the experiment. the error is calculated and the curvature is detected. is a key part of the improvement cycle. with only 2 center points. Including the current level of the factors as well as the high and the low levels.e. studying a few variables in depth. "We perform "n" replicates run at the mid-level for each of the factors in the factorial design. k By adding center points to a 2 factorial design. 4 runs for the 2 with one replicate plus 2 runs at center point) compared to a 2 3 full factorial design which needs 9 runs. "Can you outline the steps I should take to ensure that it is successful?" "We follow a 10 step process to ensure that we cover all the main topics of a good experiment. Estimating the error. The curvature cannot be assigned to a particular factor. the number of center points does not affect the other effects. Also.

"Well. all interested parties should contribute to the objective. Having representatives from other disciplines working on the objective of the experiment. after completing the measure and analyze phases. The makeup of the team should depend upon the nature of the problem. is that it should be helpful in identifying the response variables. a statement of the results required should be considered in the objective when appropriate. Team Selection for DOE’s Key tasks for the Experimental Team: • Write Problem Statement • Define the Objective of the DOE Study • Record relevant background details of the process • Define Response Variables • Define Controllable Factors • Define Held Constant Factors • Define Uncontrollable Factors Improve Experiment Design Tools 63 1_05_01_062 Team selection for DOE's "Who is involved in completing a problem statement and objective statement for an experiment"? asks the Black Belt. because the team determines all aspects of the experiment". replies the Master Black Belt. the objective for an experiment might be to identify factors that call be used to improve yield by 5%. can often result in a study that is more useful to a range of interested parties. For example. As an example. before other work on the experiment is begun. The team selection process is crucial. and appropriate factors for the study"." Another consideration in the statement of the objective."Where possible. or identify the factors that have the greatest effect on the variation of the parts. an experiment to be designed and conducted in manufacturing may be of interest to those in research or engineering. replies the Master Black Belt .

it may matter that the temperature is taken from a single position in the bath. 3.002 Experiment Design Tools Defining the CT Characteristics A Black Belt asks a Master Black Belt how he goes about choosing the CT Characteristics to be studied by an experiment. In using a designed experiment. prior to the start of the experiment". Improve 64 10 .Define the “CT Characteristics” Experiment Planning Sheets Agent: _______________________ Department: ___________________ Process: ________________________ Experimental Planning Sheet Page 1 Date: ________________ Problem Statement: DOE Objective: Relevant Background: List of Experiment Parameters CT Characteristics Relationship to Objective Specifications C C T haracteristics W thickness on eb W Pickup ing Brackets Surface Finish 1_05_01_063 R elationship to O bjective Estim absolute M ate ean difference from target and Standard D eviation Team has determ ined that the surface finish should be as high a quality as possible. diameter. to the knowledge that leads to optimum operating conditions. with the minimum of variation. For example. has usually been defined in the measure and analyze phases. but there are a few things we should consider in their selection: 1. the CT Characteristics for experiments.20 M icrons (R oughness ) . To reduce measurement error. or concentration etc. but what makes a good response variable?" asks the Black Belt.000 . temperature. and they may be relative units. if the experimental measure is a chemical bath temperature. or at different positions throughout the bath. such as percent of concentration by weight or by volume.0. Where possible. a gauge R&R study has been carried out on measurement systems. such as weight. 2. The response should capture as much information from the process as possible. The measurement units should be appropriate for the study. replies the Master Black Belt. or averaged over the surface area of the bath. it is vital that for each CT characteristic. "This is step 3 in conducting a good experiment. 4. Specifications N inal target om + 0. Where possible continuous variables should be selected for output responses. Each experimental condition will produce data that can be used to find conditions that will optimize the process output on that desired target. As you said. we should try to have a response that meets a target condition. we always try to bridge the gap between our current knowledge and experience on the process under study. "We usually have defined the CT characteristics during the measure and analyze phases.

or combination of factors. replies the Master Black Belt. In considering step 4 of conducting a good experiment. 2. all the potential factors that can be used in the experiment can be broken down into three types: 1. The wider the range of conditions included in the experiment. Factors that are uncontrollable. believe will have the greatest effect on the CT Characteristics. Factors that are held at a constant level and are controllable. We do this to determine which factor. 3. usually called nuisance or noise factors. As with response variables it is seemingly easy to generate a list of candidate control factors. The control factors are chosen carefully. The selection. will be the conclusions from the experiment". while reducing variation. "Factors chosen for the experiment are usually those that the Black Belt. of important control factors. Improve 65 1_05_01_064 . Control factors (discrete or continuous) thought to be very influential. has the greatest effect of centering our process on a target. the more generally applicable. as they are the factors that we will use to vary the input settings of the process throughout the experiment. and estimation of the effects these control factors will have on the response variables is very important.Define the “Control Factors” Experimental Planning Sheet Top of Page 2 Controllable Factors Control Factors Normal Settings Proposed Settings How are they Set Control Factors Feed Rate Normal Settings 3” per minute Proposed Settings 1 Inch per minute and 5 Inches per minute 90 PSI and 150 PSI How are they Set Machine Control pre-set by the operator Operator control set using a calibrated gauge Air pressure 120 PSI Experiment Design Tools Define the "Control Factors" "What should I think about when considering factors for an experiment"? asks the Black Belt. and the team.

but cannot vary in our experiment". "Yes. It is vital. and held constant throughout the experiment. asks the Black Belt. . All operators to know how to operate the process How to Control Conducted runs when machine is at temperature Use one “average” operator in the experiment How are they Set Machine to be running 2 hours prior to start of experiment Chosen by supervisor ( For the experiment ) Improve 66 Experiment Design Tools Define "Held Constant Factors" "Is it important to think about factors that we can control and keep at a constant setting. The personnel carrying out the experiment must be made aware of this fact. this is a key aspect of designing any experiment. that show up as special cause variation during our analysis of the completed experiment". because the team only considered the control factors. they were going to change throughout the experiment. that other factors that you feel are important. but that you cannot vary are monitored closely. and they must record any occurrence where something changes. Many good experiments fail.Define “Held Constant Factors” Controllable Factors Control Factors Normal Settings Proposed Settings How are they Set Experimental Planning Sheet Bottom of Page 2 Factors Held at a Constant Setting Constant Factors Desired Setting How to Control How are they Set Constant Factors Temperature of machine oil Operators 1_05_01_065 Desired Settings 88 to 96 0F. This may allow us to determine a reason for any data points. in one of these factors that are held constant. replies the Master Black Belt.

and hold constant during an experiment?" asks the Black Belt. In many experiments. is a factor we think would have an effect on the CT characteristic. . "These factors are called noise. We can use the blocking technique if we think a noise factor. that is known and controllable. those that are known and controllable. or nuisance factors. replies the Master Black Belt. However. the variation due to background or nuisance variables will be as great. will have a significant effect on the CT characteristic. those that are known but uncontrollable. it important to try to minimize the effects of all types of noise throughout the experimental sequence. Generally a noise factor. and finally those that are unknown and therefore uncontrollable. than the variation due to the chosen factors. or materials stacked during experimental runs. or greater.Define “Noise Factors” Noise Factors Experimental Planning Sheet Top of Page 3 N oise Factors Strategy Em ployed to Control N oise Factors Random ization & Replication Conclusions from the Statistical Analysis Noise Factors Ambient temperature in the Clean Room Vibration of Machine Strategy to Control Noise Factors Make all runs below 70 0 F Experiment Design Tools Define Noise Factors "What do you do about factors that you cannot control. Improve 67 1_05_01_066 Ensure that no fork lift trucks pass by. and this usually achieved through a complete randomization of the experimental sequence". they can be grouped into three types.

and adding this amount to the upper specification limit. If we have a quantitative variable with specifications.Choosing Variable Levels Y Factor Settings True Effect I think you can turn off the water now. In the majority of cases. a key consideration in selecting factor levels. "Where possible. The upper limit of safety has been reached! Experimental Effect Lo (-) Y Hi (+) Factor Settings True Effect Lo (-) Hi (+) Factor Settings Experiment Design Tools Choosing the Variable Levels "Once I have chosen my factors for the experiment. However. then it may be possible to assess its impact. "In step 5 of conducting a good experiment. with many possible levels. and make it difficult to assess the significance of the control factors. the range of factor levels. and tightly controlled throughout the experiment. unless you vary the factor levels enough. the high setting for the factors can be determined by computing approximately 20% of the distance between the specifica-tion limits. Where possible. replies the Master Black Belt. selected for the study. the 20% value is subtracted from the lower specification limit. this may inflate the experimental error. For the low level. if you can define levels for a nuisance factor. must be the safe operation of the experiment across all the experimental conditions". "What about factors that are held constant throughout the experiment and nuisance factors"? asks the Black Belt. through the use of blocking". they differ from held constant factors in that they cannot deliberately be kept at a constant level throughout the experiment. then we should select levels using extreme values when possible. If we have qualitative variables. As far as nuisance / noise factors are concerned. is there any tips you have for selecting factor levels for the experiment"? asks the Black Belt. Improve 68 Experimental Effect 1_05_01_067 . will ultimately determine the degree of belief in the data obtained as a result of the experi-ment. held constant factors should be kept at their nominal settings. replies the Master Black Belt. the chances of getting good experiment results are greatly reduced.

from experience gained. which is to select the experimental design that will meet the stated objective of your study. to focus on finding the key factors affecting a process. two to four factors can be studied in a full factorial design. what is the next step in planning an experiment". and some new factors. "Well! You are now ready for step 6. Improve 69 Much Knowledge 1_05_01_068 Confirmatory Full Factorial Studies . and their associated levels. experiments can be repeated using new levels for factors previously studied. in order to increase the degree of belief in the data obtained from the previous experiments". New factors) Experiment Design Tools Selecting the Experimental Design "Once I have established all the factors.Selecting the Experimental Design Current Knowledge Type of Experiments Little knowledge Fractional Factorials (Screening Studies) Some Knowledge Factorial Studies (New levels. If we wish to study the relationships between the factors in depth. replies the Master Black Belt. asks the Black Belt. of conducting a good experiment. As new theories are developed. These fractional factorial designs are used to examine five or more variables in these initial studies. As we gain knowledge. The initial phases of DOE's will usually be screening studies. a confirmatory experiment is usually run.

Replication provides a measure of the magnitude of variation in the experiment due to nuisance variables. but what benefit does it give us"? asks the Black Belt. will not give us confidence in our experimental results. that many studies will have budgetary constraints. directly affects the amount of resources needed (time. of the resources allocated to the total DOE study.) for the study". replies the Master Black Belt. "Surely the amount of replication in an experiment. DOE runs on a milling machine carried out on different days. materials. it's a fact of life. "As a guideline. is the primary means for analyzing the stability of factor effects. Your costs in PETMET should reflect the full cost of all your proposed DOE cycles. replies the Master Black Belt. over a small range of factor settings. Improve 70 . but the breadth of conditions under which the comparisons are made. is not the number of replications per level of a factor. and give us increased confidence in the results of the experiment. etc. However. budget. on the main factor effects. the initial experiment should consume about 25% or less. For example. by enabling nuisance variables to be averaged out". "Replication in a well designed experiment. Replication. As a rule of thumb.Replication Pressure : Temp: HHHH LLLL HHHH LLLL HHHH LLLL HHLL HHLL HHLL HHLL HHLL HHLL 1st Replication Yiel d 130 120 2nd Replication 110 100 In d e x 1_05_01_069 5 1 0 1 5 2 0 2 5 Test Sequence Experiment Design Tools Replication "I know that replication refers to repeating aspects of an experiment. would be preferred to a study based on consecutively produced units which in DOE terms is called repetition. it is desirable to carry out between 2 to 5 replications for each run". and made from different material batches. This minimizes the impact of noise variables. But always remember the most important consideration in conducting a DOE. asks the Black Belt.

why does this happen?" asks the Black Belt. after restricting important background variables in the form of blocking. For example. "A rule of thumb with respect to randomization for analytical studies is. -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 2 1 1 2 1 2 2 1 1 2 2 1 2 1 1 2 Reac Temp Time A B C 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 230 230 300 300 230 230 300 300 230 230 300 300 230 230 300 300 30 30 30 30 60 60 60 60 30 30 30 30 60 60 60 60 Cat D Std Std Std Std Std Std Std Std New New New New New New New New Run Day Run Night 8 8 7 1 4 13 7 3 6 3 2 5 6 2 1 5 Randomized sequence for both experimental blocks Improve 71 1_05_01_070 Blocking disallows the given variable from influencing the primary response variable. would be randomized to assign combinations of factors to be carried out during each shift. Noise Factors are process variables that affect the CT Characteristic but have not been identified by the Black Belt or have been identified. changing the level of one particular factor might require a machine to be shut down for most of the day. and found to be prohibitive". or position of parts in an oven etc". is a tool that addresses the problem of noise (nuisance) Factors. Typical nuisance variables are environmental effects such as temperature or humidity. but are not controllable. the runs are automatically randomized. the randomization of the DOE is carried out in Minitab within each block. an experiment with a block based on day and night shift. randomize in all remaining situations unless the constraints to randomization have been considered. In cases where it is difficult to fully randomize the design. rather than over all runs in the design. the effect of a machine warming-up. Randomization. "In producing any design in Minitab. that standard order of the experimental runs is not in the order shown in my textbooks. Experiment Design Tools Randomization "I notice in Minitab. . batch-to-batch variation in raw materials. replies the Master Black Belt. whereas changing all the other factors could be done in a few minutes. In some other cases. unless you elect to turn off this function. "When blocking is employed. For example.Randomization Run Order 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 B 2 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 C 3 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 D Block 4 No. a random order for the conduct of a test could be prohibitively expensive. a great deal of care should be taken in interpreting the results of the experiment". or have been identified and are controllable.

replies the Master Black Belt. • Record any unusual occurrences that happen when running the experiment. that needs to be carefully monitored throughout the experimental cycles". the runs are randomized in accordance with the blocks. asks the Black Belt. • Hold all other factors as stable as possible during the experiment. They will need to re-set the control variables. they say that there is no such thing as a poor experiment. the accurate setting of controllable factor levels. It is vital that the person. or persons. are a key component for a successful experiment. Poor planning leads to poor execution. are another aspect in successful experimentation. TA DA • Reset each controllable factor as accurately as possible after each experimental run. I cannot over emphasize the need for highly capable measurement equipment. as the results of the experiment can be greatly affected due to a large degree of measurement error. "Collecting the data is step 7 of conducting a good experiment. they will also need to record in detail any unusual occurrences at are present during the experiment. to verify and improve the data collection procedures. throughout the experimental sequence. Improve 72 1_05_01_071 • Measure the CT characteristics as accurately as possible. . (Ensure the design has been randomized in Minitab). The operators running the process. then hold them constant. Experiment Design Tools Collecting the Data "Are there any things I should take into account when running the DOE. who will be carrying out the DOE and setting the factor levels. only a poorly designed or poorly executed one. are fully conversant with the need for rigor and attention to detail. • Ensure that if Blocking is involved in the design.Collecting the Data Key tasks in Collecting Data: • Ensure that the experiment is run in a randomized sequence. So. before starting the actual experimental sequence. and usually measure the output from each experimental run. and collecting the data?". is to run a pilot study. A way to ensure that this rigor and attention to detail is covered. as defined by the DOE design. not only will they have to run the experiment in the randomized sequence.

Improve 73 Investigate 1_05_01_072 . and checking the residuals I always like to determine if the results of the ANOVA analysis have any practical significance". "Yes. but be of little practical use in improving the process. and interaction effect. has on the process". "Are there any more steps in analyzing the data?" asks the Black Belt. The first task in analysis is to construct an ANOVA table for the full model in Minitab. replies the Master Black Belt. asks the Black Belt. Using the function Stat > ANOVA > Residual Plots. and Stat > ANOVA > interaction Plot. Stat > ANOVA > Main Effect plot. surely we have determined all factors and interactions that are significant from the ANOVA table?" asks the Black Belt. I then store the "Residuals" and "Fits".Analyzing the Data (ANOVA) Create a General Linear Model ANOVA No Does the residual model fit the assumptions of normality? Check the reason for the non-normality in the residuals and rerun that experimental combination if required Create Main Yes Identify the vital few factors with Practical Significance (Pareto or Pie Chart) and or the Normal Probability Plot of Effects Experiment Design Tools Effect Plots Interaction Plots Analyzing the Data (ANOVA) "What steps should I follow when I analyze my experimental data?". "Analyzing the data is step 8 of conducting a good experiment. after that we produce main effect plots and interaction plots using the functions. We can also produce a "Normal Probability Plot for Effects". for a graphical interpretation of main effects. So. "What do you mean by practical significance. replies the Master Black Belt. and ensure that the residual model fits all the assumptions of normality. and from analyzing these charts we can determine what practical significance each main effect. to produce a Pareto chart or pie chart. for graphical interpretation of two-way interactions". "It is possible for something to be statically significant. replies the Master Black Belt. After producing the ANOVA table. we analyze Sum of Squares in the ANOVA table.

Confine all initial conclusions and deductions. pertinence of problems. and make our recommendations to senior management. if further experiments are required. says the Master Black Belt. that it is vitally important to point out to the Champion. Limit conclusions to summary evidence taken directly from the experiment. but limit conclusions to objective summary of evidence provided by the study". A rule I always try to adopt. that you have considered all the observed data. "I always write a report clearly describing the experiment work. • Experiment Design Tools Conclusions and Recommendations "Once we have analyzed the data. is to supply sufficient information to permit readers of the report to verify results. When we have proven our findings. I find it of great benefit to focus on using tabular and graphic methods of presenting data. and ensure that the results can be reproduced. to the experimental evidence at hand. Point out the way ahead. and to draw their own conclusions. It is good practice to replicate the optimum set-up.Conclusions and Recommendations This way to an Optimized Process • • • • Check all observed data for accuracy. explain the results / conclusions in terms of practical meaning. says the Master Black Belt. "Some key points are to check. "What steps do you follow in drawing your conclusions?" asks the Black Belt. and if there is a need for further work". and gain buy-in from the Champion. and the owners of the process. This is step 9 in conducting a good experiment". the implication of the results. and have determined the practical significance of each factor and interaction. the results of the study. and the meaning of the results. Improve 74 1_05_01_073 ." "From this data. and the personnel who work on the process. Explain results of the experiment in terms of graphs as well as numbers. As we will need to explain the results of the study. always try to explain the analysis in both graphical as well as numerical terms. we determine the combination of factors and interactions. we have to draw our conclusions. as well as their statistical significance. that optimize our process to meet our CT objectives. I have found out. State results in practical terms as well as statistical terms. giving the background to the study.

Lessons Learned • In order to design an experiment. This is the main reason for the control phase. You will remember from your training. Once the experiment is completed. • • • 1_05_01_074 Experiment Design Tools Achieve the Objective "What do we do. and is set at the optimum operating configuration. Control key CT Characteristics using methods outlined in the control phase. we must first clearly understand the process and decide what feature it is that we want to investigate. "Well." replies the Master Black Belt. what comes next?" asks the Black Belt "If you have reached this stage in your experimental process. "When we have proven that the process is improved. that it was recommended that the first experiment consume only 25% of the budget and resources. such as SPC to control the critical to process parameters (CTPs)". Keep the process operating at that the peak of its performance. usually it is impossible to fully optimize a process based on one experimental sequence.Achieve the Objective • When the process has been optimized it is time to institutionalize the changes. It has been my experience. to ensure that entropy does not affect the optimized process. over time. Entropy. additional understanding of what is important will be gained and another experiment can be designed. Improve 75 . using control techniques. We do this until the desired result is obtained. is the fact that everything in nature decays. this knowledge. it will revert back to a less than perfect process. that you always find something from an experiment that no one knew about the process. and put controls on the key aspects of the process. You are now ready to institutionalize the changes. over time. replies the Master Black Belt. if we find that we have not optimized the process from our experiment?" asks the Black Belt. Control key process control factors using methods outlined in the control phase. formulated by the DOE's. We can improve and optimize the process but if we do not institutionalize the recommended changes. then you are ready to implement step 10 of conducting a good experiment. usually points out the direction that the Black Belt should go in setting factor levels for another experiment.

This notation (-1. We should consider both the main effects and interaction plots prior to specifying optimal settings. The two elements must be well thought through in order to collect appropriate data that can be meaningfully analyzed. A successful experiment does not happen by chance. actively communicate your plan with all those concerned is vital to the success of an experiment. The most common way to design an experiment (so that all combinations are considered) is to use Yates Standard order. A factor may be statistically significant. 1) is used to set factor levels and coefficients. It is common practice to do a fractional factorial experiment when we have five factors or more. • • • • • • • • • • • . all within one experiment. We use pie charts and Pareto diagrams to visually display the relative effects of several factors. The general steps for conducting a statistical analysis are ANOVA (with residual analysis). The ANOVA table is used to assess statistical significance initially and then practical significance of factor effects and interaction effects on the output. but not so much so that they extend beyond the operational range of the factor. Two factor interactions are common. Even if the main effect is not statistically significant. practical significance (sum of squares. but three factor and higher interactions are rarely seen in our industry. only an experiment that is poorly designed or executed.) and main effect plots (with interactions considered). In general. pie chart. We then often plot the main effects on a graph in order to visualize the difference in the response output between the low and high settings of each important factor. but practically insignificance. Compared to the traditional one-factor-at-a-time approach to experimentation. The Contrast concept is used in statistical analysis to determine if a design is balanced and to determine orthogonality.• • Each DOE has two elements: the "Experimental Design" and the "Analytical Approach". a DOE can provide a wealth of knowledge about several factors (and their interactions) in a very cost effective manner because several factors can be varied at the same time. There is no such thing as a poor experiment. Remember. Graphs are also useful to visualize interaction effects between factors and to find the optimal settings for both factors. etc. it may affect the output through an interaction with another factor. it takes careful planning and attention to detail to ensure that all designed experiments produce meaningful results. factor levels should be set wide.

Sign up to vote on this title
UsefulNot useful