Six Sigma

Experiment Design Tools
Experiment Design Tools
Most companies collect data to monitor the health of their processes, but most of this data is not acted on in any way because it does not contain meaningful information. Often, when these compa-nies decide to perform an experiment in an effort to produce more meaningful data on a specific CT characteristic, they design a onefactor-at-a-time test, but progress is slow and expensive using this technique. On the other hand, companies who learn to use Design of Experiments (DOE) are able to leverage their efforts by conducting well thought out experiments where several factors are altered simultaneously in one efficient series of tests. DOE provides a wealth of information and in a shorter period of time. The role of a statistically designed experiment is to identify the most influential factors (vital few) associated with a particular CT characteristic and to define their relationships using analytical quantities. Since interactions between various factors are also defined, a single DOE can yield many revealing facts allowing the experimenters to quickly improve their process. It is most effective to conduct DOE's early in the design process, experimenting with the effect of tolerancing and failure modes, so that when the product or service is introduced to the market place, customer satisfaction will be high and warranty claims minimal. The same experimental techniques can then be used throughout the life cycle to continuously improve, taking into account new information as it becomes available. There are two components to a DOE: Experimental Design and Analytical Procedure. It is best to think these out carefully before conducting the DOE in order to produce the best result. Depending on the level of understanding available about a process, there are various types of DOE that can be employed. Screening designs are typically used early in the process where many factors are involved, Characterization designs narrow the number of factors down to only a few, and Optimization designs focus on only one or two factors, but in much more depth. To ensure that a DOE is designed correctly, it is necessary to consider such topics as Yates standard order, factor levels using the –1, 1 notation, column contrast, balance, orthogonality, confounding and interactions between factors. Also, there are design factors that influence the time and cost aspects of the DOE, such as replications and the basic design type. The two-level two-factor design is relatively common in practice and makes a good starting point for most problems. When replications are taken into account, very meaningful results can be achieved with as few as 8 to 16 runs. When you don't know which factor has a substantial effect on a response variable, and you want to verify several factors, then the fractional factorial design is the preferred analytical tool. It is common practice to do a fractional factorial experiment when we have five factors or more. In this case, you only conduct runs which represent a portion of the matrix, because, for example, a 6-factor two-level experiment with no replications would involve 64 runs. The advantage, therefore, of a fractional factorial experiment is that fewer tests are required. When it comes to analysis of the data from a DOE, we must first evaluate the statistical significance. This is done by computing the One-Way ANOVA or the N-Way ANOVA (for more than one factor). The practical significance can be evaluated through the study of sums of squares, pie charts, Pareto diagrams, main effects plots, and normal probability plots. Note that there are situations where factors are statistically significant, but not practically significant. In any analysis, it is important to analyze the residuals prior to any conclusions being drawn. A step-by-step approach to designing and conducting any DOE must be adopted. The use of cross-

functional teams in the selection of factors to study, the design of the experiment, conducting the experiment, analyses of the data, recommendations, and implementation of DOE results all require careful planning and attention to detail to achieve success.

Key Questions
• • • • • • What is a "statistically designed experiment" and what roles do they play in Six Sigma? What are the guiding principles surrounding statistically designed experiments? How can the practical and statistical influence of a single CTP be exposed? How can the practical and statistical influences of two or more CTP's be exposed? How can conflicts between competing performance objectives be resolved? What is the process and guidelines for planning statistically designed experiments?

Key Questions A DOE is a systematic method where a number of factors are changed simultaneously, following a predetermined pattern to investigate their effect on the response. The role of a statistically designed experiment is to determine the most influential factors associated with a particular CT characteristic (vital few) and to understand their relationships using analytical quantities. The fundamental guiding principles surrounding statistically designed experiments are: Process Relationships, Replication, Randomization, and Experimental Control. The statistical influence of a single CTP can be exposed through hypothesis testing using tools such as OneWay ANOVA, and the practical influence can be exposed through the study of sums of squares presenting the results in pie charts or Pareto diagrams. The statistical influence of two or more CTP's can be exposed through hypothesis testing using tools such as NWay ANOVA, and the practical significance can be exposed with the use of graphs such as main effects and normal probability plots. Conflicts between competing performance objectives can be resolved by the use of interaction effects between factors in order to find the best compromise settings. The process and guidelines for planning statistically designed experiments are: Define the problem, set the objective, select response variables, select factor levels, conduct experiment(s), analyze results, and ensure results are well understood and communicated.

Why/When to Experiment
High Focus on defect detection and correction


Cost to deal with poor quality Focus on defect prevention

= f(X1, X2, …, Xn)  CTPs  Factors Improve

 CTQs or CTCs or CTDs


Low Product Design
Experiment Design Tools

Process Design



Why/When to Experiment At a planning session, Six Sigma team members are evaluating whether or not they should conduct an experiment on their process. They ask the Black Belt to outline a criteria for this. The Black Belt explains why and when to use experiments: • • • • • • • Experimentation can be used to improve the performance, quality and cost associated with key customer requirements. We use them to confirm and quantify the important inputs (CTPs) to our processes. Determine the "vital few" variables when there are no analytical equations to guide us. Experimentation can significantly reduce new product (or service) development cycle times and it is conceivable that manufacturing tolerances can be relaxed. A robust design can be created if we test it prior to design freeze. A product or service that is robust to variations in input factors and operating environment will cost society less over the life of the product, resulting in its customer loyalty. The need for product inspection and performance testing can be significantly reduced with extensive experimentation. We should always experiment as early in the cycle as possible, so as to improve quality, thus reducing rework, warranty claims and dissatisfied customers.

The Black Belt draws the above chart on the blackboard, summarizing his point that we should begin to test key business opportunities with a series of well planned experiments as early in the development cycle as possible in order to reduce the cost of poor quality, and in addition, throughout the product life cycle to achieve continuous improvement.

and probably the most important principle". Replication Increased precision of effect estimates Provide measure of experimental error 3. etc. the energy imparted by the falling apple is the process of interest to Isaac Newton. says the Instructor. "The first." "Replication is the second principle of the experimentation process. The Instructor outlines the four basic principles. wind velocity. Replication allows a measure of experimental error. He set out to design a series of experiments to determine the relationships between the mass of the apple and the fall distance as they relate to the energy imparted. These variables will probably have small effects on the results.Fundamental Principles of Experimentation 1. Usually." "The fourth fundamental principle of experimentation is what is known as experimental control. while carefully controlling all other variables as much as possible. which will result in experimental error." . This simply means that you deliberately change some variables. Randomization Select experimental units randomly Randomize run sequence 4. Understand Process Relationships • • • • • • • • Among process variables Among response variables 2. using the example of the classical apple falling on the head of Isaac Newton. a new Black Belt asks the Instructor to review the fundamentals of Experimentation. "is to fully understand the process. In this example. two to five replications are conducted for each set of conditions in order to ensure that the "white noise" is captured. such as temperature. Experimental Control Active variation of process variables Control other sources of variation Fundamental Principles of Experimentation During Black Belt training." "The third characteristic of an experiment is to ensure that the order of conducting a series of runs is randomized in order to avoid false conclusions due to factors which may vary over the course of the experiment. noting the effect.

which we cannot control. we can then decide how to change the settings or levels of some factors in order to produce better yield. i. and so on. The Master Black Belt concludes by saying that unless we experiment with our processes.the manipulation of controllable factors ( indepen dent variables ) at different levels to see their effect on some response (dependent variable). Factor D has no effect Y Experiment Design Tools Y Role of Experimentation The Master Black Belt is asked to define the role of experimentation by the Champion. Factor C affects the mean and the standard deviation C1 1_05_01_004 4. improved quality or some form of improvement of our CT characteristic. The examples above imply that each factor can have independent effects on the result.e. Experimentation provides us with a sound. that is. We would then design a new experiment. The test results may lead us to select new factors which were dismissed or overlooked initially. He explains that a statistically designed experiment is a test or a series of tests in which purposeful changes are made to the input variables of a process or system so that we may observe and identify reasons for changes in the output response. The objective in many cases may be to develop a robust process. guided by our judgement. Factor B affects the standard deviation B1 A2 Y A1 B2 Y 3. 1. but this is not always possible.: white noise.Role of Experimentation Experimentation . Factor A affects the mean 2. Improve 5 C2 D1 D2 . a process affected minimally by the many sources of variability. Once we have obtained the results of these tests. until we achieve the desired results. we will never identify ways to improve those processes. We strive to find factors which have independent effects like this. effective and economical method for determining the best way to change.

The team now felt that they knew how to go about selecting factors for their first experiment.Sources of Experimental Factors Input Factors (Xs) People Material Equipment Policies Procedures Methods Environment Output (Ys) Service PROCESS CTC A blending of inputs to achieve the desired outputs Product CTQ CTD Task 1_05_01_005 Experiment Design Tools Sources of Experimental Factors At a Six Sigma team meeting. etc. each factor needs to be independently controllable. etc.) or can be discrete (like new/old. machine 1/machine 2. Improve 6 . The Black Belt also says that they need to be aware that factors can be continuous in nature (like volts. He suggests that they review the FMEA's that had been completed recently by the team. a team member asks the Black Belt if he could help the team select factors for an experiment on their process. since these contain an excellent checklist of what is important to this particular process and act as a summary of the collective knowledge about the process from the brainstorming sessions.). Also. pressure. The Black Belt points out that factors can come from a wide range of sources as shown in the list above and that it is impossible to select the most influential factors unless you fully understand the process.

Types of Experimental Factors CONTROLLABLE INPUTS Factors that we can control. the factors that cannot be controlled are important to identify because there may be ways that their effect can be reduced by careful design of the experiment. and elect to hold constant for the experiment OUTPUT (Y) INPUTS (X’s) Factors that we wish to vary in a DOE • Experimental Error – Measurement – Other • Environment UNCONTROLLABLE INPUTS 1_05_01_006 White Noise PROCESS UNDER STUDY Response variable has variation due to: • Effects: – Factors – Interactions Factors that we cannot control • Environmental • Unknown Experiment Design Tools Types of Experimental Factors The team realizes that there are different types of factors involved in their process – some controllable and others uncontrollable. "The first step". or just because it doesn't make sense. and which ones you can move to the controllable (but hold constant) category. The Master Black Belt draws the above chart on the board and describes a process he uses to ensure that all factors are considered when designing a DOE. Before beginning. you need to decide which ones you wish to vary in the experiment. "is to divide all known factors into one of the three categories as shown in the chart. Also. he reminds the team that a review of the FMEA for their process may provide clues as to how to categorize the various factors. They ask the Master Black Belt for advice in how to handle these. resulting in as few runs as possible.e. Of all the possible inputs that could be used in a DOE. the experimenters will already be more knowledgeable about their process and be ready to begin the design of their DOE. or recording of data that would otherwise have been ignored or overlooked. especially ones that could be varied. It may be time well spent to put as many factors into this category as possible. but are not likely to be allowed to vary for certification reasons. He pointed out that. What is "Design of Experiments" (DOE) Definition Improve 7 . says the Master Black Belt." The Master Black Belt goes on to say that the input factors remaining are likely to be few (i. 2 to 5) so that the experiment can focus on only these. by the time this exercise is finished.

The Black Belt outlines other features of a DOE: • • • • • It is a way of separating the vital few from the trivial many factors. It is a structured way to economically improve any process. He explains that a DOE is a way to generate information about a process which can then be used to improve it.General The controlled. The Black Belt explains that a DOE must have two basic elements – a design where the team decides what to focus on. What is "Design of Experiments" (DOE) When the Black Belt is requested to define the concept of a DOE to the Champion. the DOE process consists of a series of experiments where competing settings of each factor are combined in order to efficiently reach valid and relevant conclusions. to investigate their effect on the response or output. It is a planned set of trials (runs) with specific measures. engineering design. or to manufacturing processes. Specific A DOE is a systematic method where a number of factors are changed simultaneously. It is dynamic. and a procedure for analyzing the data once it has been collected. I am interested to see if I can apply this tool in the Customer Services area". . following a predetermined pattern. carefully recorded. active variation of a process to identify better settings or procedures to improve the product quality. As can be seen. he uses the above definition. testing many factors (and interactions) in a short time. The Champion says: "Tell me more. It is a systematic scientific approach which allows decisions to be based on facts. In conclusion. this definition applies equally to transactional.

In order to get worthwhile results. Improve 9 . The two knobs represent the two variables or factors of the experiment. raw material. meaning that four combinations are possible to get results". "This is a very simple example that we've done on the board and it represents a part of the planning of a DOE. why don't we use this tool more often?" asks a Champion." "It seems to be very easy. we will use the shower example. conducting an experiment requires human and material resources. X2) Water Temperature = f (flow of hot water. In this case. i. Who wants to volunteer?" asks the Master Black Belt. flow of cold water) Experiment Design Tools The Shower Experiment During a refresher session on Design of Experiment (DOE). a Champion asks a Master Black Belt: "Do you have a simple example to show us?". 2 . "We now can prepare the design matrix with factor Flow Rate of Cold Water and Flow Rate of Hot Water. you will have optimal water temperature which represents the response of the process.The Shower Experiment Flow Rate of Run 1 2 3 4 Cold Water Low High Low High Flow Rate of Hot Water Low Low High High 1_05_01_008 Y = f (X1." "In most industries. etc. "Remember. we use two levels to do an experiment. The process is quite simple. "Of course. each DOE should be carefully planned with all the required resources" concludes the Master Black Belt. Is it obvious to determine which are the best settings? The experiment itself still needs to be done.e. two-factor matrix. money. let's suppose that the knobs can be set only at high or low levels. parts. "Then we 2 have a two-level. time. By adjusting hot and cold knobs.

. Experimental Design: A means to structure the variables for experimental manipulation. Analytical Procedure • ANOVA • Regression • Etc. • • • • • • • • • Fixed Effects The experimental levels are selected by the team. Most experiment designs are of this type. 2. Experimental Design • Fixed Effects • Random Effects • Mixed Effects Questions: What result do we expect to get? What are we going to measure? How will we collect the data? How will we present the data? Questions: How will we analyze the data? How will we find relationships? How will we find interactions? 1_05_01_009 2. Mixed Effects A combination of fixed and random models. This can be broken down into three types of effects. Conclusions can be generalized to the population. Improve 10 How sophisticated does the analysis have to be? Experiment Design Tools Components of a DOE In his presentation describing the various features of a DOE. Conclusions apply only to selected factor levels.Components of a DOE 1. Random Effects Levels are randomly selected from a population. the Master Black Belt feels he should go into more detail. • Analysis of variance (ANOVA) • Regression • Find a relationship between variables. Analytical Procedure: A means to extract relevant information from the experiment. He summarizes the questions that one needs to ask as shown above and explains the two components as follows: 1.

trying to convince him otherwise. it would be very time consuming and expensive to do a series of experiments "one-factor-at-a-time". The Six Sigma Black Belt. The Black Belt illustrates this with a two-factor example. this becomes more pronounced when more variables are involved. and would be missed completely in the "one-factor-at-a-time" approach. but if we overlay this picture onto the real situation (3 above). Why Not One-Factor-at-a-Time? The autoclave operator insists that he should not vary more than one parameter at a time. fearing that the results would be impossible to interpret. explains that since there are so many factors (potential CTPs) involved in even simple processes. Benefits of Statistical Experiments Change One Evaluation criteria Time to get results Cost Chance of detecting optimum Confidence in results Ability to identify independent interactions and main Expert Observe Very Long Very High Very Low Very Low Long High Medium Medium Long High Overall High Overall Suboptimum likely High Guess Opinion Everything Factor Stat at once At-a-time Long Short Rel Low Very Good High DOE Low Low Moderate Moderate Moderate Very Low . the pressure (factor B) is varied to determine its best response (2 above). in random order. One may conclude that this process defines the optimum settings. to yield a matrix of effects. A much better approach is what is termed a "full factorial design" where a few factors are varied at the same time. Then by keeping temperature constant. This approach includes interaction effects which can be very important in some cases. He explains that by varying the temperature (factor A). As one can imagine. in the majority of cases. He also points out how focusing on "onefactor-at-a-time" will probably miss the optimum combination of factors. the Master Black Belt states that. we use "Fixed Effects" designs and ANOVA for the analysis procedure. the so-called optimum is far from defining the maximum potential value (4 above). a zone of maximum response is determined (1 above). The Black Belt sits down with the autoclave operator and the Six Sigma team to design such an experiment.In conclusion. They are anxious to try a DOE because they feel that there are significant improvements that can be achieved.

The cost of a DOE is much lower than the cost of poor quality (long-term). The above table is the result of his efforts.effects None None None None None Yes Conclusion: Statistical Experiments are Rich With Information!! Benefits of Statistical Experiments A Black Belt observes over the course of the past year that there is a lot of folklore about the benefits of observation. low cost. . the statistical experiment is far superior than any of the others. in the short-term. not replace it. optimal results with a high degree of confidence. the Black Belt makes some notes in the way of conclusions: • • • • • • Statistical experiments provide fast. judgement and "one-factor-at-a-time" as techniques for deciding how to make a change to a process. so they must be properly planned. Proper experimental design is considered to be more important than sophisticated statistical analysis. Using a subjective scale. one can see that. they do cost money and disrupt the process. for all criteria selected. In addition. We use experiments to supplement our judgement. He decides to map the various techniques and to identify their strengths and weaknesses compared to statistical experiments. Properly designed experiments will permit simple interpretation of results. which may not be possible otherwise. statistical experiments are the only technique to provide the data for "main effects" and "interactions" between various factors. However. After creating the table.

"let me write out what I'll call the "hierarchy" of experiment types." replied the Instructor.Types of Experiments Very Informal • Trial and Error Methods – Introduce a Change and See What Happens – Produced Under Controlled Conditions • Pilot Runs – Set up to Produce a Desired Effect • One-factor-at-a-Time Experiments – Vary One Factor and Keep All Other Factors Constant • Planned Comparisons of Two Methods – Background Variables Considered in Plan – Study Separate Effects and Interactions • Experiment With 5 to 20 Factors – Screening Studies 1_05_01_012 – Modeling. At this point in the course. but they are amazed to think that up to 20 factors can be dealt with within one experiment. He went on to write the headings that describe the Six Sigma approach. Improve 13 Very Formal • Comprehensive Experimental Plan With Many Phases Six Sigma Approach • Experiment Planned With Two to Four Factors Traditional Approach • Running Special Lots or Batches . a Black Belt in training asks how the traditional approach that he had been familiar with compares to the Six Sigma approach. they would readily see that it gives much better information than the traditional approach and that the new tools would be easier to use. "Good question." He writes down the first four headings and everyone agrees that they know what these involve. the students are not familiar with these. The Instructor replies that they will still be used on occasion – a special lot or a pilot run could still be used on occasion if you were wanting to check the data collection process for calibration purposes. The Black Belt in training asks: "Does this mean that we will no longer use the traditional types of experiments?". etc. Multiple Factor Levels. He said that once everyone had some experience with the Six Sigma approach. Optimization Experiment Design Tools Types of Experiments After having some exposure to the basic concept of a DOE.

Such designs are usually based on full factorial models using two or three levels. In practice. he draws the above chart and elaborates on each as follows: • • • Screening designs are generally used to identify the "vital few" process factors. each experiment is different from the ones before. Optimization designs (also referred to a modeling designs) are used to study complicated effects and interactions involving one or two factors. and rarely with optimization designs. the results of one being used to design the next. such designs are employed to yield a precise mathematical model of the process under investigation and may involve several levels (to capture nonlinearities). Improve 14 . Characterization designs are generally used to study simple effects involving a small number of factors (3 or 4). being tailor-designed for the particular factor and levels being investigated. The Black Belt points out that the above class descriptions are very general in nature and should not be used as the sole benchmark when designing an experiment. more than one experiment is generally required to obtain a meaningful model.Classes of DOE Screening Characterization Optimization A Factors C F I E B K J D F B K J F K H G Time 1_05_01_013 Experiment Design Tools Classes of DOE There are several classes of DOE that can be applied to improve a process. Usually. He also points out that. such designs involve a large number of factors and are limited to only two levels. In general. When a team member asks the Master Black Belt to give an overview of how the various classes of DOE are applied. occasionally with characterization designs. putting the "trivial many" aside. Of course. we work with screening designs. most often.

etc. and that later on we will see how to arrange this data so the experiment can be conducted. which he does as follows: • • • • • • • • • DOE – Design of Experiment. Treatment Combination – Combination of levels for this element of the experiment.g. the Black Belt presents the above chart which represents the design anatomy of a typical experiment.: 20 psi. High Level – High setting (e. Order – Sequence order or run order for the experiment. CTD. He is asked to review the primary terms. Low Level – Low setting (e.e. Improve 15 .Terms Used in DOE Process Factor Variable A B C Coded Value +1 -1 +1 -1 Setting Pressure High (40psi) Low (20 psi) Machine Shift New Old Number 1 Number 2 Y = f (A.1 7. Level – Factor setting. B.7 9. C) Factors Treatment or Combination Process Variables B C -1 -1 +1 +1 -1 -1 +1 +1 -1 -1 -1 -1 +1 +1 +1 +1 Response Variable 5. The Black Belt points out that the matrix shown is in the form convenient for the design of the experiment.2 3. new. Factor – Process variable that will be purposely altered during the experiment.). the magnitude of the response variable.7 5. each team member has a clear understanding of the design structure of a DOE and begins to plan an experiment that applies to their process. Data – The result – i. From this brief description.g. shift 2. old.: 40 psi.3 10. or "an invitation for an information event to occur".3 8. etc. shift 1.). CTC). Response Variable – The subject of the experiment (CTQ.4 6.5 +1 -1 Sequence Order 1 2 3 4 5 6 A -1 Order +1 -1 +1 -1 +1 -1 +1 Data 1_05_01_014 Levels 7 8 Experiment Design Tools Terms Used in DOE As a means of introducing the topic of DOE to the Six Sigma team.

The two charts on the right hand side above show the interaction effect. 7).0 Y 5 3 ∆=2 AB+ Condition A+ 12 ∆=7 Main Effect 1_05_01_015 AExperiment Design Tools A+ Main Effects and Interactions Before beginning to design their experiment. B is another main effect. The interaction effects can be used to achieve the desired result. Likewise.: when B is high. Factors A and B are generally assumed to be totally independent of one another (2 vs.0 5.Main Effects and Interactions Y B. Interaction effect describes the effect of the product A x B on the response variable (Y) when two factors have a combined effect on the response variable.Condition Run 1 2 3 4 A -1 1 -1 1 #1 B -1 -1 1 1 #2 A*B 1 -1 -1 1 Interaction Effect Y 3. Improve 16 5 . The Black Belt uses the above example to describe the two terms. i. The Black Belt points out that one of the powerful advantages of a well designed DOE is that it has the benefit of being able to identify interactions which would be totally missed in a one-factor-at-a-time experiment.0 12. the team leader asks the Black Belt to explain what is meant by "main effect" and "interaction". the yield (Y) of the process is over three times better than when B is low. Main effect describes the direct effect of factor A on the response variable (Y).0 5.e.

Improve 17 . such as Low. -1) called coefficients in a design matrix." "Is it then accurate to state that the term contrast is used for more than one thing?" asks a Black Belt. A negative result means the inverse. a positive contrast means that the response variable is greater when the factor is set at its high level (+1). compared to the high level (+1). To define a contrast.5 = 4. for a main effect.5 4. and it also defines a set of coefficients used in a design matrix" concludes the Master Black Belt." "Thus.5 13. a Black Belt asks: "You mentioned the word CONTRAST in your explanation.ΣΥFVectored Vectored Column Column Response Response A -1 1 -1 1 Y DESIGN MATRIX Run Order 1 2 3 4 (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y 1.5 * 1.5 13. It is just a special way of making a comparison. etc.Contrast .e. 120 psi. In other words.5 = -4.5 12. i.5 = -1.5 * 4. High.5 4. "Yes.5 4. 80 psi. which we will learn later. The coefficients of a factor or interaction are used to calculate its contrast.0 B -1 -1 1 1 AB 1 -1 -1 1 Y1 1. that the response variable is greater when the factor is set at its low level (-1).5 = 13. a set of coefficients have to conform to conditions known as "balanced" and "orthogonality".0 Example Computation Run Order 1 2 3 4 1_05_01_016 A -1 1 -1 1 12. but I don't understand what it means". the contrast of one factor or interaction is the result of the sum of the coefficients multiplied by the response of each run. are replaced by coded values (+1. Then contrast compares two things or groups of things by comparing their difference.5 (1) a b ab Contrast Column Contrast Sum of Vectored Responses Experiment Design Tools Contrast – Definition During a training session on Design of Experiment (DOE).5 * 13. The Master Black Belt responds: "Actual factor settings for an experiment. actually contrast is an intermediate calculation used in statistical analysis of a DOE.5 4.Definition Contrast of factor F = ΣΥ F+ ." "As you can see on the visual. compared to the low level (-1).5 * 4. we can say that a column contrast is the linear sum of vectored responses.

Properties DESIGN MATRIX Run 1 2 3 4 A B AB 1 -1 -1 1 -1 -1 1 -1 -1 1 1 1 Balanced ΣX i = 0 for each factor sum 1_05_01_017 Orthogonal Experiment Design Tools Σ Xi X j = 0 for all dot product pairs Contrast – Properties During a training session on Design of Experiment (DOE). then the design matrix is said to be orthogonal. This feature ensures the effects are independent. this condition means that for one column. small effects will not be washed out or masked by large effects. For every pair of contrasts. take a design matrix for example." "When you closely look at the picture. we see that the AB interaction is created by multiplying the coefficients of factor A by the coefficients of factor B. This is the basic condition to have contrast. you don't have to build orthogonal contrast. a Master Black Belt says: "We know that a column of a design matrix is considered as a contrast under certain conditions. To be considered as a contrast. Actually. there is the same number of High (+1) and the Low (-1) levels. Because all effects can be independently estimated. and the sum of the coefficients of AB contrast is also equals to zero. Minitab software will automatically provide a balanced and orthogonal designs" concludes the Master Black Belt. When the sum of each column equals zero. We have factors and interactions between factors. Improve 18 . but I don't have a clue as to what these mean". In a case of a twolevel design. the experiment is said to be balanced. but do you know what these conditions or properties are"? A Black Belt responds: "Balanced and orthogonality. when the sum of the product equals zero. The Master Black Belt says: "Well. the sum of a set of coefficients must equals zero.Contrast .

If we wanted to have three replications instead of two. The use of Minitab makes it easy to lay out how to actually run the experiment. For a three-factor. if some unknown factor is changing continuously over the period of the experiment." Improve 19 . its influence will be minimized by this technique.Run Order for a DOE Randomized RunOrder 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1_05_01_018 Design Order 16 15 10 11 3 5 13 8 12 11 4 6 2 9 7 4 Factor A 1 -1 1 -1 -1 -1 -1 1 1 -1 1 1 1 -1 -1 1 Factor B 1 1 -1 1 1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 Factor C 1 1 -1 -1 -1 1 1 1 -1 -1 1 1 -1 -1 1 -1 15 16 Experiment Design Tools Run Order for a DOE Once the factors and levels to be investigated are identified. with two replications. In other words. you must then decide how many replications you wish to do. The original design order is given in the second column." A team member asks: "Why is it necessary to randomize the run sequence?". The Black Belt responds by saying: "Running the experimental trials in a random order protects against the effects of unknown nonexperimental variables (like environment) that might bias the results. the above table shows how the run sequence would appear. the list of runs would be increased by 8 and the whole list would then be randomized. The Black Belt states: "You should note that the original order from the design matrix is now randomized. two-level experiment.

" said the Champion. Select step sizes to ensure not missing the optimum.How to Select Factor Levels 80 Response Variable (Y) 70 60 50 40 B." The Black Belt outlines the important aspects as follows: • • • • • • • • • Draw on all past experience to provide knowledge of your process. Improve 20 . Levels for each factor should be balanced in relation to each other (i. Initial levels should be bold or wide. "Yes. and safe. If no spec limits exist.: no one dominant). reducing to narrower steps in subsequent experiments. "If that's true. practical. "but remember. "can you help us with a criteria to do this?". rational. Use a pilot run to verify sensitivity to level selections. it is much better to plan a series of experiments from the beginning than it is to try to do it in only one.e. A good rule of thumb is 20% beyond input spec limits. use ± 3 standard deviations to set levels.high 400 Y True Effect True Effect 500 A-high 300 30 B-low 20 10 0 A-low 200 B difference A difference 100 600 Factor Settings Y 1_05_01_019 Experimental Effect Lo (-) Hi (+) Lo (-) Hi (+) Experimental Effect Factor Settings Factor Settings Experiment Design Tools How to Select Factor Levels A Black Belt makes the statement that selecting factor levels is a critical step in determining the success of a DOE. Levels need to be logical. You should plan to spend only 25% of your allotted time and budget on your first experiment." said the Black Belt. well beyond white noise or measurement resolution. The intent is to capture a large meaningful result.

" The Black Belt asks the Master Black Belt for advice. The Black Belt says "We want to know the effects of the "quality of paper" on printing quality. the effect due to "quality of paper" will be confounded or mixed in with the printer in use.Confounding Effects MATRIX A Printer 1 Paper Type Low Paper Type Low Paper Type Low Paper Type Low ST AN DA RD Printer2 Paper Type High Paper Type High Paper Type High Paper Type High Coded values -1 1 Printer #1 #2 Paper Low Quality High Quality MATRIX B Run 1 2 1_05_01_020 Printer 1 -1 1 -1 1 Paper -1 -1 1 1 3 4 Experiment Design Tools Confounding Effects A Black Belt is working on a project to minimize the number of rejected reports due to bad printing quality that occur with the two main printers located in the computer room. we hypothesized that an increase in "quality of paper" increases the quality of printing. Strategy of Experimentation 1. I see that by doing this. After a rigorous study. assuming that the influence of all other background variables have been adequately controlled or otherwise. the factor "quality of paper" will be analyzed. "What is the objective of the experiment?" asks the Master Black Belt. Define the Problem 2. In other words. Establish the Objective Improve 21 . "If we design a new matrix as shown in B with both types of paper in both printers. During a coaching session with a Master Black Belt. but in this case it shows all low settings for printer 1 and high settings for printer 2. neutralized" says the Master Black Belt. we will be able to differentiate the contribution of each paper type in the response variable. I fear that the differences in the response variable due to "quality of paper" won't be separated from the unique effects of the printer. To test this theory. "Yes. we did the design matrix A initially. In this case. we will be able to draw meaningful conclusion from the collected data" concludes the Black Belt. the Black Belt proposes to conduct a Design of Experiment (DOE).

"The selection of the experiment design. An experiment design is the deliberate arrangement of competing settings among one or more independent variables." says the Champion. eliminating some of these barriers is within your reach. For example." William Diamond IBM – Retired Statistician Strategy of Experimentation A Champion asks the Master Black Belt to list the general steps involved in conducting a Design of Experiment (DOE). so I will learn more about this tool and be able to provide more assistance to the Black Belts". Achieve the Objective "The purpose of an experiment is to better understand the real world.e." "I'm impressed. If you don't mind.3. the X1. The objective is to reduce the PPM defective to avoid rework. X2. we select the independent variables that we want to test. "let me show you a list of the main barriers to effective experimentation. As we can see." "All of these preceding steps constitute the planning part of an experiment. During the question period. Select the Response Variable(s) 4. We then assign these levels -1 and +1 notation. Select the Experimental Design 7. …. not to understand the experimental data. a guest speaker presents the powerful tool "Design of Experiment" (DOE). At this phase. Draw Conclusions 10. and when properly planned or designed. and as result we are producing a large amount of rework. Analyze the Data 9. "you seem to be very comfortable with this subject. an Black Belt asks: "I see that there are many advantages to using this tool. "Well. Full Factorial. the mean of the process under study is not on target or there is too much variation. Ask why there are negative perceptions in your organization? You are deploying efforts but do you adequately communicate your plan and progress to your organization? Think about it". Collect the Data 8." "When do we start collecting data?" asks a Champion. XN. which may not be possible otherwise. I would like to assist you in conducting one experiment. after the Measure and Analyze phases. Select the independent factor(s) 5. Why should we conduct a DOE if the problem definition or the objectives are not clear? Some of these barriers may be attributable to a lack of training and information. or the structure of the experiment i. it will permit relatively simple interpretation of the results. He describes all the advantages and steps to conduct a valuable experiment. Choose the factor levels 6. we should define and therefore state the problem. . Barriers to Effective Experimentation At the Annual Six Sigma Symposium. Fractional Factorial. we are ready to select appropriate levels of each factor knowing what would make sense for our process. replies the guest speaker. etc. "One more step is required before collecting data" replies the Master Black Belt. but what are the difficulties or obstacles that we should be aware of before using it?" "This is a good question". Now.

I insist on the importance of carefully planning your DOE. An interaction between the windows and the air conditioner proves to be the second largest effect (not even imagined). My final message to you is DON'T GIVE UP"."As in any organization. He makes the following points as a way of illustrating the various aspects that need to be considered in any DOE. Type of Gasoline B. The actual order of importance of the various factors is contrary to his original thoughts (order in the first box above). Air Conditioner Results: Factor (in Order Of Importance) B.C. says the guest speaker. With perseverance. which is adequately communicated may prevent negative perceptions and promote management's active involvement. • • • • This one relatively simple experiment produced a wealth of knowledge. D. The Car Mileage Experiment Experiment Design: Objective: How to improve fuel economy on the highway. the type of gasoline had been more significant. management may not always be able to provide you with adequate support and coaching. Windows Level 1 (-) Regular 100 Km/hr 20psi Open Off Level 2 (+) High Octane 130 Km/hr 30psi Closed On Measure : Litres/100 Km. Windows * A. What probably really matters is minimizing cost. C. E. he presents the results to a class of Black Belts in training. understand and promote the use of these powerful tools. Factor A. A good plan. Tire Pressure D. one needs to include the extra cost for super grade to determine if it is still an advantage. But once again. A. because of a lack of time and because of other priorities.C. Off 30psi Off High Octane Closed Cruise Speed Tire Pressure Windows Air Conditioner Type of Gasoline DE. During a training session. 1_05_01_023 Best Factor Level 100 Km/hr Windows Closed/A. Improve 24 . You have to involve them in the realization of a DOE. Cruise Speed C. E. for instance. not maximizing fuel economy. If. Experiment Design Tools The Car Mileage Experiment A Master Black Belt conducts a DOE on his automobile to try to understand which factors are important contributors to fuel economy. so that they will start to learn. we can initiate change even in an environment of excessive bureaucracy.

17466 0. These must be kept constant as much as possible in order to obtain valid results. Improve 25 Level 1 Wilsons 0.79582 1.88709 0. One Factor Analysis Re sponse 0. An Engineer on the team is tasked with evaluating the pastes to determine which one is the best. is the paste. further classification may not be required as the subgroups naturally exist.63187 1.27382 1.27879 Factor Response 1.17466 M 0. In all cases the factor must be treated as a categorical variable.70682 1.79582 1.27879 .88709 Level 2 Acme 0. Continuous factors must be classified into subgroups or levels. is significant.• • Before concluding that the type of gasoline. "The dependent variable is called the response. There are generally many other factors involved. in our case the measurements will be in pounds." "Now that you have defined the factor levels and have measured each response. After careful consideration of the evidence the team hypothesized that the problem is related to the type of solder paste used during production.e. After curing he subjects each board to a tensile test to quantify the adhesion of the different pastes.70582 1. If the factor is discrete.61013 1.63187 1. of course.87233 in 1.: is measurement error causing a false conclusion). one needs to ensure that the ability to measure fuel mileage is accurate enough to differentiate such small effects (i. Following an analysis of the problem a cross-functional team concludes that the primary failure is due to low adhesion of the circuitry. At a team meeting the Engineer gave the team some general information about the single factor analysis which he was conducting. To make this evaluation he randomly selects 15 printed wiring boards and randomly assigns them to the three paste types.20943 1.87233 1. as in our case.99265 Fa ctor 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 it0. for instance.06742 0.70582 1. Our factor.00341 1. and Twecho.12497 1.20943 ab 1.27382 1. Three types of paste are available.06742 1_05_01_024 Experiment Design Tools One Factor Analysis A printed circuit board is related to high field failure rates. Wilsons. The independent variable is commonly known as a factor.70682 1.99265 1.61013 1.12497 1.00341 Level 3 Twecho 1. Acme. how will you analyze this data?" one team member asked. It must be measured on an interval or ratio scale.

12497 1. between (SSB). The SSB represented the source of variation or the variation attributable to the factor under study.06742 0. create a main effects plot.000 a St ti al ic st 1_05_01_025 Experiment Design Tools Sum of Squares as Related to One-Way ANOVA An Engineer was evaluating three brands of soldering paste used on printed circuit boards.064 75% 25% Level 2 Acme 0.126) he established the SSW (0. These values expose the statistical influence of the different levels. five for each of the different types.5064 Error 12 0." The Black Belt entered the data into Minitab. Fifteen tensile strength measurements were taken.036 0. The practical influence of different levels can also be exposed using Minitab by taking the SS and calculating a ratio as we did in Excel. completed a one-way analysis of variance (one-way ANOVA) and reviewed the results with the Engineer. analyze the sum of squares. The practical can then be displayed using charts such as a pie chart. He reviewed his findings with the Black Belt. To estimate the practical significance.70682 1. the Engineer calculated the sum of squares within (SSW).0127 0.126 1. conduct a one-way ANOVA. is derived.61013 1." Sum of Squares As Related To One-Way ANOVA Analysis of Variance Source DF SS MS Factor 2 1.01 0. Next he subtracted the SSW from SST which resulted in the SSB.00341 0. this represented the error.016 0.0278 Total 14 1.35 0. To better understand the implications of the differences you have uncovered we need to look at the data statistically. He concluded that the difference in the levels was an appreciable influence of variation. "The one-way ANOVA also calculates the SS within.05) in at least two of the means differ from the rest.99265 1.143 Level 3 Twecho 1. Based on the SS values a mean square (MS) ratio.64 + 0.88709 0.143 + 0.20943 1.3458 Level 1 Wilsons 0.63187 1. This report also generates a P value which also indicates that there is a statistically significant difference (P < 0. By adding the SS for each of the levels (0."We will look at the influence of the different levels both practically and statistically. To do this we will enter the data in Minitab.87233 1. and do a residual analysis. "Your analysis has revealed the practical influence of the factor under study.17466 0.3331 0.33 1. After reviewing the Engineer's analysis the Black Belt remarked. stack it. which is the F value. and total (SST).27879 F 18.70582 1.33). between and total." Improve 26 ct ra P i l ca 0. He calculated a ratio of the SST compared to SSB and the SSW and found that the SSB amounted to 75% of the variation and SSW was attributable to 25% of the total variation.79582 1.27382 1. By combining the response data for all of the levels he calculated SST.032 Variance Column SS Column SS Between SS Within SS Total (Source) (Error) (Total) .24 P 0.

One-Way ANOVA Table An Engineer is working on a project to maximize the tensile strength of the bond of circuitry on a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. The Engineer was unsure how to analyze the data and asked the Black Belt for help. After reviewing the data, the Black Belt suggested two possible analysis tools, the t test and the one-way ANOVA. "In Minitab we can run both of these tests. However, ANOVA has the advantage of comparing various levels, while the t test makes paired comparisons. It appears that the Twecho paste is producing a significantly higher tensile strength." "How do you know this?" the Engineer asked. "The one-way ANOVA output in Minitab produces a graphical representation of the mean confidence intervals and an F statistic and a P value. We can see that the confidence interval around the mean of the Twecho paste is completely outside the range of the confidence interval of the other two paste. The use of ANOVA has some underlying assumptions of the data; the population variances of the response are equal across all levels for a given factor, and the residuals are constant and normally distributed with a mean of zero and a constant, but finite variance." "How do I analyze the residuals?" the Engineer asked. "This can also be done in Minitab with the residual analysis tool."

One-Way Main Effects Plot

Level 1 Wilsons 0.99265 1.06742 0.87233 1.17466 0.88709

Level 2 Acme 0.79582 1.12497 1.27382 1.20943 1.00341

Level 3 Twecho 1.70682 1.63187 1.70582 1.61013 1.27879

Main Effects Plot - Means for Response











Experiment Design Tools





One-Way Main Effects Plot An Engineer is working on a project to maximize the tensile strength of the bond of the circuitry to a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. To analyze the data the engineer created a one-way main effects plot. On the chart he depicted the average response for the three different levels, that is, the three different types of paste. Having completed the analysis the Engineer presented the results to the project Black Belt. "As you can see by this main effects plot the paste at level three provides us with the maximum tensile strength of the three types; this is the type that will be specified from now on." "This looks very good and seems to support the selection that you have made. However, do you have statistical evidence to support this selection?" The Black Belt asked. "Statistical evidence?" The Engineer asked. "The main effect plot is a very good graphical tool to help substantiate your conclusions; however, it should be used to compliment other tools. With this tool alone I can't be sure that the variation presented in the plot is significant. I need to know with a certain degree of confidence if the differences between the averages could have happened as a result of chance random sampling variation or not. To do this we need to first use tools such as one-way ANOVA."

Residual Model Diagnostics
Normal Plot of Residuals
4 3 2 1 0 -1 -2 -3 -4 -5 -2 -1 0 1 2

I Chart of Residuals






-10 0 5 10 15

Normal Score

Observation Number

Histogram of Residuals
4 3 2 1 0 -5 -4 -3 -2 -1 0 1 2 3 4 4 3 2 1 0 -1 -2 -3 -4 -5 10

Residuals vs. Fits









Residual Analysis An Engineer was performing a one-way ANOVA comparing three different types of soldering paste. The

Engineer remembered from his training that one of the underlying assumptions of the ANOVA is that the residuals are random and normally distributed. However, he did not know how to verify this. He asked the project Black Belt for help. The Black Belt explained. "When you are doing your one-way ANOVA in Minitab you need to store both the data's fits and residuals. This data is then used to construct a residuals plot." Together they generated a Residual Model Diagnostics plot. The Black Belt reviewed the plot and proclaimed, "Very good, it appears that your assumptions have been meet." "How can you tell?" the Engineer asked. "The two left charts, the Normal Plot of Residuals and the Histogram of Residuals, both give an indication of the normality of the residuals; both look normal. The I Chart for Residuals gives us an indication of change in the residuals through time. This chart implies that the points are randomly distributed through time; that is, the points are evenly distributed above and below the central line, most points fall near the central line, some points approach the control limits, and no points are beyond the control limits. The last chart, Residuals vs. Fits, gives us evidence of changes in residual variation at the different levels. Each group appears to be similar in variance."

The Tw o-Level Tw o-Factor (2 D esign - 4 R epresentations
G eom etric


Low Interior Space

A High


Low B

(1) b

a ab





A lgebraic A
High Run O rder



1 2 3 4



Experim ent D esign Tools

The Two-Level Two-Factor (22) Design – 4 Representations "A two-factor, two-level design can be represented in at least four different ways" says a Master Black Belt to a Champion. In the "Geometric" representation, the corners of the square represent the levels of factors A and B, from "lowlow", to "high-high". The "Standard", or Yates designation (in the upper right image), shows the notation of all treatments taken from the treatment combination. "a" indicates "A high" and "B low", "b" indicates "B high" and "A low". "ab" indicates that both factors (A and B) are high. The notation used in the Yates designation refers to the factor set at the



-1 +1 -1 +1

-1 -1 +1 +1

Commonly the minus sign refers to lower level. and is the most common representation of full factorial experiments". Is there any difference and advantages in using one compared to the others?" k The Master Black Belt replies: "Remember. except for the case of (1). not lubricated vs. k each at two levels. I have just read a book where the author is using another designation. labeled A and Improve 31 . has widespread industrial applicability. The Champion recognizes that the "Algebraic" and "Yates Standard Notation" are directly related to the runs of the experiment. with a plus (+1) or minus (-1) sign to denote if the factors are set to their high or low settings. we always identify individual trials with –1 or 1 . illustrates the experiment design for factors A and B. The Master Black Belt adds: "the "Algebraic" notation respects Yates Standard Order. i. different notations are used. called "Algebraic" form is used to represent the settings used for each run (from 1 to 4).high level. The experiment consists of 2 trials. or absence of the factor (e. One convention is to label each factor by a letter (or numeral) and then to denote the two levels (versions) of each factor by a plus (+) and a minus (-) sign. with their respective levels (low and high). The fourth image. The "Tree". lubricated bearing).e. one trial at each combination of levels of the factors. a factorial experiment with k factors.g. Other Factorial Design Notation Matrix (22) Two-Level Two-factor Design Sign Notation Run No. the 2 factorial designs. which refers to both factors (A and B) being set at the low level. the standard condition. To identify the individual trials. Alternative Notation Japanese Notation Classical Notation A + + B + + A 0 1 0 1 B 0 0 1 1 A 1 3 1 2 B 1 1 2 2 A (1) a b ab B 1 2 3 4 1_05_01_030 Experiment Design Tools Other Factorial Design Notation Matrix (22) A Black Belt asks her Master Black Belt: "In our DOE. also called "Taxonomy" or "Characterization Chart". Thus if there are two factors at two levels.

here the trial in which all factors are at their "low" level is denoted by (1). and Type of bath set at -1 (low. The classical convention is to denote the two versions of each factor by the presence and absence of its corresponding lowercase letter. Alternative notations are to employ 0 and 1. 1 and 2 for the two version of each factor. and as +1 when it is set at "high"." Yates Standard Order (2 2) Quenching Speed = A Low: 1 m/s = High: 2 m/s = m/s = meter per second Aging Time = B Low: High: Bath = C Low: High: 4min = 8min = Water = Glycol = Factors Run Order -1 +1 Yates Standard Order Table (2 Factors Run Order Treatment combination 3) -1 +1 -1 +1 A -1 +1 -1 +1 -1 +1 -1 +1 B -1 -1 +1 +1 -1 -1 +1 +1 C -1 -1 -1 -1 +1 +1 +1 +1 A Quenching Speed B Aging Time C Bath 1_05_01_031 Settings for a two-level two-factor design Experiment Design Tools Yates Standard Order (22) A Six Sigma team conducts an experiment in the heat treatment process. or 4 min. The levels of each factor are set as follows: • • • Quenching speed (factor A): Aging time(factor B): Bath type (factor C): Low: 1 m/s Low: 4 min Low: water (-1) (-1) (-1) High: 2 m/s High: 8 min High: Glycol (+1) (+1) (+1) "When designing the experiment. Aging time (B). which is similar to the sign notation. and Type of bath (C).2 B. following the Japanese tradition earlier established by Taguchi. 1 notation. or 1 m/s). respectively. the corresponding factor level is identified as -1 when the factor is set at "low". says the Master Black Belt. Three factors are identified: Quenching speed (A). We used a two-level. or water). which makes it possible to identify. the Yates Standard order is the one most often used". For each run. factor levels and contrast coefficients used to calculate factor and interaction contrasts when performing DOE analysis. Aging time at -1 (low." "…You are familiar with the –1. 1 notation used in Minitab.)." The Master Black Belt adds: "The convention we have adopted is to use the -1. the four trials comprising 2 factorial design can be represented as shown in this table. or. The first combination shown in the Yates Standard Order will be a treatment with Quenching speed set at -1 (low. This run is Improve 32 1 2 3 4 5 6 7 8 1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s 4 min 4 min 8 min 8 min 4 min 4 min 8 min 8 min Water Water Water Water Glycol Glycol Glycol Glycol 1 2 3 4 5 6 7 8 (1) a b ab c ac bc abc . three-factor design.

0 3.0Ω ±50% Factor B: 1.0 3.5 13.& AB +) (AB .0 ? and an Amperage of 1.0 7. The Black Belt explains that since the effect of B is equal to +6.0A ±50% Y Factor Low High -1 +1 A 3.g. The experiment is designed according to Yates Standard Order for one replicate only. The team recognizes that using this Yates Standard Order can help them avoid errors in preparing an experiment. +1. Ahigh = 9. The Master Black Belt explains that the same table can be used to plan an experiment for a two-level.5 9.0 ? and Blow = 0.0 A ? 50%.0 Y1 1.0 6.& A +) (A.0 Avg ∆ Experiment Design Tools Computing Contrasts and Effects (22) A Six Sigma team conducts an experiment.0 9. the voltage (output) will vary in the same direction as the direction of change of factor B.5 3. Two factors are considered (Resistance and Amperage).0 B -1 -1 1 1 12. The Black Belt computes the contrasts for each factor (e.0 9. he calculates the effect for the interaction of the two factors (3. and the objective is to maximize output (voltage).0 3. 12 for factor A). -1.0 AB 1 -1 -1 1 6. is specified at 6. and Amperage (factor B) at 1.5 4.0.0 B 0.5 1.5 A and Bhigh = 1.g. Computing Contrasts and Effects (22) Ohm’s Law Voltage = Resistance x Amperage Specifications Uv Y = RΩ * I A = A*B Response Y: V Maximize Factor A: 6.0 ? ? 50%.5 Main Effect Plot for B Run Order 1 2 3 4 (1) a b ab Contrast Effect Avg + 1_05_01_032 A -1 1 -1 1 12. Resistance (factor A). Improve 33 A and AB effects are neutralized Factor B -1 +1 B B+ (A. All other runs are set in the same represented in the first row by the numbers: -1.5 V for a Resistance of 3.& AB +) .5A. and the corresponding effects (6. The Master Black Belt explains that it is important to randomize the Yates Standard Order prior any experiment in order to average out the noise caused by nuisance variables if they exist.& A +) (AB .5 4. and the output (voltage) is recorded for each run (e.0 3. up to the 8 run where the settings are +1.0 9. Run #3: Voltage equals 4.0 ?. the voltage (output) increases by an average of 6 volts when the amperage (factor B) increases from low to high. The team defines the factor settings at Alow = 3. +1. Since the "B effect" is positive.5 A). Similarly.5 4.0 for both factor A and B).0 6. -1.0 for the interaction AB). two-factor design.0 6.0 6.

The team recognizes the advantage of using a pie chart to graphically represent the practical significance of the effects exerted by the factors.The team concludes that to maximize voltage (output). Computing the Sums of Squares and Practical Significance (22) "Why do we calculate the contrasts"? Ask a team member. In other words. created from the Sums-of Squares in the ANOVA table. the contrasts for factors A and B. "This chart. the total change in the response variable. the contrast of A represents variation due only to factor A since B and AB are neutralized. The Black Belt highlights the fact that with this full factorial design. "Contrasts are part of the foundation of Analysis of Variance". . asks another team member. along with their interaction AB allow us to compute the Sums of Squares. B or AB). the Black Belt indicates that a Pareto Diagram can also be used instead of a Pie Chart. replies the Black Belt. The Black Belt recalls that before considering the practical significance we must evaluate the statistical significance of the different effects. SSfactor B for 38% and SSinteraction A*B for 10% of the SStotal (i. Finally. "In our experiment. In our experiment we see that contrast A is positive. and note that the remaining 10% represent the interaction effect (factors AB)." "What is the purpose of the pie chart?". SSfactor A accounts for 52%. They make it possible to compute the effects and the Sums of Squares. represents the effect of only one particular factor or interaction since the other two are neutralized by the combination of + and – signs. each contrast (A. We can also conclude that 90% of the variation of the response variable (CT Characteristic) is explained when factors A and B vary between low and high settings.e. In our two-level. to ensure independence of effects. and b) Orthogonal. two-factor design. meaning that the output is higher when we set factor A at the high level compared to when it is set at the low level. shows the practical significance of the sources of variation" replies the Black Belt. Amperage (factor B) must be kept at the high level. Finally. the Black Belt reminds the team of the characteristics of the contrasts: a) Balanced.

The change in the output i. and when it's low (-1).0 when factor A is high and factor B is also high. The team calculates the effect of factor A and then they conclude that factors A and B have the different effects.0 Average Y at Low condition of factor A Y Main Effect Plot for B 10. We perform an analytical procedure by calculating and plotting the Main Effects. and to visualize the difference in the response output between low and high settings for each factor. and the interaction AB by the combination of their – and + at each level of A.5 Analysis of Factor A Y 15.0 B:-1 A:-1 A:+1 -1 Factor B +1 Experiment Design Tools Plotting the Main Effects (22) After conducting an experiment and recording the responses. the output average is 10. the average equals only 3.5 5.0 can only be attributed to the difference between the levels of factor A. they acknowledge the advantage of plotting the main effects to depict the results of the experiments. 7. Improve 35 .5 6.0 Average Y at High condition of factor B Average Y at Low condition of factor B B:+1 -1 Factor A ∆Y 10. The Black Belt then asks "Which settings maximize output. This is the result of contrasts that are balanced and orthogonal in well-designed experiments. the team asks the Black Belt to explain the next steps. says the Black Belt. In our experiments we can see that when factor A is high (+1).5.0 B:+1 B:-1 1_05_01_034 3. "This is the correct and complete answer" says the Black Belt. Finally. and what will be the output level for such settings?" A team member answers "When factor A is set at high.5". This is because the settings neutralize both the effects due to factor B. "…however. we cannot only consider factor A.Plotting the Main Effects (2 2) EffectA = ∆y = y A + − yA − replicates taken at the treatment Effect A = 1 { contrast A } combination 2n _ _ n=Number of or Y 10.5 Main Effect Plot for A Average Y at High condition of factor A 3.e. Another team member adds "Maximum output is 15.0 2.5. the average output equals 10.0 +1 4. "You are partially correct". We must also consider factor B.

when factor A changes from its low to high level. then there is no interaction between the factors. output changes by an average of 4. and factor B is kept at its low level.e. if the lines are not parallel. Improve 36 B:-1 .5 6.5 5. "A major part of this is to determine if there is an interaction effect between factors".0 B:+1 B:-1 A:-1 A:+1 1_05_01_035 2. replies the Black Belt. they have a combine effect on the output. i. we extract relevant information" explains the Black Belt.0 B:-1 A:-1 A:+1 2. The Black Belt adds: "two factor interactions are common.0 Both lines are not parallel thus indicating an interaction effect 6." "If the lines in the interaction plot are parallel. an interaction exists between the factors. the output changes by an average of 10. "Yes". The second factor was "Invoice value".Plotting the Interaction Effect (2 _ _ n=Number of replicates taken at the treatment combination 2) Effect or AB AB = ∆y = y AB + − y AB − = 1 { contrast 2n AB Effect } Interaction Plot for AB Y 15. Conversely. When B is kept at its high level and factor A varies from its low to high level. We were surprised to discover that there was a important interaction effect between the two factors (Invoice value and Customer). For example. and two levels were considered: "Airlines" and "Manufacturers". "I remember a project where we analyzed invoices that were overdue.0).0 Experiment Design Tools Plotting the Interaction Effect (22) "During the analytical procedure of the experiment.0 units (from 5.0 units (from 2.0 B:+1 B:+1 B:-1 ∆Y 8." "Do interactions apply to transactional processes"? ask a team member.0 5.0 B:+1 Analysis of Interaction AB Y 15.0 to 15. Graphs are useful to visualize significant interactions between factors but they should not be the only means of interpreting data. "Interaction occurs when the effect of a factor is a function of the level of another factor. This is an indication of interaction between factors A and B. Note that graphs should be used to visualize the result of the ANOVA table. three factor and higher are rarely seen in practice in our type of industry. However.0).0 to 6.0 5. The first factor was "Customer".

The Black Belt states that the engineer correctly conducted a two way ANOVA using Minitab.2 44.0197 0.961. Three replicates per setting were done.71 3.961.0000 0.12 8. and that the condition can be reproduced. This means that changes in factors A and B significantly affect the output of the process.9 Percentage % 89% 5% 2% 4% 100% Practically Significant .4% of the total variation.1 85.1247 Significant *** *** A*B 2% Residual Error 4% Source A B A*B Residual Error Total Experiment Design Tools Statistical Significance and Practical Significance (22) An engineer consults with a Black Belt after conducting a two-level.52 63.5 63.Statistical Significance and Practical Significance (2 2) Output Run 1 2 3 4 Standard Order (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y1 40. "This is called Practical Significance and in this case changes in factor A represent 89.48 21.543.4 41.6 St Si atis gn ti ifi ca ca lly nt B 5% A 89% Source A B A*B Residual Error Total Degree Of Freedom 1 1 1 8 11 Sums-ofSquares 3.6 Y2 38.6 68.5 172. thus with little interest for him at this stage. and now he evaluates the statistical significance of the main effects for each factor and the effect of interaction between factors A and B.59 F-value 164.7 74.45 2.20 182.4 64.543.543.1 72.92 Mean Squares 3." The engineer recognizes that although factor B is Statistically Significant it does not account for more than 5 % in practical terms. "The F test indicates p values for factors A and B are smaller than 5%. It is a clear indication that the change is not due to chance.7 3. The Black Belt also highlights the importance of verifying the practical significance of the results using the Sums of Squares.52 63. Improve 37 1_05_01_036 Sums-ofSquares 3.20 182. two factor experiment (factors A and B). It is what we call Statistically Significant".2 182.4 42.94 p-Value 0.1 86.0 Y3 38.48 172.

we verify that residuals are normally distributed center around zero.909 0 5 10 Observation Number Histogram of Residuals 4 Frequency 3 2 1 0 -3 -2 -1 0 1 2 Residual 1_05_01_037 Residuals vs. thus we may conclude that the residuals are randomly distributed regarding the experiments run order." The Black Belt notices: "First. which is assumed constant for all levels of the factors. and to estimate them.Residual Analysis (2 2) Normal Plot shows no non-normality Shows no residuals out of control beyond the control limits Residual Model Diagnostics Normal Plot of Residuals 3 2 1 0 -1 -2 -3 -2 -1 0 1 Normal Score 2 5 4 3 2 1 0 -1 -2 -3 -4 -5 I Chart of Residuals 3." The Black Belt remembers the importance to analyze residuals prior to any conclusions about experiment result. we must verify that residuals are normally distributed. a Black Belt notices that in Minitab he can graph the residuals.0SL=3. thus we conclude that the residuals are normally distributed. The histogram of the residuals shows this normal distribution and the centering around zero. no data out of the control limits and no particular pattern. The normal plot shows dots aligned on a straight line indicating that residuals are indeed normally distributed. the Black Belt consults with a Master Black Belt.000 -3. The dots are randomly distributed. we check the random distribution of the residuals versus the settings.0SL=-3. The fourth graph shows no particular pattern and the dots are randomly distributed. To refresh the concept of residuals." "Finally." "Third. our objective is to test the appropriate hypotheses regarding the treatment of effects." "Second.909 Residual Residual X=0. Fits 3 2 Residual 1 0 -1 -2 -3 40 50 60 Fit 70 80 90 Histogram of Residuals normally distributed around 0 Experiment Design Tools Variable does not change across values of the Output Residual Analysis (22) While analyzing his first 2 x 2 factorial design. "When conducting a DOE. The models of the errors are assumed to be independent random variables that are 2 normally distributed with mean equal to zero and variance ? . Improve 38 . we consider the Individual Chart of Residuals.

017 +17.48 21.1.9*(-1) = 71. Improve 39 ˆ Y =58." "We usually assume a linear response. we use other techniques such as center points or multi-level factorial design" The Champion recognizes the advantage to represent the process behavior using a mathematical model which makes it possible to estimate process output for factor levels other than the pre-established settings. then our process will have an average yield of 58.72 63. or is it possible to know more about the process behavior?" "It is correct to say that the DOE analysis gives us a good knowledge about the factor effects and their interaction" answers the Black Belt.017 + 17.71 172.59 21.017 17. "However.59 F 86. says the Black Belt.341 1.25 12.48 172.01% .183*1 .3. We can also see that the level of Factor A (Temperature) is key in controlling the yield of the process as compared to Factor B (Pressure).800 4.367 7.183 3.94 P 0.Modeling .86 63.71 Adj MS 1862. From the mathematical model we can see that if no change is made. With a similar calculation.125 Experiment Design Tools Modeling – 22 "The results of the DOE quantify the effect that the two factors and the interaction have on the process response" says the Champion.000 0. the average yield will be Y = 58. "To confirm this linearity.81 2.71 172. there are other things to understand.341 43.125 P Analysis of Variance for yield Source Main Effects 2-Way Interactions Residual Error Pure Error Total DF 2 1 8 8 11 Seq SS 3725. "are we limited two these factor settings.48 172.92 Adj SS 3725.3.020 0." "The average response output Yield when Temperature (A) and Pressure (B) are set to their High level is computed using the model: Y = 58.9*1 = 79. in such a case this mathematical model is valid".000 0.91 1.900 2.183 * A +3.72 63.29 2.300 StDevCoef 1.341 1.2 2 Estimated Effects and Coefficients for yield Term Constant A B A*B Effect 34.183*1 + 3. we can determine that when A is High and B is Low.017 + 17.71 T 0.600 Coef 58. He then continues.9 * B 1_05_01_038 .000 0.71 3961.341 1.

" . even when only two levels of each factor are being investigated. I will need 32 runs.Multi-Factor Experiment On Lo Off In Out Hi Front Back Left Right Up Down 2 × 2 × 2 × 2 × 2 × 2 × 2 Improve 40 1_05_01_039 5 factors : 2 5 = 32 combinations 6 factors : 2 6 = 64 combinations 7 factors : 2 7 = 128 combinations Experiment Design Tools Multi-Factor Experiment A Black Belt from a manufacturing area is discussing her project with a Master Black Belt. "leverage variables". "During the team's brainstorming session. a full factorial experiment will involve a large number of tests. the angularity of the parts formed on the Fluid Cell Press. but this would be too large an experiment to do a full factorial. We would like to do an experiment with at least 5 factors. and you want to verify several factors. 64 runs and. What do you recommend I do?" The Master Black Belt replies. In fact. when you don't know which factor has a substantial effect on a response variable. So far. With 6 factors. We had identified these factors a long time ago but we don't know yet which ones have the biggest impact. it is common practice to do a fractional factorial experiment when we have 5 factors or more. 7 factors will require 128 runs and I have not included any replications yet. we have identified several factors that can influence our CTQ characteristic. So. "if there are many factors in the experiment. I have calculated that with these 5 factors at two levels. then the fractional factorial design is the preferred analytical tool.

BD.Fractional Factorial Designs Full Factorial Fractional Factorial Example with 4 Factors Main Effects A B C D 1_05_01_040 Interactions AB." "The benefits of conducting a fractional factorial experiment are that fewer tests are required compared to the full factorial design. which represents 16 runs (2 ) and is very time consuming. BD. you can obtain nearly the same information as a full factorial with a well-designed fractional factorial experiment. for most situations in industry. We want to study four factors at two levels. ABD. reducing the required time and resources. asks the Champion. we will do a half fraction of it. Statisticians have shown that for 5 factors or more. However." "I don't understand how you can do that. CD Improve 41 Experiment Design Tools Fractional Factorial Designs A Black Belt is presenting her Design of Experiment plan to a Champion: "For my project on the Fluid Cell Press. When compared to a full factorial design. ACD. The trade-off is mainly that we lose information regarding the interactions between three factors or more. but the impact is minimal. and yet we can still assess many factors at the same time. BC. it is considered reasonable to assume that third order and higher order interactions (interaction between three factors or more) are statistically negligible. which consists of 8 runs. CD ABC. a certain amount of information will be lost. AC. "A fractional factorial design is a carefully prescribed subset of all possible combinations of a full factorial design. How will you be able to test all the possible factor combinations if you do not use a full factorial experiment?". I will use a fractional factorial design. BCD ABCD Main Effects A B C D Interactions AB. AC." . AD. AD. Instead of doing a full factorial 4 design. BC.

it is considered reasonable to assume third order interactions to be statistically negligible." "The design becomes these 4 runs and we only need the three columns for A. we must sort on the ABC column. The same thing happens for B with AC. First. Therefore. B. and C. "OK. Second. For three factors with this design. In our case. but for my own understanding I am interested in knowing how to construct one. so we will use the ABC interaction to create the fraction. the ABC effect is lost since it was used as the generator to form the half fraction of the full factorial matrix. we have to sort the entire matrix by the highest order interaction. Look what happens to the matrix. and C with AB. if we choose the generator ABC=1." "The sign of the ABC effect constitutes a "block". 5 and 8. Could you explain this to me?". 3. The other columns constitute a repetition of these three.Forming a Half-Fraction of a Full Three Factor Design Run A 1 2 3 4 5 6 7 8 -1 1 -1 1 -1 1 -1 1 B -1 -1 1 1 -1 -1 1 1 C -1 -1 -1 -1 1 1 1 1 AB 1 -1 -1 1 1 -1 -1 1 AC 1 -1 1 -1 -1 1 -1 1 BC ABC 1 1 -1 -1 -1 -1 1 1 -1 1 1 -1 1 -1 -1 1 Generator: ABC=1 Same Contrast Run A Confounding : A & BC or alias B & AC structure C & AB B -1 1 -1 1 C -1 -1 1 1 AB -1 -1 1 1 AC -1 1 -1 1 BC ABC 1 -1 -1 1 1 1 1 1 Experiment Design Tools Forming a Half Fraction of a Full Three Factor Design "I know that Minitab creates fractional factorial designs. First. we will keep only the runs 2. the design generator can be ABC=-1 or ABC=1. we will take a simple example with a three factor design. the main effects are confounded with the two factor interactions. For most situations in our industry. Note that I could have chosen ABC=-1 as the design generator and arrived at the same conclusions. The fractional design is formed by selecting all those treatment combinations where the ABC effect is either "1" or "-1". the unique effect of "A" can not be separated form the unique effect of the BC interaction because both columns have the same sign at each experimental run. This is called confounding or alias structure." Improve 42 1_05_01_041 2 3 5 8 1 -1 -1 1 . asks a Black Belt to a Six Sigma Instructor. In this instance.

"Well." "That design doesn't seem to be very useful". "Yes. I have noticed that for a specific number of factors that we want to study. the important factors that have been found with the resolution III design and their interactions effects. sure! Depending on the number of factors and runs. the quantity of information is limited. if I use the function "Create Factorial Design" (under Stat > DOE). continues the Master Black Belt. (Optimization Designs) V Experiment Design Tools Design Resolution "In Minitab. different types of designs are available. "We consider them as Characterization Designs and they are generally used to study relatively uncomplicated effects and interactions from a smaller number of factors compared to resolution III designs. "However. these designs are called Screening Designs and a small number of runs are required to use them". Another experiment is then needed to study. Resolution v and higher designs are called Optimization Designs and are most often used to study more complicated effects and relationships involving two or three factors. …. Such designs are also employed to obtain a mathematical model describing the relationship between the CT characteristic and the factors (Y = f(X1. says the Black Belt. I am not sure that I understand the terminology associated to the designs. the resolution defines the alias or confounding structure of a design. could you explain this to me?". XN)). so these designs are generally used to initially screen a large number of factors. asks a Black Belt to a Master Black Belt. main effects are not confounded with interactions but interactions are confounded together". in more detail. For resolution V and higher: to study more complicated effects involving 2 or 3 factors and obtain amathema tical model. replies the Master Black Belt. For example. a resolution III design means that the main effects are confounded with interactions between factors." "With resolution IV and V designs. as their name implies. Improve 43 1_05_01_042 .Design Resolution Resolution III IV Main Confounding Pattern Use Main effects are confounded To screen many factors with second order interactions (Screening Designs) Second order interactions are To study uncomplicated confounded with other second effects and interactions order effects interactions (Characterization Designs) Second order interactions are confounded with third order interactions Same as resolution IV.

there are four types of factorial designs that are available: a resolution III design which consist of 8 runs. we have identified two levels (high and low) that can be used in an experiment. Interactions between factors are possible but we don't know if they exist." . but we don't really know much about them. "k" represents the number of factors and "p" what fraction of a full factorial the design is." "In this situation. In general we use 2 where "2" indicates that the factors are studied at two levels. The design you select will depend on what you need to know. a resolution VI with 32 runs and a full factorial with 64 runs. "This refers to the notation that is used to describe k-p factorial design. you don't have a lot of knowledge about the factors and you are definitely not at an optimization stage. your level of knowledge of the process and resource availability. From the Analysis Phase certain factors appeared to have an influence on our CTQ. p=2 means that your design is ¼ fraction of a full factorial for 6 factors at two levels. "For six factors. Then. six factors have been selected as potential Critical to Process characteristics." 6-2 "Why is this design called a 2 ?" asks the Black Belt. p=1 it means a ½ fraction of the full factorial. a resolution IV design with 16 runs. I suggest you use the resolution IV design with 16 runs. p=2 means a ¼ fraction and so on. Don't forget that you will have to include a few replications which will significantly increase the amount of runs. What design do you suggest I use?" asks a Black Belt to a Master Black Belt. it is my first experiment on this machine. replies the Master Black Belt. In your case.Choosing a Factorial Design Number of Runs* 2 4 8 16 32 64 128 full 3 III full IV full III V full III IV VI full III IV IV VII full IV IV V VIII Number of factors 4 5 6 7 8 2 6 −2 IV Improve 44 1_05_01_043 *Note: the number of runs is for 1 replication. Experiment Design Tools Choosing a Factorial Design "For my project on the automatic riveting machine. For each of them." The Black Belt replies. "well.

Objective. The result of the Gage R&R study detected a problem with the measuring system. and a Gage R&R study has been conducted with the team members who will measure the parts. "Yes. and Environment. one for each type of part." "The Black Belt continues. "Sure". we have identified more than 60 causes that can affect the angularity of the parts. we have organized a location where all the parts will be measured. Response Variables And Factors A Black Belt presents her Design of Experiment plan to a group of Champions. Methods. but it has been corrected. "The objective of the Design of Experiment is to identify the factors that affect the quality of the parts formed in the Fluid Cell Press. Can you please clarify the problem with the CTQ?". "with the information gathered since the beginning of the project and a recent brainstorming session. Materials. Tools." "Do you have a good measuring system to obtain accurate data for your DOE?". We used a fishbone diagram to summarize the causes in the following categories: Machine. Two separate statistical analyses will be done. An operation of check and straighten has to be done practically on every part to bring the CTQ on target. This CTQ will be measured on two typical parts that we produce: "curved" and "straight" parts." "We regularly rework the parts produced in this process. Response Variables and Factors CTQ target: 90 O CTQ: Angularity Machine Tools Materials Angularity of the parts People Experiment Design Tools Methods Environment Objective. replies the Black Belt. asks the Champion from the Quality department. asks a Champion. The CTQ characteristic that we want to improve is the angularity of the parts. People." Improve 45 1_05_01_044 . "The parts produced have an angle that do not meet the target of 90 degrees.

"Well. says the Methods' Champion. but not so much that they extend beyond the operational range of the factor to have a better chance of detecting the effects that are present. we have identified seven factors that can influence the CTQ characteristic (Angularity of the parts).Select Factors and Levels Factors A: Loading of tray B: Height of blocks C: Grain direction D: Covering type E: Pressure F: Forming Speed G: Spring-back Levels Low (-1) High (1) 20% 50% 1/4” 1 1/4” 45º 90º Soft Hard 8. How did you select these seven factors? I am sure that there are more than seven factors". Response Experimental Effect Lo (-1) Hi Factor Settings (+1) Experiment Design Tools Select Factors And Levels During a presentation of a DOE to a group of Champions. However most industrial experiments are done with 8 factors or less. asks another Champion.000 PSI Normal Slow 2º 4º Y True Effect In general. replies the Black Belt." "How did you select the levels?". we used a Nominal Group Technique where each team member selected and rank the factors of his choice by order of importance. the levels should be set "wide". and a recent brainstorming session. we separated the causes into three different categories: controllable. asks a Champion. a Black Belt explains the selection of factors and levels. the levels should be set wide but not so much that they extend beyond the operational range of the factor. noise and held constant." Improve 46 1_05_01_045 . From the group of the controllable factors. "With the information we gathered since the beginning of the project on the Fluid Cell Press. more than 60 causes were identified but we classified these causes to narrow down the number of factors to study in the experiment"." "Is it a reasonable number of factors for an experiment?". the 7 factors were identified. "Initially. replies the Black Belt. In general.000 PSI 15. Other levels come from the operators' experience and by pushing the factors to reasonable extremes. "Some factors such as forming speed only have two options"." "This process is so complex. Finally. First. The controllable factors are studied in the experiment because we can control and change them during an experiment. replies the Black Belt. generally the number of factors studied in an experiment is less than 15.

F. it is considered valid to assume that third order and higher interactions are negligible. we have to consider that the label A*B represents in fact A*B+D*G+E*F. D*G or E*F that is important. the same contrast is used for these three interactions. A*G and B*C. so a total of 32 runs were done. 2 with 2 replications. we use the first interaction as a label because when we analyze the data. and second order interactions are confounded with other second order interactions. However. For example. we verify the alias or confounding structure using the DOE function of Minitab. To perform a statistical analysis such as ANOVA. "Sure. A*E. but I am not sure which function to use in Minitab to analyze them. A*B." "This function confirms that we have a resolution IV design. G. what type of experiment did you conduct?". we cannot separate the effect. E. The main effects are confounded with third order interactions. "A fractional factorial experiment with seven factors at two levels each. D. we know that no main effects are confounded with any other main effect or second order interaction. because in the design matrix. A*C. asks the Master Black Belt. B. A*D. C. if the A*B effect is significant. asks a Black Belt.Alias Structure Alias Structure for a 2 A B C D E F G A*B A*C A*D A*E A*F A*G B*C 1_05_01_046 7-3 iv design + B*D*G + B*E*F + C*D*F + C*E*G + A*D*G + A*E*F + C*D*E + C*F*G + A*D*F + A*E*G + B*D*E + B*F*G + A*B*G + A*C*F + B*C*E + E*F*G + A*B*F + A*C*G + B*C*D + D*F*G + A*B*E + A*C*D + B*C*G + D*E*G + A*B*D + A*C*E + B*C*F + D*E*F + D*G + E*F + D*F + E*G + B*G + C*F + B*F + C*G + B*E + C*D + B*D + C*E + D*E + F*G Improve 47 A*B*C + A*D*E + A*F*G + B*D*F + B*E*G + C*D*G + C*E*F Note: The terms presented here are only up to the third order interactions Experiment Design Tools Alias Structure "I have the results of my DOE on the Fluid Cell Press. First. I used a 7-3 resolution IV design." "Note that since the interactions are confounded with other interactions. we can input in Minitab the following terms for the factors of the model: A." "Since it is a resolution IV. it can be either A*B. For most situations in industry. But second order interactions are confounded with other second order interactions. Can you please help me?"." . A*F.

85 2457. "Yes".000 0.038 Adj MS 9.889 375.06 28.722 10.230 1.133 3.389 583." Improve 48 1_05_01_047 . "In order to obtain the sums-of-squares of each term in a model. the function "General Linear Model" can be used".000 0.30 45.27 564.84 1582.445 3." 7-3 "My design is a fractional factorial 2 of resolution IV.679 134.170 1.60 5.897 5.897 5.389 583.001 0.889 375.836 2.147 3.753 6.ANOVA Table Analysis of Variance for Angularity Source A B C D E F G A*B A*C A*D A*E A*F A*G B*C Error Total DF 1 1 1 1 1 1 1 1 1 1 1 1 1 1 17 31 Seq SS 9.14 5.836 2.170 1. asks the Black Belt. If we look at the Alias structure associated with your design. G and A*B have the largest sums-of-squares compared to the total. The next step is to graph the residuals to verify the assumptions associated with ANOVA. asks the Black Belt. They are also used to evaluate the practical significance of the effects. we see that the A*B interaction is confounded with D*G and E*F. "The sums-of-squares are the basic measure of variation associated with each term of the model.070 13.170 1. and to detect the presence of outliers that can contaminate the results.001 0.753 6.188 4.722 10.753 6.897 5.188 0.039 Experiment Design Tools ANOVA Table A Master Black Belt explains to a Black Belt how to analyze the results of a DOE in Minitab.722 10.230 1.230 1. replies the Master Black Belt.000 0.26 9. "The ANOVA table shows that all the effects are statistically significant at an alpha level of 0.002 0.000 0.836 2.147 3.50 16." "Don't forget that A*B is only a label here.000 0. Since D and G are both important as main effects.586 Adj SS 9.40 21.238 F 41. we select the main effect and the two factors interaction".679 134.05. "Generally.679 134.43 55.00 P 0. It seems that D.133 3.29 14.070 13.038 1158.445 3. replies the Master Black Belt. for a resolution IV design from the alias structure. Although they are all statistically significant.000 0.008 0. How do I know which model to use in Minitab?.000 0.027 0.000 0.147 3. it is more likely that the D*G interaction is present here.889 375. "Is it really important to calculate all the sums-of-squares?".133 3.445 3.070 13.188 4. they are not all practically significant as demonstrated when we consider the magnitude of each sum-of-squares compared to the total sum-of-squares.67 13.389 583.

and since both D and G are important main effects.0 Experiment Design Tools Practical Significance "How can I identify the effects that are really important? The ANOVA table shows that almost all the effects are statistically significant".4 82. asks a Black Belt. "From the alias structure associated with the design." "The A*B effect represents 11.6 % of the total variation.5 100.Practical Significance Pareto Chart for Source 1200 1000 800 100 80 60 40 20 0 600 400 200 0 Source Count Percent Cum % G 583.4 %.133 1.1 95.8 A*B 134.4 50. To estimate the practical significance." "From the Pareto chart we see that the main effect G represents 50.679 50. It seems that G and D are the vital few variables since they account for more than 80 % of the variation in the response variable. asks the Black Belt.4% of the total variation of the response variable (CT characteristic). replies the Master Black Belt.4 D 375.5 ers Oth Improve 49 1_05_01_048 51. What does it mean taking into account the fact that the 7-3 design is a fractional factorial 2 of resolution IV?".6 94.070 11. D*G and E*F are confounded with A*B.868 4. "This situation happens especially when the experimental error is small".836 32." Percent Count . and that D represents 32.4 A*C 13. In this case. we use the sums-of-squares from an ANOVA table to create a Pareto or a Pie chart. we know that other interactions are confounded with the A*B interaction. there is a good chance that the interaction D*G is exerting the most influence here.

" "We can clearly see in our plot that G. A lpha = ." Improve 50 1_05_01_049 -40 -30 -20 -10 0 10 20 30 40 50 . then these contrasts are plotted on the lower left or upper right of the straight line formed by the other contrasts.Normal Probability Plot of the Effects Normal Probability Plot of the Standardized Effects (response is A ngulari. If all of the contrasts represent nothing but random variation (i. if there are no real differences between the averages at -1 compared to 1). and A*B is a significant interaction effect. and since both D and G are important main effects there is a good chance that the interaction D*G is exerting the most influence here. In other words. says a Master Black Belt to a group of Black Belts. "Well. asks a Black Belt. G and D are the vital few variables. "How do I interpret this plot?". we know that D*G and E*F are confounded with A*B. asks a Black Belt. if some contrasts represent real differences rather than random variation.e. D and A*B represent real differences. the "outliers" represent significant effects. Therefore. replies the Master Black Belt. "The main difference is that instead of using the sums-of-squares. From the alias structure associated with our design.10) G A : B : C : D : E : F: G : A B C D E F G 1 0 -1 D A C C A E A D E F AC B B C A G A F B A A B N al S orm core Standardized Effect Experiment Design Tools Normal Probability Plot of the Effects "A graph used to analyze the results of a DOE is the Normal Probability Plot of the effects. "What is the difference between this graph and a Pareto chart of the sums-ofsquares?". However. then the contrast effects should form a reasonably straight line on the Normal Probability Plot. the Normal Probability Plot represents the estimated contrast effects but both graphs are used to identify the important effects".

replies the Black Belt. " "Where does this data come from?". "What is the CT characteristic and the type of design you used?". "The CT 7-3 characteristic is the angularity of the parts. and the design is a fractional factorial 2 of resolution IV with two replications. asks the Master Black Belt. . "If there is an interaction between these two factors. Is there a tool that creates a visual representation of the results?". asks the Master Black Belt. asks a Black Belt. It helps us to visualize and confirm the important main effects. "The data is the result of a fractional factorial design related to our project". it is preferable to do an interaction plot to find the best setting for both factors". "The Main Effects Plot is a very useful tool". "Those are the exact same results I found using the sums-of-squares in the ANOVA table". replies the Black Belt. replies the Master Black Belt. says the Black Belt.Main Effects Plot M Effects for Angularity ain -1 1 4 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 Angularity 1 2 1 0 8 6 A 1_05_01_050 B C D E F G Improve 51 FACTORS Experiment Design Tools Main Effects Plot "I find that the ANOVA table is not user friendly because it displays many numbers arranged in a tabular form. we should set factor D at its high level (1) and factor G at its low level (-1)". Since our objective is to minimize the response variable (CT characteristic). "The mean of the response variable is plotted for the levels of each factor." "The Main Effects Plot shows a large difference in the mean of the response variable between the low (-1) and high (1) levels of factors D and G". says the Master Black Belt. says the Master Black Belt.

When combined with factor G. replies the Black Belt. factor D has a large impact on the mean of the response variable. the difference in the response variable between "G low" and "G high" is smaller than when D is set at the low level (-1). asks the Master Black Belt. replies the Master Black Belt. The Interaction Plot helps us to visualize and confirm this interaction". and factor G at its low (-1) level. "We then can see that when factor G is set at its low level (-1). Do you need to maximize or minimize the response variable?". "We use it when the results of an ANOVA table make us suspect the presence of an interaction. asks a Black Belt. I create an Interaction Plot." "Considering that we want to minimize the response variable." Improve 52 ." "To interpret this plot we must consider our objective. "I need to minimize it". we confirm that there is an interaction between factors D and G. Since the lines are not parallel. we conclude from this graph that factor D must be set at its high (1) level. the response variable is smaller compared to when G is set at its high level (1). We can also see that when factor D is set at its high level (1).Interaction Plot Interaction Plot for Angularity D -1 1 -1 1 D Low (-1) 20 15 M ean 10 D High (1) 5 1_05_01_051 -1 1 G Experiment Design Tools Interaction Plot "When do we use the Interaction Plot in a DOE?". "Since I believe that there can be an interaction between factors D and G in our DOE.

and second. we use "Blocking Variables". we must not forget that generally a Blocking variable is confounded with higher order interactions". since it makes it difficult to identify factor effects. shifts. However. suppliers." "Blocking variables allow us to remove the noise from the error term by creating a separate term. blocking provides two benefits: First. The reason for grouping units or runs into blocks is to reduce the amount of experimental error or noise.. "When using Blocking Variables. when creating a design matrix. and the run within each block is performed in a random order. "to reduce the effect of noise variables such as batches of material. Thus. etc. the results will be "contaminated". every batch of material is different from the others". we use a "randomized block design" where the homogeneous units are formed into blocks. "To remove the effect of the added noise. the experimental runs are sub-divided into groups in such a way that the runs within a group are relatively homogeneous. the experimenter can test the significance of the blocking variable.Blocking Variable Non-homogeneous units Formed into blocks Batch 1 Batch 2 StdOrder 7 6 8 5 3 4 1 2 RunOrder 1 2 3 4 5 6 7 8 Block 1 1 1 1 2 2 2 2 A -1 -1 1 1 1 -1 -1 1 B -1 1 1 -1 -1 1 -1 1 C 1 -1 1 -1 1 1 -1 -1 1_05_01_052 Experiment Design Tools Blocking Variable A Black Belt planning a DOE related to a process in the composites material department says: "I am afraid that by using different batches of material during our DOE. A Master Black Belt replies. After all. operators. it reduces the experimental error which makes it easier to detect effects." Improve 53 .

003 0.004 0. C and D.56 B*D 1 0.71 36. thus making it more difficult to identify the factor effects.06 390.00 43. Statistical significance means that we took an accurate decision in using such blocking variable.06 0. main effects plots.006 0. for an alpha level of 0. However.56 855.06 A*C 1 1314.06 Error 4 120.22 1.30 12.94 Experiment Design Tools A Norm Score al 1 0 -1 F 0. The same conclusion is obtained when we use the Normal Probability Plot of effects. "When conducting a DOE on a chemical process." Improve 54 1_05_01_053 .56 0.75 0.56 C*D 1 5.56 B*C 1 22.966 0. and analyze it with the other factors contained in a detailed ANOVA table".56 1870.06 A*D 1 1105.642 0. we can continue the analysis without considering the blocking variable. then we can proceed with the analysis. knowing that the results are not contaminated.898 0. we continue our analysis with the prescribed steps (Residual analysis. After completing these steps." "From the ANOVA table we see that. the variable "days" was the blocking variable.17 P 0. and that factors A. I don't know how to analyze the data.56 1105.98 28." The Master Black Belt replies. If the block effect is significant.Analyze an Experiment With a Blocking Variable Y: Filtration Rate Blocking Variable: Days A: Temperature B: Pressure C: Fomaldehyde Concentration D: Stirring Rate Analysis of Variance for Filtrate Source DF SS MS Block 1 7. I followed your recommendation and used a blocking variable.56 22.56 B 1 39. as well as the interaction effects A*C and A*D are statistically significant.56 A*B 1 0.25 30.56 A 1 1870.06 1314.56 7. Since it took two days to complete the experiment.06 C 1 390.25 62. Two days can have a large effect on any chemical process and I don't want to increase the amount of experimental error.435 0. The practical significance of these factors and interactions is confirmed when we compare their sums-of-squares with the total sums-of-squares.06 D 1 855.06 Total 15 5730.703 C AD D AC -5 0 5 Standardized Effect Analyze an experiment with a Blocking Variable During a coaching session.46 0. and interaction plots). "Well.78 0. the block effect is not statistically significant.023 0.06 39. and if necessary. you just have to put the blocking as a term in the model used.318 0. "If the block effect is not statistically significant.06 5.02 0. a Six Sigma Black Belt says.05.001 0. redo the model without considering the blocking variable.

From previous screening studies. the team is ready to analyze the data using Minitab's statistical functions. the team assigned to this project identifies two variables which influence the process: • • Curing Pressure (factor A). The team prepares to conduct a two-factor three-level DOE with two replicates. 2 for an intermediate setting and 3 for a high setting.Two-Factor Multi-Level Full Factorial CTQ Y: Composite part (Measurement Scale: 0 . . So what calculations must be made ? Experiment Design Tools Multi-Level Two-Factor Design The Champion responsible for the manufacturing of composite parts believes that the autoclave settings are not optimal. Improve 55 A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 . where 100 is a perfect composite part. Curing Temperature (factor B). To code each setting for factors A and B. After conducting the experiment and recording the responses. she chooses 1 as the label to denote a low setting. . the Black Belt creates this data table in Minitab showing the 18 runs. Current settings create problems related to shape and other CTQ characteristics of a particular group of parts.100 where 100 is perfect part) Minitab Entry Shown in Standard Order Factor A: Curing Pressure Factor B: Curing Temperature Data Collection Sheet A 1 1 1 2 2 2 3 3 1_05_01_054 B 1 2 3 1 2 3 1 2 3 Y1 40 50 60 50 70 50 30 20 70 Y2 20 30 50 10 60 60 40 10 80 3 . The Six-Sigma team adopts a measurement scale for the output (Y) that ranges from 0 to 100. To collect the data in an organized way.

Statistically Significant? Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 The Anova Table Experiment Design Tools Statistically Significant A Six-Sigma team conducts a DOE related to the curing process of composite parts. After recording the responses. for an ? risk of 5%. Two factors are considered: Pressure (factor A).6% probability that the change in the output is due to chance." Team members recognize that even if the main effect of factor A is not statistically significant. "From this table we see that factor A (Pressure) is not statistically significant. and thus is repeatable. factor B (Temperature) has a great influence on the output. meaning that there is a 46. "In this case we choose "Balanced ANOVA". the Black Belt generates the ANOVA table in Minitab to determine if there is a statistical difference between the output at different levels of the factors (Pressure and Temperature). which one do we choose?". replies the Black Belt. asks a team member.05. Improve 56 1_05_01_055 .008. as shown by a p value smaller than 0. it is highly improbable that the change in output is due to chance alone. this allows us to see if the interaction between factors A and B has an impact on the output". The p value is 0. The Interaction between factors A and B (A*B) has a significant effect on the CTQ. it affects the output through its interaction with factor B. On the other hand.466. and Temperature (factor B). "There are various ANOVA functions in the Minitab menu. We now can say that the change in the output is not random in nature. The team conducted two replicates for this two-factor threelevel experiment. With a p value of 0. The Black Belt explains to the team the interpretation of the results shown in the ANOVA table.

8 1394. We conducted an experiment with two factors: A) Type of coolant and B) Type of end-mill.2 100.6 .000 *** Pie Chart of Source Practical Significance Source A B A*B Error Total 1_05_01_056 Sums-of-Squares B (5755. In order to take advantage of this interaction.8 .11. we must first take advantage of the effect caused by factor B.9 % 4.6 . we must work on both factors A and B.2%) ." says the Champion.6 5755. for an ? risk of 0.9 MS 277. both factors.Practically Significant ? Multi-Level Full Factorial Analytical Significance Analysis of Variance for Y Source A B A*B Error Total DF 2 2 4 27 35 SS 555.8 1500. but not practically significant. Can you clarify this apparent contradiction.6 5577. "You just told me that factor A (Type of coolant) is statistically significant.0 41. In fact. thus "type of coolant" is statistically significant." The Black Belt replies "There are situations where certain factors are statistically significant.80 25. A and B must be considered.0 13388.00 51.0 %). "We must not decide based only on main effects. "No" says the Black Belt. the p value for factor A is 0.41. we see that. In the ANOVA table.0 A (55.6 5755.05. because this factor accounts for the largest percentage of the total Sum-of-Squares (43.43%) A*B (5577.6 5577.8 1500.6 F 5.0 13388.000 *** 0. the interaction A*B is statistically significant and also practically significant. but now you say that this factor is not significant." "Does this means that factor A is not to be considered?" asks the Champion.1% of the Total Sum-of-Squares (practical significance). However. from the analysis of the Sum-of-Squares.7%) Experiment Design Tools Practically Significant "There is something I do not understand. we see that this factor accounts for only 4. Improve 57 DF 2 2 4 27 35 SS 555. As we can see.8 2877.4.014 *** 0.4 55.014.1 43. Each factor was set at three levels.7 11.1%) Error (1500 .10 P Significant 0." The Champion recognizes that in order to maximize the process output Y.

0SL=-27. We must analyze the residuals. "There is a very important statistical analysis left to do. Does this mean that we have finished the DOE data analysis?" asks a team member.38 X=0." Using Minitab. the Black Belt performs the residual analysis and prints out the corresponding four graphs." • • • • The first shows that the residuals are normally distributed. Finally. the fourth graph reveals if the residuals are randomly distributed around the fitted values. Fits 0 -20 -15 -10 -5 0 5 10 15 20 Residual 10 20 30 40 Fit 50 60 70 80 Experiment Design Tools Residual Analysis "We have conducted the DOE. as it should be if nothing special is affecting the residuals. "Not yet" replies the Black Belt.Residual Analysis Multi-Level Full Factorial Residual Analysis for Composite Part 1 20 10 Residual 0 -10 -20 -2 -1 0 1 Normal Score 2 Residual Normal Plot of Residuals 3 30 20 10 0 -10 -20 -30 0 I Chart of Residuals 3. We can see that the residuals are randomly distributed and that no point lies outside of the control limits.000 -3. The second is the histogram. The team members conclude the residuals are normally distributed around zero and that they are not affected by special causes.0SL=27.38 2 6 Frequency 5 4 3 2 1 1_05_01_057 10 Observ ation Number 20 Histogram of Residuals 20 10 Residual 0 -10 -20 4 Residuals vs. interpreted the ANOVA table and identified the factors that have statistical and practical significance. The third is the chart of the residuals function of sequence run order. It indicates that the residuals are well grouped around zero. Improve 58 . The residuals on this graph look alright. which is what we expect to see in this graph.

" "Does this mean that in order to obtain the highest average output we should specify the autoclave's temperature to the value corresponding to the level 3. We can see that average output is the highest when factor B is set at level 3. Factor A (pressure) has relatively little effect since the average output remains almost constant at each of its three levels. "The graph confirms the results of the ANOVA table. The team studies the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine. 2 Each factor was set at three levels (1. "We should study the interaction plot to find the optimal setting for both factors. Experiment Design Tools Main Effect Plot An across functional team conducts a DOE during the design of an improved composite part." "Factor B ( temperature) however. The Black Belt plots the Main effects. and 2 replicates are conducted for each setting of this full factorial design." replies the Black Belt.Main Effect Plot Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 Main Effect plot . "It is too early to make that statement. and explains the interpretation to the team. has a great effect on the output. The factor Temperature (B) has a great effect on the output.Means for Composite Part 63 55 47 39 31 A B Y 1_05_01_058 The factor pressure (A) has relatively little effect on the response Y. asks a design engineer. and helps us to interpret the effects of the factors on the output. 2 and 3) for a 3 experiment." Improve 59 . and not worry about the pressure?".

That is. of the maximum or the minimum of the curves relative to factor A settings.e. each one corresponds to a particular level of factor A. After discussing the main effect plot. 2 experiment). the Black Belt creates and interprets the "Interaction Plot". The setting 2 for B is interesting since it gives the largest output difference between the three settings of factor A. the output or response is almost linear for the different levels of factor B. 2 Each factor was set at three levels (1. Improve 60 1_05_01_059 The largest output average correspond to factor A at level 3 and factor B at level 3 . and take advantage. "When factor A is set at level 1. The lowest output average is obtained when factor A (pressure) is set at level 3. The highest average output is obtained when both factors A and B (pressure and temperature) are at their respective level 3." "Another interesting point to note is that the response average is independent of factor A when factor B is Low. Notice 2 that if we had conducted only a two-level design (i.Means for Composite Part A 1 2 3 1 2 3 Curvature on the CTQ output average of Composite Part for B (Temperature) average when A (Pressure) is at level 3 75 65 A2 A1 A3 M ean 55 45 35 25 15 1 2 3 B The smallest output average correspond to factor A at level 3 and factor B at level 2 Experiment Design Tools Interaction Plot An across functional team conducts a DOE during the design of an improved composite part. we would have missed the non linearity of the response average corresponding to level 2 setting of factor B. we would not be able to discover. The team studies the effect that factors A (pressure) and B (temperature) have on the curing process in the autoclave machine. they should consider the Main Effect and Interaction plots in order to identify the optimal settings." The design engineers recognize that before specifying any setting. There is no output difference for A at level 1 and 2 when B is set at level 3 but a difference exists for factor A at level 3 for this setting of B. and 2 replicates are conducted for each setting of this full factorial design.Interaction Plot Multi-Level Full Factorial A 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 B 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Y 40 50 60 20 30 50 50 70 50 10 60 60 30 20 70 40 10 80 Interaction Plot . "We see three curves. 2 and 3) for A 3 experiment. and factor B (temperature) is set at level 2".

and to establish the objective for the experiment. 2 needing only 6 experiments (i. since its p value is less than 5%. but the center point gives an indication of whether it is appropriate or not to interpolate between the factorial points. studying a few variables in depth. k By adding center points to a 2 factorial design.Center points Sometimes it is desirable to study quantitative factors at more than two levels. Estimating the error. the ANOVA table shows that it exists a significant effect due to the curvature at the center point. Also. 4 runs for the 2 with one replicate plus 2 runs at center point) compared to a 2 3 full factorial design which needs 9 runs. we may address these concerns without a significant increase of the number of experiments. "Can you outline the steps I should take to ensure that it is successful?" "We follow a 10 step process to ensure that we cover all the main topics of a good experiment. . preferably with a statement of planned action. Steps 1 and 2 are to define the problem. Strategy of Good Experimentation Step 1 – Define the Problem Step 2 – Establish the Objective Step 3 – Select the Response Variable(s) Step 4 – Select the Independent Variable(s) Step 5 – Choose the Variable Levels Step 6 – Select the Experiment Design Step 7 – Collect the Data Step 8 – Analyze the Data Step 9 – Draw Conclusions Step 10 – Achieve the Objective Strategy of Good Experimentation "I want to conduct an experiment on my process". "We perform "n" replicates run at the mid-level for each of the factors in the factorial design. to find which has the most important influence. The curvature cannot be assigned to a particular factor. the error is calculated and the curvature is detected." The Champion recognizes that. is a key part of the improvement cycle. or confirming the results of past studies under new conditions. with only 2 center points. Examples of situations that would lead to three or more levels include: • • • Checking for quadratic effects (curvature). A well-constructed problem statement.e. combined with a good objective statement for the planned experiment. says an Black Belt to a Master Black Belt." "Note that the magnitude of the curvature effect can be directly compared to the factor effects for relative importance if the number of center points is equal to one-half the number of factorial points. "How do you add center point to your design?" asks the Champion. the number of center points does not affect the other effects. The objective statement should clarify whether the experiment involves a screening experiment. The objective of the cycle. In this case. which is used to study a large number of variables. Including the current level of the factors as well as the high and the low levels. should be stated in such a way that it provides guidance to those designing the experiment.

or identify the factors that have the greatest effect on the variation of the parts. replies the Master Black Belt . and appropriate factors for the study"."Where possible. a statement of the results required should be considered in the objective when appropriate. the objective for an experiment might be to identify factors that call be used to improve yield by 5%. before other work on the experiment is begun. Having representatives from other disciplines working on the objective of the experiment. after completing the measure and analyze phases." Another consideration in the statement of the objective. can often result in a study that is more useful to a range of interested parties. The team selection process is crucial. As an example. For example. because the team determines all aspects of the experiment". all interested parties should contribute to the objective. "Well. The makeup of the team should depend upon the nature of the problem. is that it should be helpful in identifying the response variables. Team Selection for DOE’s Key tasks for the Experimental Team: • Write Problem Statement • Define the Objective of the DOE Study • Record relevant background details of the process • Define Response Variables • Define Controllable Factors • Define Held Constant Factors • Define Uncontrollable Factors Improve Experiment Design Tools 63 1_05_01_062 Team selection for DOE's "Who is involved in completing a problem statement and objective statement for an experiment"? asks the Black Belt. replies the Master Black Belt. an experiment to be designed and conducted in manufacturing may be of interest to those in research or engineering.

2. diameter. The response should capture as much information from the process as possible.000 . such as percent of concentration by weight or by volume. temperature. replies the Master Black Belt.002 Experiment Design Tools Defining the CT Characteristics A Black Belt asks a Master Black Belt how he goes about choosing the CT Characteristics to be studied by an experiment. we always try to bridge the gap between our current knowledge and experience on the process under study. In using a designed experiment. we should try to have a response that meets a target condition. 3. but there are a few things we should consider in their selection: 1. but what makes a good response variable?" asks the Black Belt. Each experimental condition will produce data that can be used to find conditions that will optimize the process output on that desired target. Specifications N inal target om + 0. As you said. or averaged over the surface area of the bath. To reduce measurement error.20 M icrons (R oughness ) . prior to the start of the experiment". "We usually have defined the CT characteristics during the measure and analyze phases. Improve 64 10 . and they may be relative units. has usually been defined in the measure and analyze phases. with the minimum of variation. 4. The measurement units should be appropriate for the study. Where possible. a gauge R&R study has been carried out on measurement systems. it is vital that for each CT characteristic. or at different positions throughout the bath. to the knowledge that leads to optimum operating conditions. if the experimental measure is a chemical bath temperature. it may matter that the temperature is taken from a single position in the bath. or concentration etc.0. such as weight. Where possible continuous variables should be selected for output responses. the CT Characteristics for experiments.Define the “CT Characteristics” Experiment Planning Sheets Agent: _______________________ Department: ___________________ Process: ________________________ Experimental Planning Sheet Page 1 Date: ________________ Problem Statement: DOE Objective: Relevant Background: List of Experiment Parameters CT Characteristics Relationship to Objective Specifications C C T haracteristics W thickness on eb W Pickup ing Brackets Surface Finish 1_05_01_063 R elationship to O bjective Estim absolute M ate ean difference from target and Standard D eviation Team has determ ined that the surface finish should be as high a quality as possible. For example. "This is step 3 in conducting a good experiment.

3. In considering step 4 of conducting a good experiment. while reducing variation.Define the “Control Factors” Experimental Planning Sheet Top of Page 2 Controllable Factors Control Factors Normal Settings Proposed Settings How are they Set Control Factors Feed Rate Normal Settings 3” per minute Proposed Settings 1 Inch per minute and 5 Inches per minute 90 PSI and 150 PSI How are they Set Machine Control pre-set by the operator Operator control set using a calibrated gauge Air pressure 120 PSI Experiment Design Tools Define the "Control Factors" "What should I think about when considering factors for an experiment"? asks the Black Belt. We do this to determine which factor. all the potential factors that can be used in the experiment can be broken down into three types: 1. usually called nuisance or noise factors. replies the Master Black Belt. believe will have the greatest effect on the CT Characteristics. of important control factors. as they are the factors that we will use to vary the input settings of the process throughout the experiment. or combination of factors. Factors that are uncontrollable. will be the conclusions from the experiment". and the team. 2. the more generally applicable. The control factors are chosen carefully. Improve 65 1_05_01_064 . Control factors (discrete or continuous) thought to be very influential. Factors that are held at a constant level and are controllable. has the greatest effect of centering our process on a target. "Factors chosen for the experiment are usually those that the Black Belt. and estimation of the effects these control factors will have on the response variables is very important. As with response variables it is seemingly easy to generate a list of candidate control factors. The selection. The wider the range of conditions included in the experiment.

in one of these factors that are held constant. and they must record any occurrence where something changes. but that you cannot vary are monitored closely. this is a key aspect of designing any experiment. "Yes. replies the Master Black Belt. Many good experiments fail. because the team only considered the control factors. It is vital. The personnel carrying out the experiment must be made aware of this fact. All operators to know how to operate the process How to Control Conducted runs when machine is at temperature Use one “average” operator in the experiment How are they Set Machine to be running 2 hours prior to start of experiment Chosen by supervisor ( For the experiment ) Improve 66 Experiment Design Tools Define "Held Constant Factors" "Is it important to think about factors that we can control and keep at a constant setting. but cannot vary in our experiment". that other factors that you feel are important. . that show up as special cause variation during our analysis of the completed experiment".Define “Held Constant Factors” Controllable Factors Control Factors Normal Settings Proposed Settings How are they Set Experimental Planning Sheet Bottom of Page 2 Factors Held at a Constant Setting Constant Factors Desired Setting How to Control How are they Set Constant Factors Temperature of machine oil Operators 1_05_01_065 Desired Settings 88 to 96 0F. they were going to change throughout the experiment. asks the Black Belt. and held constant throughout the experiment. This may allow us to determine a reason for any data points.

is a factor we think would have an effect on the CT characteristic. Improve 67 1_05_01_066 Ensure that no fork lift trucks pass by. those that are known but uncontrollable. and hold constant during an experiment?" asks the Black Belt. replies the Master Black Belt. and this usually achieved through a complete randomization of the experimental sequence". will have a significant effect on the CT characteristic. However. We can use the blocking technique if we think a noise factor. than the variation due to the chosen factors. it important to try to minimize the effects of all types of noise throughout the experimental sequence. those that are known and controllable. and finally those that are unknown and therefore uncontrollable. or greater. or nuisance factors. they can be grouped into three types. that is known and controllable. Generally a noise factor. the variation due to background or nuisance variables will be as great. . or materials stacked during experimental runs. "These factors are called noise.Define “Noise Factors” Noise Factors Experimental Planning Sheet Top of Page 3 N oise Factors Strategy Em ployed to Control N oise Factors Random ization & Replication Conclusions from the Statistical Analysis Noise Factors Ambient temperature in the Clean Room Vibration of Machine Strategy to Control Noise Factors Make all runs below 70 0 F Experiment Design Tools Define Noise Factors "What do you do about factors that you cannot control. In many experiments.

must be the safe operation of the experiment across all the experimental conditions". then it may be possible to assess its impact. is there any tips you have for selecting factor levels for the experiment"? asks the Black Belt. this may inflate the experimental error. with many possible levels. unless you vary the factor levels enough. For the low level. The upper limit of safety has been reached! Experimental Effect Lo (-) Y Hi (+) Factor Settings True Effect Lo (-) Hi (+) Factor Settings Experiment Design Tools Choosing the Variable Levels "Once I have chosen my factors for the experiment. If we have qualitative variables. the range of factor levels. selected for the study. "What about factors that are held constant throughout the experiment and nuisance factors"? asks the Black Belt. Improve 68 Experimental Effect 1_05_01_067 . the high setting for the factors can be determined by computing approximately 20% of the distance between the specifica-tion limits. if you can define levels for a nuisance factor. As far as nuisance / noise factors are concerned. held constant factors should be kept at their nominal settings. In the majority of cases. If we have a quantitative variable with specifications. replies the Master Black Belt. then we should select levels using extreme values when possible.Choosing Variable Levels Y Factor Settings True Effect I think you can turn off the water now. will ultimately determine the degree of belief in the data obtained as a result of the experi-ment. Where possible. they differ from held constant factors in that they cannot deliberately be kept at a constant level throughout the experiment. and make it difficult to assess the significance of the control factors. the 20% value is subtracted from the lower specification limit. the chances of getting good experiment results are greatly reduced. "Where possible. However. replies the Master Black Belt. a key consideration in selecting factor levels. and tightly controlled throughout the experiment. through the use of blocking". and adding this amount to the upper specification limit. "In step 5 of conducting a good experiment.

If we wish to study the relationships between the factors in depth. "Well! You are now ready for step 6. of conducting a good experiment. replies the Master Black Belt. asks the Black Belt. two to four factors can be studied in a full factorial design.Selecting the Experimental Design Current Knowledge Type of Experiments Little knowledge Fractional Factorials (Screening Studies) Some Knowledge Factorial Studies (New levels. experiments can be repeated using new levels for factors previously studied. These fractional factorial designs are used to examine five or more variables in these initial studies. a confirmatory experiment is usually run. in order to increase the degree of belief in the data obtained from the previous experiments". from experience gained. As we gain knowledge. Improve 69 Much Knowledge 1_05_01_068 Confirmatory Full Factorial Studies . The initial phases of DOE's will usually be screening studies. and some new factors. to focus on finding the key factors affecting a process. what is the next step in planning an experiment". As new theories are developed. New factors) Experiment Design Tools Selecting the Experimental Design "Once I have established all the factors. which is to select the experimental design that will meet the stated objective of your study. and their associated levels.

For example.) for the study". DOE runs on a milling machine carried out on different days. that many studies will have budgetary constraints. but what benefit does it give us"? asks the Black Belt. the initial experiment should consume about 25% or less. and give us increased confidence in the results of the experiment. Your costs in PETMET should reflect the full cost of all your proposed DOE cycles. it is desirable to carry out between 2 to 5 replications for each run". over a small range of factor settings. but the breadth of conditions under which the comparisons are made. Improve 70 . "Surely the amount of replication in an experiment. materials.Replication Pressure : Temp: HHHH LLLL HHHH LLLL HHHH LLLL HHLL HHLL HHLL HHLL HHLL HHLL 1st Replication Yiel d 130 120 2nd Replication 110 100 In d e x 1_05_01_069 5 1 0 1 5 2 0 2 5 Test Sequence Experiment Design Tools Replication "I know that replication refers to repeating aspects of an experiment. by enabling nuisance variables to be averaged out". budget. is the primary means for analyzing the stability of factor effects. will not give us confidence in our experimental results. of the resources allocated to the total DOE study. and made from different material batches. replies the Master Black Belt. "Replication in a well designed experiment. etc. As a rule of thumb. is not the number of replications per level of a factor. directly affects the amount of resources needed (time. on the main factor effects. However. This minimizes the impact of noise variables. Replication. would be preferred to a study based on consecutively produced units which in DOE terms is called repetition. But always remember the most important consideration in conducting a DOE. "As a guideline. replies the Master Black Belt. asks the Black Belt. Replication provides a measure of the magnitude of variation in the experiment due to nuisance variables. it's a fact of life.

whereas changing all the other factors could be done in a few minutes. "In producing any design in Minitab. the randomization of the DOE is carried out in Minitab within each block. or have been identified and are controllable. the effect of a machine warming-up. the runs are automatically randomized. In some other cases. unless you elect to turn off this function. . and found to be prohibitive". or position of parts in an oven etc". "When blocking is employed. "A rule of thumb with respect to randomization for analytical studies is.Randomization Run Order 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 B 2 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 C 3 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 D Block 4 No. Noise Factors are process variables that affect the CT Characteristic but have not been identified by the Black Belt or have been identified. is a tool that addresses the problem of noise (nuisance) Factors. would be randomized to assign combinations of factors to be carried out during each shift. Experiment Design Tools Randomization "I notice in Minitab. a random order for the conduct of a test could be prohibitively expensive. Typical nuisance variables are environmental effects such as temperature or humidity. an experiment with a block based on day and night shift. why does this happen?" asks the Black Belt. changing the level of one particular factor might require a machine to be shut down for most of the day. rather than over all runs in the design. that standard order of the experimental runs is not in the order shown in my textbooks. In cases where it is difficult to fully randomize the design. a great deal of care should be taken in interpreting the results of the experiment". after restricting important background variables in the form of blocking. but are not controllable. -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 2 1 1 2 1 2 2 1 1 2 2 1 2 1 1 2 Reac Temp Time A B C 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 230 230 300 300 230 230 300 300 230 230 300 300 230 230 300 300 30 30 30 30 60 60 60 60 30 30 30 30 60 60 60 60 Cat D Std Std Std Std Std Std Std Std New New New New New New New New Run Day Run Night 8 8 7 1 4 13 7 3 6 3 2 5 6 2 1 5 Randomized sequence for both experimental blocks Improve 71 1_05_01_070 Blocking disallows the given variable from influencing the primary response variable. For example. batch-to-batch variation in raw materials. randomize in all remaining situations unless the constraints to randomization have been considered. Randomization. replies the Master Black Belt. For example.

not only will they have to run the experiment in the randomized sequence. is to run a pilot study. are another aspect in successful experimentation. A way to ensure that this rigor and attention to detail is covered. TA DA • Reset each controllable factor as accurately as possible after each experimental run. Improve 72 1_05_01_071 • Measure the CT characteristics as accurately as possible. • Ensure that if Blocking is involved in the design. It is vital that the person. then hold them constant. to verify and improve the data collection procedures. asks the Black Belt. who will be carrying out the DOE and setting the factor levels. The operators running the process. as defined by the DOE design. Poor planning leads to poor execution. are a key component for a successful experiment. or persons. • Record any unusual occurrences that happen when running the experiment. the runs are randomized in accordance with the blocks. So. only a poorly designed or poorly executed one. are fully conversant with the need for rigor and attention to detail. replies the Master Black Belt. and collecting the data?". as the results of the experiment can be greatly affected due to a large degree of measurement error. I cannot over emphasize the need for highly capable measurement equipment. before starting the actual experimental sequence. .Collecting the Data Key tasks in Collecting Data: • Ensure that the experiment is run in a randomized sequence. they say that there is no such thing as a poor experiment. • Hold all other factors as stable as possible during the experiment. They will need to re-set the control variables. throughout the experimental sequence. Experiment Design Tools Collecting the Data "Are there any things I should take into account when running the DOE. "Collecting the data is step 7 of conducting a good experiment. (Ensure the design has been randomized in Minitab). that needs to be carefully monitored throughout the experimental cycles". the accurate setting of controllable factor levels. and usually measure the output from each experimental run. they will also need to record in detail any unusual occurrences at are present during the experiment.

Improve 73 Investigate 1_05_01_072 . "What do you mean by practical significance. After producing the ANOVA table. for graphical interpretation of two-way interactions". and checking the residuals I always like to determine if the results of the ANOVA analysis have any practical significance". We can also produce a "Normal Probability Plot for Effects". after that we produce main effect plots and interaction plots using the functions.Analyzing the Data (ANOVA) Create a General Linear Model ANOVA No Does the residual model fit the assumptions of normality? Check the reason for the non-normality in the residuals and rerun that experimental combination if required Create Main Yes Identify the vital few factors with Practical Significance (Pareto or Pie Chart) and or the Normal Probability Plot of Effects Experiment Design Tools Effect Plots Interaction Plots Analyzing the Data (ANOVA) "What steps should I follow when I analyze my experimental data?". replies the Master Black Belt. has on the process". and ensure that the residual model fits all the assumptions of normality. "It is possible for something to be statically significant. and interaction effect. "Analyzing the data is step 8 of conducting a good experiment. I then store the "Residuals" and "Fits". surely we have determined all factors and interactions that are significant from the ANOVA table?" asks the Black Belt. "Are there any more steps in analyzing the data?" asks the Black Belt. to produce a Pareto chart or pie chart. but be of little practical use in improving the process. Stat > ANOVA > Main Effect plot. The first task in analysis is to construct an ANOVA table for the full model in Minitab. "Yes. and Stat > ANOVA > interaction Plot. So. and from analyzing these charts we can determine what practical significance each main effect. asks the Black Belt. we analyze Sum of Squares in the ANOVA table. replies the Master Black Belt. Using the function Stat > ANOVA > Residual Plots. for a graphical interpretation of main effects. replies the Master Black Belt.

Confine all initial conclusions and deductions. giving the background to the study. explain the results / conclusions in terms of practical meaning. always try to explain the analysis in both graphical as well as numerical terms. This is step 9 in conducting a good experiment". and if there is a need for further work". to the experimental evidence at hand. and ensure that the results can be reproduced. Point out the way ahead. Explain results of the experiment in terms of graphs as well as numbers. is to supply sufficient information to permit readers of the report to verify results. and the meaning of the results. and make our recommendations to senior management.Conclusions and Recommendations This way to an Optimized Process • • • • Check all observed data for accuracy. that it is vitally important to point out to the Champion. I find it of great benefit to focus on using tabular and graphic methods of presenting data. as well as their statistical significance. It is good practice to replicate the optimum set-up. "I always write a report clearly describing the experiment work. and the owners of the process. says the Master Black Belt. and the personnel who work on the process. says the Master Black Belt. and have determined the practical significance of each factor and interaction. A rule I always try to adopt. Limit conclusions to summary evidence taken directly from the experiment. • Experiment Design Tools Conclusions and Recommendations "Once we have analyzed the data. we determine the combination of factors and interactions. When we have proven our findings. but limit conclusions to objective summary of evidence provided by the study". and to draw their own conclusions. "What steps do you follow in drawing your conclusions?" asks the Black Belt. Improve 74 1_05_01_073 . the results of the study. I have found out. that you have considered all the observed data. As we will need to explain the results of the study. the implication of the results. pertinence of problems. that optimize our process to meet our CT objectives." "From this data. and gain buy-in from the Champion. State results in practical terms as well as statistical terms. "Some key points are to check. if further experiments are required. we have to draw our conclusions.

this knowledge. Lessons Learned • In order to design an experiment. and put controls on the key aspects of the process. • • • 1_05_01_074 Experiment Design Tools Achieve the Objective "What do we do. Control key CT Characteristics using methods outlined in the control phase. and is set at the optimum operating configuration. "Well. You are now ready to institutionalize the changes. such as SPC to control the critical to process parameters (CTPs)". to ensure that entropy does not affect the optimized process. we must first clearly understand the process and decide what feature it is that we want to investigate. over time.Achieve the Objective • When the process has been optimized it is time to institutionalize the changes. replies the Master Black Belt. using control techniques. "When we have proven that the process is improved. This is the main reason for the control phase. It has been my experience. over time." replies the Master Black Belt. if we find that we have not optimized the process from our experiment?" asks the Black Belt. We do this until the desired result is obtained. what comes next?" asks the Black Belt "If you have reached this stage in your experimental process. Improve 75 . usually it is impossible to fully optimize a process based on one experimental sequence. We can improve and optimize the process but if we do not institutionalize the recommended changes. Control key process control factors using methods outlined in the control phase. Keep the process operating at that the peak of its performance. usually points out the direction that the Black Belt should go in setting factor levels for another experiment. You will remember from your training. Entropy. additional understanding of what is important will be gained and another experiment can be designed. Once the experiment is completed. that it was recommended that the first experiment consume only 25% of the budget and resources. that you always find something from an experiment that no one knew about the process. formulated by the DOE's. is the fact that everything in nature decays. then you are ready to implement step 10 of conducting a good experiment. it will revert back to a less than perfect process.

practical significance (sum of squares. actively communicate your plan with all those concerned is vital to the success of an experiment. The ANOVA table is used to assess statistical significance initially and then practical significance of factor effects and interaction effects on the output. Even if the main effect is not statistically significant. It is common practice to do a fractional factorial experiment when we have five factors or more. Two factor interactions are common. factor levels should be set wide.) and main effect plots (with interactions considered). Graphs are also useful to visualize interaction effects between factors and to find the optimal settings for both factors.• • Each DOE has two elements: the "Experimental Design" and the "Analytical Approach". The most common way to design an experiment (so that all combinations are considered) is to use Yates Standard order. A factor may be statistically significant. Remember. We use pie charts and Pareto diagrams to visually display the relative effects of several factors. all within one experiment. a DOE can provide a wealth of knowledge about several factors (and their interactions) in a very cost effective manner because several factors can be varied at the same time. it may affect the output through an interaction with another factor. pie chart. etc. A successful experiment does not happen by chance. Compared to the traditional one-factor-at-a-time approach to experimentation. There is no such thing as a poor experiment. In general. 1) is used to set factor levels and coefficients. • • • • • • • • • • • . but not so much so that they extend beyond the operational range of the factor. We then often plot the main effects on a graph in order to visualize the difference in the response output between the low and high settings of each important factor. only an experiment that is poorly designed or executed. but three factor and higher interactions are rarely seen in our industry. but practically insignificance. The two elements must be well thought through in order to collect appropriate data that can be meaningfully analyzed. This notation (-1. it takes careful planning and attention to detail to ensure that all designed experiments produce meaningful results. The Contrast concept is used in statistical analysis to determine if a design is balanced and to determine orthogonality. We should consider both the main effects and interaction plots prior to specifying optimal settings. The general steps for conducting a statistical analysis are ANOVA (with residual analysis).

Sign up to vote on this title
UsefulNot useful