You are on page 1of 9

– The Need for Experiments

• Begin Module 1

– Example of Experiment with One Factor

• Module 5

– Design and Analysis of Experiments

• Module 4

– Testing the Change

• Module 3

– Sequential Experiment s and PDSA

• Module 2

– The Need for Experiments

• Module 1


Rajiv Gupta
BITS Pilani

Design of Experiments



Control Charts: X-bar and R Charts

• In control charts we plotted data points over time to
monitor whether a process was in control/stable
• Only when an abnormality was observed in the
process output that we intervened to investigate the
cause of the abnormality and to correct it
• This method is a passive method because we are not
taking proactive action to bring about a change in
the system. Rather we are reacting to specific
occurrences that are observed.
• Only special causes are addressed

Control Charts

• Common causes cannot be explained. This is the
inherent variation in a process. Only management can
take action to change the system.
• Trying to adjust to common causes may increase
• Special causes need to be investigated and corrective
action taken

– Common causes
– Special causes
– 85 % of the problems are caused by the system
(management) and 15% by the workers

• Deming’s causes of variation

Process Variation




there are common causes of variation which require management action to bring about improvement in the system/process • Such action needs to be proactive where we attempt to determine what changes will bring about positive change • Experiments are needed to evaluate the effect of making changes to the system so that improvement can be effected.(outputs of a process of shooting arrows) Location and Dispersion 8 Reducing Fraction Defectives 9 – Sequential Experiments and PDSA • Begin Module 2 • End of Module 1 • In order to be reasonably confident that a particular change will bring about a desired/positive outcome.. the experiment needs to be carefully planned • Design of experiments is intended to provide a framework for conducting the experiments in a planned. how the system is expected to behave in the future as a result of the changes investigated through experimentation • In addition to special causes. i. methodical fashion • The results of the experiment are interpreted as a prediction. 7 Experiments Proactive Action on a Process 12 11 10 .e.

e. etc.What changes can we make that will result in an improvement? How will we know that a change is an improvement? • Examples could be performance. aesthetics. time.Moen. who will conduct the test. what will be outcome measures. i. how many replications will be needed. usability. where and when will it be done. Nolan. which can be used in subsequent tests – Be innovative in the design of the test without getting too complicated • During the Plan phase of the testing cycle: 17 Implementing a change S Testing Using PDSA Cycles Developing a change P A P A S Sequential Building of Knowledge – Need to understand the nature of the problem at hand – Need to be familiar with new technologies that could help improve the process or system – Need to consider alternative materials and alternative suppliers of the materials – Need to be innovative • What changes can we make that will result in an improvement? . – We should look for ways to scale down and decrease the time required for the initial test so that more knowledge is gained quickly. durability. Lloyd P. consistency. – Need to clearly identify the criteria or measures for improvement – Multiple measures that could be in conflict – Need to have a systematic way to prioritize measures • How will we know that a change is an improvement? • Customer complaints • Too much rework • Delays in delivery – Define objectives of the study • What are we trying to accomplish? 15 14 From “ Quality Improvement through Planned Experimentation” by Ronal D. etc.. Provost 13 Cycle for learning and improvement Current Knowledge What are we trying to accomplish? The Model for Improvement D S A S Testing a change D P D A D P 16 18 – We need to clearly state how the test is to be performed. Thomas W.

we compare what our predictions were with regard to the outcome of the change. we get information that helps us in planning further tests and retests. we actually conduct the test or experiment. to the actual results. if required – The test not being conducted properly – The support processes not being adequate – The result is actually not favorable • During the Do phase of the testing cycle. we should attempt to gain understanding why the prediction was not accurate • If the test outcome matches the prediction. • If the outcome is not as expected. The outcome should provide data for analysis and gaining knowledge about the effect of the change. then our degree of belief in the change outcome is increased Testing Using PDSA Cycles 19 • Whatever be the outcome. it might be due to: Testing Using PDSA Cycles • It is important to design the test or experiment carefully as the outcome and the interpretation of the results will determine future course of actions • As discussed earlier. it is recommended that tests be conducted in sequential cycles so that our degree of belief increases that the change will result in improvement • It is also important to ensure that the improvement will be sustained in a variety of circumstances • It is also essential to make sure that the outcome of the experiment is due to the tested change as opposed to any other reason Testing the Change – Testing the Change • Begin Module 3 • End of Module 2 24 23 22 .21 – Is further testing required to increase the degree of belief? – Should we test other changes to study their effect? – Should modification be made to the current change and be retested? – Should we alter the experiment conditions to retest the effect of the change? – Are we ready to roll out the change on a full-scale basis? • During the Act phase of the testing cycle. • If the results do not match our prediction. we need to determine what is the next course of action: Testing Using PDSA Cycles 20 • During the Study phase of the testing cycle.

For example consider a change made in Week 7 and the results of the pre-test in Week 4 and the post-test in Week 11 are given below: Pre-tests and Post-tests 27 26 25 Delay Delay Delay 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 8 9 10 11 8 9 10 11 8 Week 7 9 10 11 Change made here Alternate Scenario 4 Week 7 Change made here Alternate Scenario 3 Week 7 Change made here 12 12 12 13 13 13 14 14 14 Alternate Scenario 2 30 29 28 .Before the change After the change Delay Delay 1 1 2 2 3 3 4 4 5 5 6 6 8 9 10 11 8 Week 7 9 10 11 Change made here Alternate Scenario 1 Week 7 Change made here 12 12 13 13 From”Quality Improvement through Planned Experimentation” Delay 14 14 One of the methods to ascertain the effects of the change is to conduct a measurement before (pre-test). and a measurement after (post-test).

polling of people. process. etc.• End of Module 3 33 32 36 – Are done to bring about a change in a product. any number of which may be sampled and studied – The aim of an enumerative study is estimation about some aspect of the universe – Action may be taken on the universe based on the estimate through sampling – Statisticians play a large role in enumerative studies – Examples of enumerative studies include Census. • Enumerative studies • Test on a small scale and build knowledge sequentially • Collect data over time • Include a wide range of conditions in the sequence of tests 34 Enumerative and Analytical Studies – Design and Analysis of Experiments • Begin Module 4 Basic Principles for Testing Change 31 – Plot a run chart with data from a few weeks before and after the change – Conducting the test over a longer time period will also add to our degree of belief – Remove the change and see if the process reverts back to the performance before the change – Use planned experimentation • How can we make sure that the change in output is caused by the change we made? Testing the Change . or system in the future – Since we are dealing with the future the universe is dynamic – The aim of an analytic study is prediction that one of several alternatives will be superior to the others in the future – The role of the subject matter expert is very important in analytic studies – Examples could be selection of one vendor from several for the supply of a given part • Analytical studies Enumerative and Analytical Studies 35 – Are done with a universe that exists at present and static – The universe consists of tangible units.

. each at 2 levels. measurements. Head 2 part 1. a batch. etc. such as factorial. So an experiment with 4 factors. It is a background variable typically unknown at the time of the experiment 37 Experimental Pattern Definitions . operator. lot. and nested • In factorial design. Machine 2 Head 1. machine. defined by background variables. e. experimental units. a single part. e. fractional factorial. m2 42 m1 m2 m1 m2 pt1 Head 2 Each factor at two nested levels The measurements are not in random order Machine Head within a machine Part within a head Measurement within a part pt2 Machine 1 Pressure 2 Temp 1 Temp 2 m1 m2 m1 m2 pt1 Head 1 Nested Experimental Design m1 m2 m1 m2 Four nested factors: m1 m2 m1 m2 pt1 Head 1 Machine 1 Load 1 Batch 1 Load 2 Load 1 Batch 2 Load 2 Pressure 1 Temp 1 Temp 2 Full Factorial Design 40 • The experimental pattern defines how the factors and factor levels will be studied • There are a number of experimental patterns.. will have 24 or 16 factor level combinations to be studied • In a fractional factorial design only a subset of the possible combinations of factor levels are examined • In a nested design the levels of a factor are examined within a given level of another factor • Response variable – A variable observed or measured in an experiment. • Nuisance variable – An unknown variable that can affect a response variable. an hour of production. Tools For Experimentation 39 38 • Level – A value or qualitative setting of a factor which would be varied during an experiment • Experimental unit – The smallest division of the experiment that can receive a particular set of factor levels. It is the outcome from the experiment. The variation within a block is expected to be less than the variation among blocks Definitions pt2 pt1 pt2 Head 2 pt2 41 Machine 1. An experiment could have more than one response variables • Factor – A variable deliberately changed in a controlled manner to observe its impact on the response variable • Background variable – A variable that can potentially affect a response variable but is not a factor. etc.• Experimental pattern – the arrangement of factor levels and experimental units in the design • Planned grouping – forming blocks of experimental units • Randomization – the objective assignment of combinations of factor levels to experimental units • Replication – Repetition of experiments. time. • Blocks – Groups of experimental units given similar treatment. Usually. each possible combination of factors is examined.g. part 2 m1. etc.g.

we can combine them to form chunks using some extreme values of the background variables to test their effect on the measurement of interest Planned Grouping – We can either keep the background variables constant – Or we can measure them – Or we can use planned grouping to set up blocks • How to control the background variables so that the factor effects are not distorted • How to use background variables to establish a wide range of conditions Planned Grouping No.43 45 • Need to test material from 3 different suppliers. of replication is constrained by the cost of the experiment – – – – • Replication – The assignment of factor combinations to experimental units and sequence of testing is randomized – Helps reduce the variation due to nuisance variables from being confused with variation due to factors • Randomization Randomization and Replication A. One factor at 3 levels • Five Background Variables Background Variable Levels Machine No. A natural grouping/blocking would be a car where 4 brands could be tested on a car • Or if we wished to test the wear on the soles of shoes. 1 Day 1 Block 2 B C A No. 4 Day 4 Block 4 46 • End of Module 4 48 47 Repeated measurements of experimental units Multiple experimental units for each combination of factors Helps minimizing the effect of nuisance variables Ideally it is better to get few replications on different days than get all replications on the same day – No. B. Sunil. 10 Sunil G-233 No. 2 Day 2 Block 2 Block 3 C A B No. and C are the 3 suppliers whose samples have to be tested Experiment grouping Test Block 1 1 B 2 A 3 C Machine Operator Gauge Saw blade Day Block 1 4 Blocks can be formed . a natural grouping would be 2 • When there are several background variables. 6 and No. 6 Ram G-103 No. 10 Anil G-233 No. 6 Gopal G-103 No. 10 Operator Ram. 3 Day 3 Block 3 Block 4 A C B No. Gopal and Anil Gauge G-103 and G-233 Saw blade 10 blades available Time (days) Different days possible Example of Chunk Variables 44 • Suppose we want to test the wear on different brands of tires.

Recently tools were replaced and the changes in averages were observed. Example – Example of Experiment with a Single Factor • Begin Module 5 Pressure Average lbs Range 20 16 30 10 40 8 80 5 • End of Module 5 We can conclude that clamp pressure reduces the average range of the dimension. In the R chart all points below the CL. Reduction in variation likely due to increased clamp pressure Day 1: (prior to experiment) Both Xbar and R charts are in control Summary of control charts: 54 53 52 .23 0 Day 1 Pressure 30 1 2 3 4 5 6 7 8 Day 2 Pressure 80 1 2 3 4 5 6 7 8 Day 3 Pressure 20 1 2 3 4 5 6 7 8 Control Chart for Ranges Control Chart for Averages CL UCL LCL UCL 50 Day 4 Pressure 40 51 1 2 3 4 5 6 7 8 The current clamp pressure is 30 pounds. we may wish to increase clamp pressure further Day 3 1 4 2 Day 4: (pressure 40 pounds) Bothe charts in control. No other causes noted. Slight decrease in variation Day 3: (pressure 20 pounds) Variation seems to have increased with one point out of control Likely due to reduction in pressure Day 2: (pressure 80 pounds) The Xbar chart is in control.3 10 0 7. The control chart for the 4 days of experimentation are shown next. -7. It was suspected that it was due to clamp pressure.3 49 The Xbar and R control charts for a critical dimension had been maintained for 6 months. If the effect continues. 40 pounds. We should change the clamp pressure To 80 lbs and monitor the situation to see if the effect is continued. and 80 pounds. at 20 pounds. So it was decided to test the effect of clamp pressure on the variation in the dimension. Each of these would be tested for one day. It was decided to test the dimension at 3 other clamp pressures.