A well-run DOE method leads the way

to better products and processes.

________________________
by Ranjit K. Roy, Ph.D.

Improvement can be achieved in one or more of

Figure 1: Performance Before
Experimental Study
the many characteristics of any given product or
process. In most situations, however, improvement
primarily implies that performance is enhanced.
Experimental design is one technique that can be
learned and applied to determine product or
process design for improved performance.

For a high-volume manufactured part, the two

statistical performance characteristics that
manufacturers typically aim to achieve are
improving the mean (average) and reducing the
variability around the mean. For improvement, our
goal is to move the performance of a population of
parts to the target and minimize the variability
around it (see figures 1 and 2). No matter the Figure 1: Performance Before
Experimental Study
application, performance consistency is a desirable
characteristic to achieve. Performance consistency
is achieved when the performance is on-target
most of the time.

An effective way to improve performance is to

optimize the engineering designs of products or
processes by experimental means. A structured
and economical way to study projects whose
performance depends on many factors is to apply
the experimental method known as "design of
experiments," a statistical technique introduced in
the 1920s by Ronald A. Fisher in England. In the
1950s, Genichi Taguchi of Japan proposed a much-standardized version of the technique for
engineering applications. His prescription for experiment designs, a new strategy to incorporate the
effects of uncontrollable factors and ability to quantify the performance improvement in terms of
dollars by use of a loss function, made the DOE technique much more attractive to the practicing
engineers and scientists in all kinds of industries.

In January, John Wiley & Sons Inc. published my book on DOE/Taguchi technique. Intended
primarily for the self-learner, the book takes the reader through the entire application and analysis
process in 16 different steps. One who learns the topics covered in these 16 steps well will be able
to handle more than 99 percent of the situations common to manufacturing and production
activities. Following are the 16 steps you will need to master DOE using the Taguchi approach for
your own product and process design improvement.

Step 1: Design of experiments and the Taguchi approach

A quick review and understanding of the Taguchi version of DOE is essential before diving into
the subject. The purpose here is to gather a clear understanding of what DOE is and understand
how Taguchi standardized the experiment design process to make the technique easier to apply.

Step 2: Definition and measurement of improvement

No experiment that lacks the means to measure its results is complete or useful. A clear definition
of objectives and measurement methods allows us to compare two individual performances, but a
separate yardstick is needed to compare performances of one population (multiple products or
processes) with another. In general, individual performance measures are different for different
projects, but consistency is the means by which we measure population performance. Consistent
performance produces reduced variations around the target (when present) and results in reduction
of scrap, rejection and warranty. In this step, you learn how population performances are measured
and compared.

Step 3: Common experiments and analyses methods

A common practice for studying single or multiple factors is to experiment with one factor at a
time while holding all others fixed. This practice is attractive, as it's simple and supported by
common sense. However, the results are often misleading and fail to reproduce conclusions drawn
from such an exercise. A more effective method for these situations is to study their effect
simultaneously by setting up experiments following the DOE technique. This step should lead to
some understanding of basic DOE principles.

Step 4: Designing experiments using orthogonal arrays

The word "design" in "design of experiments" implies a formal layout of the experiments that
contains information about how many tests are to be carried out and the combination of factors
included in the study. Once the project is identified, the objectives and factors and their levels are
determined by following a recommended sequence of discussion in a planning meeting. There are
many possible ways to lay out the experiment; the best method depends on the project. A number
of standard orthogonal arrays (number tables) have been constructed to facilitate designs of
experiments. Each of these arrays can be used to design experiments to suit several experimental
situations. This step should be devoted to learning about the different orthogonal arrays and
understanding how easy it is to design experiments by using them.

Step 5: Designing experiments with two-level factors

Experiments that involve studies of factors with two levels are both simple and common. There are
a set of orthogonal arrays (designated as L-4, L-8, L-12, L-16, L-32, L-64, etc.) created specifically
for two-level factors. Experiments of all sizes can be easily designed using these arrays, as long as
all factors involved are tested at two levels. By completing this step, you will learn how quickly
experiments involving two-level factors can be designed and analyzed using the standard
orthogonal arrays.

Step 6: Designing experiments with three-level and four-level factors

When only two levels of factors are studied, the factors' behavior is necessarily assumed to be
linear. When nonlinear effects are suspected, more than two levels of the factors are desirable.
Although many larger two-level orthogonal arrays can be modified to accommodate three-level and
four-level factors, a set of standard arrays such as L-9, L-18, L-27, modified L-16 and modified L-
32 are also available for this purpose. This step should help you learn the design and analysis of
these more complex experiments.

Step 7: Analysis of variance (ANOVA)

Calculations of result averages and averages for factor-level effects, which only involve simple
arithmetic operations, produce answers to major questions that were unconfirmed in the earlier
steps about the project. However, questions concerning the influence of factors on the variation of
results --in terms of discrete proportion --can only be obtained by performing analysis of variance.
In this step, you'll learn how all analysis of variance terms are calculated. Utilize this step to review
a number of example analyses to build your confidence in interpreting the experimental results.

Step 8: Designing experiments to study interactions between factors

Interaction among factors, which is one factor's effect on another, is quite common in industrial
experiments. When experiments with factors don't produce satisfactory results, or when
interactions among factors are suspected, the experiment must accommodate interaction studies. In
this step, your objective will be learning how to design experiments to include interaction and how
to analyze the results to determine if interaction is present. You will also learn how to determine the
most desirable condition in cases in which interaction is found to be significant. Although
interactions among several factors, and between factors at three or four levels, are also present,
studies and corrections for interaction between two two-level factors will suffice for most
situations.

The materials in steps 1-8 prepare you for many applications in the production floor. As long as
the factors you want to study are all at the same level, you're able to design experiments using one
of the available orthogonal arrays. You're also able to analyze the results of such experiments
following the standard method of analysis, which uses the averages (means) of the multiple sample
test results of individual experiments, and determine the optimum design conditions. With the
knowledge you should gather in these steps, you can indeed apply the DOE to solve most
production problems whose solutions lie in finding the proper combination of the controllable
factors, instead of some special causes.

The reality, however, is that you will often have factors at mixed levels; some will be at three-
level, some at four-level, and many at two-level. You also need to learn how to analyze the results
for variability. Recall that it's the reduction of variability, which instills performance consistency,
that we're after. The following additional steps address these items and prepare you to handle most
every type of experimental situation.

If your applications always involve production problem solving, you may find that your
knowledge up to this point is quite adequate for the job. Nevertheless, you may want to sharpen
your application skills before proceeding to learn about the advanced concepts in the technique
described in the eight steps that follow.

Step 9: Experiments with mixed-level factors

Experiment designs with all of the factors at one level are easily handled using one of the available
standard arrays. But these standard arrays can't always accommodate many mixed-factor situations
that you might find in industrial settings. Most mixed-level designs, however, can be accomplished
by altering the standard orthogonal arrays. Your goal will be to learn the procedure by which
columns of an array are modified to upgrade and downgrade the number of levels in creating a new
column. This way, a two-level array can be modified to have three-level and four-level columns.
Conversely, to accommodate a factor with a lesser number of levels, a four-level column can be
reduced to a three-level, and a three-level column to two-level, by a method known as "dummy
treatment."

Step 10: Combination designs

For some applications, the factors and levels are such that standard use of the orthogonal array
doesn't produce an economical experimental strategy. In such situations, a special experiment
design technique such as a combination design might offer a significant savings in number of
samples. This step will familiarize you with the necessary assumptions that must be made in order
to lay out experiments using combination design. With this technique, generally, two two-level
factors are studied by assigning them to a three-level column.

Step 11: Robust design strategy

 Variations among parts manufactured to the same specifications are common even when attempts
are made to keep all factors at their desired levels. Remember, variation reduction is our ultimate
goal. When performance is consistently on-target (the desired value), the customer perceived
quality of the product is favorably affected. Variation is most often due to factors that are not
controllable or are too expensive to control. These are called the "noise factors." In robust design
methodology, the approach is not to control the noise factors, but to minimize their influence by
adjusting the controllable factors that are included in the study. This new strategy, promoted by
Taguchi, reduces variability without actually removing the cause of variation.

Step 12: Analysis using signal-to-noise (S/N) ratios

The traditional method of calculating average factor effects and thereby determining the desirable
factor levels (optimum condition) is to look at the simple averages of the results. Although average
calculation is relatively simple, it doesn't capture the variability of results within a trial condition. A
better way to compare the population behavior is to use the mean-squared deviation, which
combines effects of both average and standard deviation of the results. For convenience of linearity
and to accommodate wide-ranging data, a logarithmic transformation of MSD (called the signal-to-
noise ratio) is recommended for analysis of results. This step will teach you how MSD is calculated
for different quality characteristics and how analysis using S/N ratios differs from the standard
practice. When the S/N ratio is used for results analysis, the optimum condition identified from
such analysis is more likely to produce consistent performance.

Step 13: Results analysis using multiple evaluation criteria

Often, a product (or process) is expected to satisfy multiple objectives. The result in this case
comprises multiple evaluation criteria, which represents performance in each of the objectives. It's
common practice, however, to analyze only one criteria at a time because different objectives are
likely to be evaluated by different criteria, each of which has different units of measurement and
relative weighting. When the results are analyzed separately for different criteria and the desirable
design conditions are determined, there is no guarantee that the factor combination will all be alike.
An objective way to analyze the results is to combine the multiple evaluations into a single
criterion, which incorporates the units of measurements and the relative weights of the individual
criterion of evaluation. You should devote your time during this step to learning the principles
involved in formulation of an overall evaluation criterion for analysis of multiple objectives, when
present.

Step 14: Quantification of variation reduction and performance improvement

Most of your DOE applications allow you to determine optimum design that is expected to
produce an overall better performance. The improvement of performance often means that either
the average or the variations (or both) have improved. When the new design is put into practice
(i.e., the recommended design is incorporated), it's expected to reduce scrap and warranty costs. In
turn, this reduction more than offsets the cost of the new design. The expected monetary savings
from the improved design can be calculated by using Taguchi's loss function. In this step, you'll
learn how to estimate the expected savings from the improvement predicted by the experimental
results. Further, you'll also learn how the expected improvement in performance from the new
design is expressed in terms of capability improvement indexes such as Cp and Cpk.

Step 15: Effective experiment planning

As far as the benefits from the technique are concerned, experiment planning is the most important
among the different application activities. Therefore, it's a required first and necessary step in the
application process. Planning for DOE/Taguchi requires structured brainstorming with project team
members. The nature of discussions in the planning session is likely to vary from project to project
and is best facilitated by one who is well-versed in the technique. Your effort in this step will be to
learn the structure of proven planning sessions documented by experienced application specialists.

Step 16: Review of example case studies

The application knowledge gained in steps 1-15 could be overwhelming if you didn't have
immediate projects on which to practice. One way to build more confidence and extend your
application expertise is by familiarizing yourself with numerous types of case studies with
complete experiment design and results analysis. In this final step, you should seek out and
thoroughly review complete project application reports. Complete case studies should contain
discussions under most of the following topics:

Project title or problem definition

Project objective(s)

Evaluation criteria and quality characteristic

Identified factors and levels and those that are included in the study

Suspected interactions and those that are selected for the initial study

Uncontrollable factors (noise factors) and how they were treated

Sequence of running of the experimental conditions

Measured results, which represent evaluation of different objectives

Main effects indicating the trend of factors' influence

Analysis of variance for relative influence of the factor to the variation of results

Optimum condition and the expected performance

Improvement and expected monetary savings

Graphical representation of variation reduction expected from the improved design

 Now that you have an idea about the topics and the study sequence, one question remains: How do
you actually go about learning them?

To get yourself comfortable with DOE application knowledge, you will need to understand four
phases in the application process: (1) experiment planning, (2) experiment design, (3) results
analysis and (4) interpretation of results. Of these, you need not --and may not be able to afford the
time --to be too good with experiment design and number crunching. These are mundane tasks, so
feel comfortable letting a computer program do the work for you; your focus should be to learn the
practiced and proven discipline of how to plan an experiment following a structured sequence of
discussion. The experiment planning process requires more the art of teamwork than experimental
science. Only the experienced can describe and share methods that have worked. Look for
references that describe and teach the technique through application examples.

Both experiment planning and interpretation analysis are areas you'll want to gain control over.
The nature of discussions and findings in these areas are always project-specific. As the
experimenter, you'll know far more about these two areas than anyone else. Good knowledge of the
project objectives, how objectives are evaluated, how the factors included in the study were
selected, and so on will help you confidently interpret results from the routine analysis. You will
benefit most when your reference book stresses application rather than theory.