Unit 4

Response surface methods:

• Introduction.

• The method of steepest ascent

• Analysis of first and second order response surface models

• Experimental designs of fitting response surfaces.

• The Taguchi approach to parameter design.

 Unit 4-RSM
Response surface methodology was developed by Box and Wilson in 1951 to aid
the improvement of manufacturing processes in the chemical industry. The
purpose was to optimize chemical reactions to obtain, for example, high yield and
purity at low cost. This was accomplished through the use of sequential
experimentation involving factors such as temperature, pressure, duration of
reaction, and proportion of reactants. The same methodology can be used to
model or optimize any response that is affected by the levels of one or more
quantitative factors. The models are generalizations of the polynomial regression
models studied earlier in Unit 3.
 Unit 4-RSM
Response surface methodology, or RSM, is a collection of mathematical
and statistical techniques useful for the modeling and analysis of
problems in which a response of interest is influenced by several
variables and the objective is to optimize this response.
For example, suppose that a chemical engineer wishes to find the
levels of temperature () and pressure () that maximize the yield () of a
process. The process yield is a function of the levels of temperature and
pressure, say
+
where represents the noise or error observed in the response y. If we
denote the expected response by , then the surface represented by = is
called a response surface.
 Unit 4-RSM
RSM is widely used in factorial design. Studying the relationship with
response as the dependent variable and the factor levels helps us
understand how the change in the factor level affects the response. Thus,
the combination of levels of factor, which produces optimum response, can
also be found.
In most RSM problems, the form of the relationship between the response
and the independent variables is unknown. Thus, the first step in RSM is to
find a suitable approximation for the true functional relationship between y
and the set of independent variables. Usually, a low-order polynomial in
some region of the independent variables is employed. If the response is
well modeled by a linear function of the independent variables, then the
approximating function is the first-order model
 Unit 4-RSM
If there is curvature in the system, then a polynomial of higher degree must be
used, such as the second-order model

Almost all RSM problems use one or both of these models. Of course, it is
unlikely that a polynomial model will be a reasonable approximation of the true
functional relationship over the entire space of the independent variables, but
for a relatively small region they usually work quite well.
 Unit 4-RSM
The objective of obtaining a response surface is twofold:

(i) to locate a feasible treatment combination x for which the mean

response is maximized (or minimized, or equal to a specific target
value); and

(ii) to estimate the response surface in the vicinity of this good location
or region, in order to better understand the “local” effects of the
factors on the mean response.
 Unit 4-RSM
In general, throughout the discussion we will think about maximizing the
response, but we show via an example that exactly the same techniques can be
used for minimizing a response. The techniques can easily be adapted when the
goal is to have the response close to a target value. One possible approach to
achieving the objective involves collecting observations at each location on a grid
of treatment combinations spanning the entire experimental region of interest.
However, the number of observations required by such a comprehensive approach
can be very large, and it grows very quickly as the number of factors under study
increases.
 Unit 4-RSM
Also, somewhat sophisticated modeling techniques would generally be
needed to obtain an adequate fit of a model over the entire region.
Instead, it is generally more efficient to conduct a sequence of small
“local” experiments with which to search out the location of the peak
mean response and then to study its vicinity.
 Unit 4-RSM
The best design will have the following features:
• Provides a reasonable distribution of points in the region of interest
• Allows investigation of lack of fit
• Allows experiments to be performed in blocks
• Allows design of higher order to be build up sequentially
• Provides an internal estimate of error
• Does not require large number of runs
• Does not require too many levels of the independent variable
• Ensures simplicity of calculations and model parameters.
 Unit 4-RSM
Linear Response Surface Design (The First Order Model):

Let there befactors each levels. Hence the design is factorial design.
Practically, all treatment combinations need not be considered as it does
not ensure the best fit of the model. Therefore, a suitable set of treatment
combinations, say , out of must be chosen. The linear response surface
that we need to fit is

where denote the level of the i-th factor.

 Unit 4-RSM
As we have observations, let denote the response due to j-th
treatment combination. In other words, is the j-th response.

Let denote the level of the i-th factor in the j-th treatment combination
and is corresponding error random variable with mean zero and
variance constant. Hence for each observation j- the model will be
 Unit 4-RSM
In matrix notation,
where,

and

We use the least square method to estimate the values. Differentiate

the sum of squares of errors, ϵ’ϵ, with respect to the parameters and
equate to zero, where,
 Unit 4-RSM
𝜖’𝜖 = (𝑌 − 𝑋𝑏)’(𝑌 − 𝑋𝑏)
= 𝑌’𝑌 – 𝑌’𝑋𝑏 – 𝑏’𝑋’𝑌 + 𝑏’𝑋’𝑋𝑏
= 𝑌’𝑌 – 2𝑏’𝑋’𝑌 + 𝑏’𝑋’𝑋𝑏
Now gives,
= – 2𝑋’𝑌 + 2𝑋’𝑋𝑏, and setting
= 0, implies
−2𝑋’𝑌 + 2𝑋’𝑋𝑏 = 0
 Unit 4-RSM
Hence, the normal equations are given by
(𝑋’𝑋)̂ = 𝑋’𝑌, where,
 Unit 4-RSM
To simplify the solution of normal equations the design has to be so
obtained that = 0, for all and for (i)

This means that the columns of the design matrix X are mutually
orthogonal. Moreover, except first column all the other columns are
contrasts. Any model that satisfies condition in (i) is called orthogonal
design. Thus, the linear response surface design that we fit is also
called orthogonal first order model. In other words, first-order
orthogonal design is one for which of X’X is a diagonal matrix.
 Unit 4-RSM
Under the conditions in (i) all the off-diagonal elements of X become
zero and the normal equations reduce to
Unit 4-RSM
 Unit 4-RSM
design does not influence the estimate of bi’s but the estimate of the b0
becomes the average of all the observations as the orthogonal property
of the design remains unaltered.

Suppose there are n observations made at the center point then these
points can be utilized to compute the estimate of pure error.

The testing of the model is done through ANOVA given below.

Unit 4-RSM
Unit 4-RSM
Unit 4-RSM
Unit 4-RSM
 Unit 4
Example : A chemical engineer is interested in determining the
operating conditions that maximize the yield of a process. Two
controllable variables influence process yield: reaction time and
reaction temperature at levels (30 min, 40 min) and (150oF, 160oF)
respectively. The engineer also makes observations at centre points.
The data are given below.
Unit 4
 Unit 4
We fit the first order model.
Unit 4
Unit 4
Unit 4
Unit 4
 Unit 4
We can observe model is adequate as the LOF is insignificant. Thus, the
individual computation of PQ and Interaction is not required but it is
discussed for the sake of demonstration.
 Unit 4-Variance Optimal Design
The first order orthogonal designs are variance-optimal design. This
means the variance of the regression coefficients, 𝑉ar, are minimized
provided all ∈{-1,1}. This means no other design, if N observations are
to be made, can produce the 𝑉ar as small as produced by orthogonal
design. This property of the design makes sure that the regression
coefficients are estimated with greatest precision.
 Unit 4-Variance Optimal Design
Example 1: If in an experiment there are two factors then observations
made at following design points will produce a variance-optimal design.
Factors

As where is a constant, the variance of regression coefficients depends

on the diagonal elements of X’X. For the above design matrix X’X =
4,thus. One can say that no other design with four experimental runs in
this region can result in variances smaller than /4.
 Unit 4-Variance Optimal Design
Note that the design discussed in example considered earlier is not a
variance-optimal design in 9 runs because it is augmented with center
points.
Find to verify this.

Do It Your self Exercise:

Construct a variance optimal design for an experiment wherein there
are three factors laid out in a single run. Also verify the orthogonality
conditions.
 Unit 4-Variance Optimal Design
Solution: The variance optimal design would be
Factors
 Unit 4-Variance Optimal Design
Example 2: If in an experiment there are four factors then observations
made at following design points will produce a variance-optimal design.

Factors
 Unit 4-Variance Optimal Design
As where is a constant, the variance of regression coefficients depends
on the diagonal elements of X’X. For the above design matrix X’X = 8,
thus. One can say that no other design with eight experimental runs in
this region can result in variances smaller than /8.

Exercise. Construct a variance optimal design for an experiment

wherein there are three factors laid out in a single run.
 Unit 4-Plackett-Burman Designs

Placket and Burman (1946) gave a series of design satisfying the condition (i). Thus,

Placket-Burman Designs are used to fit first order orthogonal designs. They produced the

design for all values of N, which are multiple of 4, to N = 100 except 92 (The design for N

= 92 is now known). For any such value of N they could get a design with as many as (N -

1) factors. The orthogonal designs with N observations in (N - 1) factors are called

Plackett-Burman Design. Hence, the design matrix has N rows and (N - 1) columns. Out of

these (N - 1) columns k columns can be chosen for k factors to fit the orthogonal first

order design.
 Unit 4-Plackett-Burman Designs

Plackett-Burman Design is a saturated design. There are zero residual

degrees of freedom. The absence of any lack-of-fit degrees of

freedom implies no test can be done for lack of fit; that is, there is no

information to test for model inadequacy. In this situation the design

is said to be saturated. In such situations one can utilize the

techniques of NPP or half NPP to find out the significant factors.

 Unit 4-Plackett-Burman Designs

Example 1: the Plackett-Burman design for seven factors can be created using

following set of plus-minus signs. {1, -1, -1, 1, -1, 1, 1} by writing it as a column (or

row). A second column (or row) is then generated from this first one by moving

the elements of the column (or row) down (or to the right) one position and

placing the last element in the first position. A third column (or row) is produced

from the second in the same manner, and the process continues until column (or

row) 7 is generated. A row of minus signs is then added to complete the design.
 Unit 4-Plackett-Burman Designs

Factors

1 1 -1 -1 1 -1 1 1
2 -1 1 1
3 -1 -1
4 1 -1
5 -1 1
6 1 -1
7 1 1
8
Unit 4-Plackett-Burman Designs
Unit 4-Plackett-Burman Designs

In fact, any one of the first seven rows or seven columns can be used to
construct the design matrix as discussed above. The following table
gives us the generator of the design matrix for N =4, 8, 12, 16, 20 and
24.
 Unit 4-Plackett-Burman Designs

The required 11-factor P-B design is given by :

N

1 1 1 -1 1 1 1 -1 -1 -1 1 -1
2 -1 1 1 -1 1 1 1 -1 -1 -1 1
3 1 -1 1 1 -1 1 1 1 -1 -1 -1
4 -1 1 -1 1 1 -1 1 1 1 -1 -1
5 -1 -1 1 -1 1 1 -1 1 1 1 -1
6 -1 -1 -1 1 -1 1 1 -1 1 1 1
7 1 -1 -1 -1 1 -1 1 1 -1 1 1
8 1 1 -1 -1 -1 1 -1 1 1 -1 1
9 1 1 1 -1 -1 -1 1 -1 1 1 -1
10 -1 1 1 1 -1 -1 -1 1 -1 1 1
11 1 -1 1 1 1 -1 -1 -1 1 -1 1
12 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
 Unit4 :Simplex Designs
Another special class of first-order orthogonal designs, that are
saturated, is called the class of simplex designs. The main feature of the
simplex design is that it requires observations to fit a first-order model
in k variables.
Geometrically, the design points represent the vertices of a regular
sided figure. For k = 2, N = 3 the design points are coordinates of an
equilateral triangle. For and , the design points are the vertices of a
tetrahedron. For the design matrix is given by
Unit4 :Simplex Designs
Unit4 :Simplex Designs
Following figure reveals that the design for contains points on the
vertices of an equilateral triangle. The three rows in the design matrix
correspond to points 1, 2, and 3, respectively.
Unit4 :Simplex Designs
Unit4 :Simplex Designs
Unit4 :Simplex Designs
For k = 3, the N = 4 points are the vertices of a tetrahedron.
Unit4 :Simplex Designs
If we use it is clear that .
Unit4 :Method of Steepest Ascent/Descent
Usually, the initial estimate of the optimum operating conditions for the
system is far from the actual optimum. In such circumstances, the
objective of the experimenter is to move rapidly to the general vicinity
of the optimum. We wish to use a simple and economically efficient
experimental procedure. When we are remote from the optimum, we
usually assume that a first-order model is an adequate approximation
to the true surface in a small region of the x’s.
The method of steepest ascent is a procedure for moving sequentially
in the direction of the maximum increase in the response. If
minimization is desired then we call this technique the method of
steepest descent. The fitted first-order model is
Unit4 :Method of Steepest Ascent/Descent
and the first-order response surface, that is, the contours of , is a series
of parallel lines such as shown in figure below.
Unit4 :Method of Steepest Ascent/Descent
The direction of steepest ascent is the direction in which response increases most

rapidly. This direction is normal to the fitted response surface. The experiment is

conducted and the response is observed along the path of steepest ascent until
no

further increase in response is seen. Then a new first order model may be fit, a
new

path of steepest ascent determined and the procedure continues.

Unit4 :Method of Steepest Ascent/Descent
Unit4 :Method of Steepest Ascent/Descent
Example 4. In example considered earlier , we found that x1 has the
largest regression coefficient. So, we select reaction time as a variable
to fix the step size. Five minutes of reaction time is the step size based
on process knowledge. In terms of the coded variables Δx1 = 1.0 and
hence, the step size in temperature is
Unit4 :Method of Steepest Ascent/Descent
Unit4 :Method of Steepest Ascent/Descent
Unit4 :Second-Order Response Surface

Second-Order Response Surface

Covered in Assignment!
Unit4 :The Class of Central Composite Designs
The CCD is the most popular class of second-order designs. Generally,
the CCD consist of a factorial design or fraction of it with factorial
points, axial points and centre points. The sequential nature of the
design becomes very obvious as after running the factorial design, one
can add the centre points and then the axial points. The centre runs
clearly provide information about the existence of curvature in the
system. If curvature is found in the system, the addition of axial points
allows for the efficient estimation of the pure quadratic terms.
The axial points are equidistant from the centre of the design. For a
design involving factors the axial points are
Unit4 :The Class of Central Composite Designs
Unit4 :The Class of Central Composite Designs

The areas of flexibility in the use of the central composite design reside
in the selection of α, the axial distance, and , the number of centre
runs. The choice of these two parameters can be very important. The
choice of α depends to a great extent on the region of operability and
region of interest. The value of the axial distance generally varies from
1 to , the former placing all axial points on the face of the cube or
hypercube, the latter resulting in all points being placed on a common
sphere. There are times when two or more centre runs are needed and
times when one or two will suffice.
Unit4 :The Class of Central Composite Designs
The following figure shows the CCD for . For the case the value of , the
axial distance, is .
Unit4 :The Class of Central Composite Designs
The following figure shows the CCD for . For the case the value of , the
axial distance is .
Unit4 :Rotatability

The guidelines regarding the number of centre points says that spherical
or nearly spherical designs require three to five centre runs.
Rotatability
It is important for the second-order model to provide good predictions
throughout the region of interest. One way to define “good” is to require
that the model should have a reasonably consistent and stable variance
of the predicted response at points of interest x. We know that . If a
second order response surface design is rotatable the is same at all
points x that are at the same distance from the design centre. The CCD is
rotatable if α =
. If the region is spherical it is already rotatable.
 Unit4 :The Box–Behnken Design

The Box–Behnken Design

Box and Behnken (1960) developed a family of efficient three-level
designs for fitting second-order response surfaces. The class of designs
is based on the construction of balanced incomplete block designs. For
example, a balanced incomplete block design with three treatments
and three blocks is given by
 Unit4 : The Box–Behnken Design

The pairing together of treatments 1 and 2 in block 1 symbolically

implies, in the response surface setting, that design variables and are
paired together in a factorial (scaling ±1) while remains fixed at the
center (). The same applies for blocks 2 and 3, with a factorial being
represented by each pair of treatments while the absent treatment
remains fixed at 0. As a result, the k = 3 Box –Behnken design is given
by
Unit 4 : The Box–Behnken Design
 Unit 4 : The Box–Behnken Design

The last row in the design matrix implies a vector of center runs. It can
be written alternatively as

In the case of the same methodology applies. Each pair of factors are
linked in a factorial. The design matrix is given by
Unit 4 : The Box–Behnken Design
 Unit 4 : The Box–Behnken Design
This design matrix can be written concisely as
 Unit 4 : The Box–Behnken Design

Note that the Box –Behnken design (BBD) is quite comparable in

number of design points to the CCD for k = 3 and k = 4. (There is no BBD
for k = 2.) For k = 3, the CCD contains 14 + nc runs while the BBD
contains 12 + nc runs. For k = 4 the CCD and BBD both contain 24 + nc
design points.
 Unit 4 : The Box–Behnken Design

In many scientific studies that require RSM, researchers are inclined to

require three evenly spaced levels. Thus, the Box – Behnken design is
an efficient option and indeed an important alternative to the central
composite design. The Box – Behnken design does not substantially
deviate from rotatability. Another important characteristic of the BBD is
that it is a spherical design.
 Unit 4 : The Box–Behnken Design

In many scientific studies that require RSM, researchers are inclined to

require three evenly spaced levels. Thus, the Box – Behnken design is
an efficient option and indeed an important alternative to the central
composite design. The Box – Behnken design does not substantially
deviate from rotatability. Another important characteristic of the BBD is
that it is a spherical design.
 Unit 4 : The Box–Behnken Design

In many scientific studies that require RSM, researchers are inclined to

require three evenly spaced levels. Thus, the Box – Behnken design is
an efficient option and indeed an important alternative to the central
composite design. The Box – Behnken design does not substantially
deviate from rotatability. Another important characteristic of the BBD is
that it is a spherical design.
Unit 4 : Stationary Point
Unit 4 : Stationary Point
Unit 4 : Stationary Point
 Design of Experiments

END OF SYLLABUS and CLASS ROOM TEACHING