A collection of mathematical and statistical techniques useful for the

modelling and analysis of problems in which a response of interest is

influenced by several variables and the objective is to optimize (minimize
or maximize) this response.
 = 1 , 2 +
 = 1 , 2 +
y = dependent variable
 = 1 , 2 +
y = dependent variable
x1, x2 = independent variables
 = 1 , 2 +
y = dependent variable
x1, x2 = independent variables
= error term
Response Surface

= 1 , 2 =
Response Surface

= 1 , 2 =
response surface
Figure 1. A three-dimensional response surface showing the expected
yield as a function of temperature and pressure
Schneider and Stockett (1963) performed an experiment aimed at
reducing the unpleasant odor of a chemical product with several factors.
The objective is to minimize the unpleasant odor of a chemical. The data
are imported into SAS as follows:
title 'Response Surface with a Simple Optimum';
data smell;
input Odor T R H;
label T = "Temperature"
R = "Gas-Liquid Ratio"
H = "Packing Height";
datalines;
66 40 0.3 4
58 40 0.5 2
65 80 0.3 2
-31 80 0.5 4
39 120 0.3 4
17 120 0.5 2
7 80 0.7 2
-35 80 0.5 4
43 40 0.7 4
-5 40 0.5 6
43 80 0.3 6
-26 80 0.5 4
49 120 0.7 4
-40 120 0.5 6
-22 80 0.7 6
;
title 'Response Surface with a Simple Optimum';
data smell;
input Odor T R H;
label T = "Temperature"
R = "Gas-Liquid Ratio"
H = "Packing Height";
datalines;
66 40 0.3 4
58 40 0.5 2
65 80 0.3 2
-31 80 0.5 4
39 120 0.3 4
proc rsreg data = smell;
17 120 0.5 2 model Odor = T R H / lackfit;
7 80 0.7 2 run;
-35 80 0.5 4
43 40 0.7 4
-5 40 0.5 6
43 80 0.3 6
-26 80 0.5 4
49 120 0.7 4
-40 120 0.5 6
-22 80 0.7 6
;
Figure 2.1 Summary Statistics and Analysis of Variance
Figure 2.2 Parameter Estimates and Hypothesis Tests
Figure 2.3 Canonical Analysis and Eigenvectors
data grid;
do;
Odor = . ;
H = 7.541;
do T = 20 to 140 by 5;
do R = .1 to .9 by .05;
output;
end;
end;
end;
data grid;
set smell grid;
run;

proc rsreg data = grid out = predict noprint;

model Odor = T R H / predict;
run;

data plot;
set predict;
if H = 7.541;
run;

proc g3d data = plot;

plot T*R = Odor / rotate=38 tilt=75 xticknum=3 yticknum=3
zmax=300 zmin=-60 ctop=red cbottom=blue caxis=black;
run;
 Figure 2.4
The Response Surface Obtained from the PREDICT Option


= 0 + 2 + +
=1 <

Where
y = dependent variable (response)
= coefficients (constants)
= independent variables
= observed error
 The stationary point could represent a point of maximum response, a point of
minimum response, or a saddle point.
To find the stationary point:
1 1
=
2
Where
1 1 ,
/ /

2 2 and B =
x= b=
/


.


 The predicted response at the stationary point

1

= 0 +
2

Where
= predicted response
= stationary point
= vector of coefficients
 Once the stationary point has been found, it is usually necessary to
characterize the response surface in the immediate vicinity of this
point. The most straightforward way to do this is to examine a
contour plot of the fitted model.

However, even when there are relatively few variables, a more formal
analysis, called the canonical analysis, can be useful.

2 2 2 Where
= + 1 1 + 2 2 + + wi = transformed independent variables
i = constants
 Stationary ridge system
 Rising ridge
 Many response surface problems involve the analysis of several
responses.
An extensive treatment of the multiple response problem is given in
Myers, Montgomery and Anderson-Cook (2009).
A relatively straightforward approach to optimizing several responses
that works well when there are only a few process variables is to
overlay the contour plots for each response.
Example 75.1: A Saddle Surface Response Using Ridge Analysis Myers
(1976) analyzes an experiment reported by Frankel (1961) aimed at
maximizing the yield of mercaptobenzothiazole (MBT) by varying
processing time and temperature. Myers (1976) uses a two-factor
model in which the estimated surface does not have a unique
optimum. A ridge analysis is used to determine the region in which the
optimum lies. The objective is to find the settings of time and
temperature in the processing of a chemical that maximize the yield.
data d; 6.3 271 83.1
input Time Temp MBT; 17.7 229 85.3
label Time = "Reaction Time (Hours)" 17.7 271 72.7
Temp = "Temperature (Degrees Centigrade)" 4.0 250 82.0
MBT = "Percent Yield Mercaptobenzothiazole"; ;
datalines; ods graphics on;
4.0 250 83.8 proc rsreg data=d plots=(ridge surface);
20.0 250 81.7 model MBT=Time Temp / lackfit;
12.0 250 82.4 ridge max;
12.0 250 82.9 run;
12.0 220 84.7 ods graphics off;
12.0 280 57.9
12.0 250 81.2
6.3 229 81.3
Figure 2.5 Summary Statistics and Analysis of Variance
 Based from the results, the cross-product terms are not significant.
However, the lack-of-fit for the model is significant, so more
complicated modelling or further experimentation with additional
variables should be performed before firm statements are made
concerning the underlying process.
Figure 2.6 Parameter Estimates and Hypothesis Tests
 Based from the results obtaining the parameter estimates, you can
see that the cross-product terms are not significantly different from
zero, as noted previously.
Factor Temp is significant, indicating that the level of yield left
unexplained by the model is still too high to estimate the effect of
factor Time. This may be due to the lack of fit.
Figure 2.7 Canonical Analysis and Eigenvectors
 Based from the results, the larger eigenvalue (2.528816) corresponds
to the eigenvector {0.953223, -0.302267}, the largest component of
which (0.953223) is associated with the factor Time; similarly, the
smaller eigenvalue (-9.996940) is associated with Temp. The coded
form of the canonical analysis indicates that the estimated response
surface is at minimum when Time is near the middle of its respective
range and Temp is relatively high; in uncoded terms, the model
predicts that the yield will be minimized when Time = 8.455935, and
Temp=240.700718.
Figure 2.8 Response Contour for MBT with Design Points
1. Provides a reasonable distribution of data points throughout the region of
interest
2. Allows model adequacy, including lack of fit, to be investigated
3. Allows experiments to be performed in blocks
4. Allows designs of higher order to be built up sequentially
5. Provides an internal estimate of error
6. Provides precise estimates of the model coefficients
7. Provides a good profile of the prediction variance throughout the
experimental region
8. Provides reasonable robustness against outliers or missing values
9. Does not require a large number of runs
10. Does not require too many levels of the independent variables
11. Ensures simplicity of calculation of the model parameters
FITTING FIRST-ORDER DESIGNS
 Minimizes the variance of the regression
coefficients {i}
Off-diagonal elements of the (XX) matrix
are all zero
o Cross-products of the columns of the X
matrix sum to zero
 Low and high levels of the k factors are coded to the
usual 1 levels
Does not afford an estimate of the experimental error
Replication:
Augment the design with several observations at the center
(the point xi = 0, i= 1,2,...,k)
Center points do not influence {i} for i1, but estimate for
0 becomes the grand average of all observations
Does not alter the orthogonal property of the design
 Regularly sided figure with k+1 vertices in k dimensions

Figure 3. Simplex Design for (a) k=2 and (b) k=3

FITTING SECOND-ORDER DESIGNS
 Consists of a 2k factorial (or fractional factorial of resolution V) with nF
factorial runs, 2k axial or star runs, and nC center runs
Sequential experimentation
o A 2k has been used to fit a first-order model, this model has exhibited lack of
fit, and the axial runs are then added to allow the quadratic terms to be
incorporated into the model.
2 parameters: the distance of the axial runs from the design center
and the number of center points nC
 Figure 4. Central Composite Designs for k=2 and k=3
 Eliminate nuisance variables
block effects do not affect the parameter estimates of the model
 Each block must be a first-order orthogonal design; that is,

= 0 = 0, 1, ,
=1

where and are the levels of ith and jth variables in the uth run
of the experiment with 0 = 1 for all u.
 The fraction of the total sum of squares for each variable contributed by
every block must be equal to the fraction of the total observations that
occur in the block; that is,

=1 2
2 = i = 1, 2, , k for all b
=1

where N is the number of runs in the design.

 1 2
Consider a rotatable central 1 1
1 1
composite design in k=2 variables 1 1
with N =12 runs. We may write the 1 Block 1
1
levels of x1 and x2 for this design in 0 0
the design matrix = 0 0
1.414 0
1.414 0
0 1.414 Block 2
0 1.414
0 0
0 0
To test condition two, consider first block 1 and note that
1 1

1 2 = 2 2 = 4
=1 =1

1 2 = 2 2 = 8 1=6
=1 =1

Therefore

=1
1
2 1 4 6
2
= =
=1 8 12
For block 2,
2 2

1 2 = 2 2 = 4 2 = 6
=1 =1

Therefore

=1
2
2 2 4 6
2
= =
=1 8 12
 Can always be constructed to block orthogonally in two blocks
first block consisting of factorial points plus center points
second block consisting of = 2 axial points plus center points

First condition for orthogonal blocking will always hold regardless of

the value used for in the design
For the second condition to hold,
2
2 +
=
1

2 +
 2
2 +
=
1

2 +

12
+
=
2 +

Using the equation above, the equation for the value of can be
obtained
 If the design is also required to be rotatable, then
1/4
=

12
+
=
2 +
 The use of center points to calculate an estimate of pure error
The block effect
Sum of squares (orthogonal blocking)

2 2
=

=1

where B is the total of the n observations in the bth block and G is the
b b

grand total of all N observations in all m blocks

There are times when an experimenter
encounters a situation where a standard
response surface design may not be the
practical choice. In such a case, optimal designs
are an alternative to consider.
The following are several situations where some type of
computer-generated design may be appropriate:
An irregular experimental design
Non-standard model
Unusual sample size requirements
 D-optimality
A-optimality
G-optimality
V
I or IV or Q
A design is said to be D-optimal if
| |
is minimized. A D-optimal design brings to its minimum range the joint
confidence region on the vector of regression coefficients. A measure
of the relative efficiency of design 1 to design 2 according to the D-
criterion is
/
| |
=
| |
where and are the X matrices for the two designs and p is the
number of model parameters.
A design is A-optimal if it minimizes the sum of
the main diagonal elements of | | - this
is also known as the trace of | | .
Therefore, an A-optimal design minimizes the
sum of the variances of the regression
coefficients.
A design is considered G-optimal if it minimizes the maximum scaled
prediction variance over the design region; in other words, if the
maximum value of
[ ]
2
over the design region is minimum, where N is the number of points in
the design. If the model has p parameters, the G-efficiency of a design
is just

=
[ ]

2
A design that minimizes the average prediction
variance over this set of m points is a V-optimal
design.
An alternative to calculating the prediction variance at a finite set of
points in the design space is to compute an average or integrated
variance over the design space, say

= [ ]

where R is the design region and A is the volume of the region. Note
that this is a more general form of the I-criterion discussed on the
previous chapters. The I-criterion is also sometimes referred to as the
IV or Q-criterion.
 Point Exchange Algorithm
Coordinate Exchange Algorithm
In the simplest form of this algorithm, an initial
design is selected (conceivably by random)
from a grid of candidate points chosen by the
experimenter. Then the algorithm exchanges
points that are in the grid but were not
selected to be in the design with points
currently in the design in order to make an
improvement for the selected optimality
criterion.
This process tends to search over each
coordinate of every point in the initial design
recursively until no need for improvement in the
optimality criterion is found. The procedure is
usually repeated a number of times with each
cycle starting with a randomly generated initial
design.
 http://www.itl.nist.gov/div898/handbook/pri/section3/pri336.htm
Montgomery, D.C. (2013). Designs and analysis of experiments (8th ed.).
New York: John Wiley