See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/228934887

A New Variable-Fidelity Optimization Framework Based on Model Fusion and

Objective-Oriented Sequential Sampling

Article  in  Journal of Mechanical Design · November 2008

DOI: 10.1115/1.2976449

CITATIONS READS

77 232

3 authors:

Ying Xiong Wei Chen

Northwestern University Northwestern University
13 PUBLICATIONS   560 CITATIONS    437 PUBLICATIONS   13,925 CITATIONS   

SEE PROFILE SEE PROFILE

Kwok-Leung Tsui
City University of Hong Kong
284 PUBLICATIONS   8,538 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Uncertainty Quantification View project

General Research Fund: Reliability and Degradation Modeling for Rechargeable Battery View project

All content following this page was uploaded by Ying Xiong on 20 May 2014.

The user has requested enhancement of the downloaded file.


ASME 2007 International Design Engineering Technical Conferences & Computers and Information in
September 4-7, 2007, Las Vegas, Nevada, USA
A NEW VARIABLE FIDELITY OPTIMIZATION FRAMEWORK BASED ON MODEL FUSION AND
OBJECTIVE-ORIENTED SEQUENTIAL SAMPLING
Ying Xiong, Wei Chen1
Department of Mechanical Engineering
Northwestern University
ABSTRACT
Computational models with variable fidelity have been widely
used in engineering design. To alleviate the computational
burden, surrogate models are used for optimization without
recourse to expensive high-fidelity simulations. In this work, a
model fusion technique based on Bayesian Gaussian process
modeling is employed to construct cheap, surrogate models to
integrate information from both low-fidelity and high-fidelity
models, while the interpolation uncertainty of the surrogate
model due to the lack of sufficient high-fidelity simulations is
quantified. In contrast to space filling, the sequential sampling
of a high-fidelity simulation model in our proposed framework
is objective-oriented, aiming for improving a design objective.
Strategy based on periodical switching criteria is studied which
is shown to be effective in guiding the sequential sampling of a
high-fidelity model towards improving a design objective as
well as reducing the interpolation uncertainty. A design
confidence (DC) metric is proposed to serves as the stopping
criterion to facilitate design decision making against the
interpolation uncertainty. Numerical and engineering examples
are provided to demonstrate the benefits of the proposed
methodology.
1
KEYWORDS
Variable fidelity, Optimization, Sequential sampling, Surrogate
model, Bayesian approach, Model fusion, Design confidence
1. INTRODUCTION
Variable fidelity methods have been developed to solve
optimization problems that involve simulations with extreme
computational expenses [1-3]. Examples of variable fidelity
models in aircraft aerodynamic analysis include low fidelity
(LF) models, such as the classical aerodynamics or linearized
supersonic panel code, and high fidelity (HF) simulations, such
as the finite element based Euler/Navier-Stokes solver. In
1
Corresponding author. Department of Mechanical Engineering,
Northwestern University, 2145 Sheridan Road, Tech B224, Evanston, IL
60208-3111, Email: weichen@northwestern.edu, Phone: (847) 491-7019, Fax:
(847) 491-3915.
Proceedings of IDETC/CIE 2007
Engineering Conference
DETC2007-35782
Kwok-Leung Tsui
School of Industrial & Systems Engineering
Georgia Institute of Technology
multiscale material design [4,5], models based on direct
numerical simulations become prohibitively expensive due to
the vast scale difference among the lowest (atomic) to the
highest (macroscopic) scale. While model reduction approaches
are available, there is a need for variable fidelity methods that
provide seamless integration between high and low fidelity
simulations. A critical issue in variable fidelity optimization is
to effectively use limited HF simulations to improve the
predictions and optimization solutions from using LF models.
In the conventional surrogate model based optimization, a
surrogate model (also called metamodel) is built based purely
on the HF simulations, then followed by optimization using the
surrogate. In the context of variable fidelity optimization, the
amount of HF simulation data is typically far from enough.
Consequently, the optimal solution might not be trustworthy.
To overcome this limitation, model management strategies,
e.g., trust-region or moving limit based approaches [1] [3],
which construct an approximation to an HF model by
incorporating the information from a LF model, have been
developed to guarantee the local convergence with less
computational expense on HF simulation. Such an
approximation could be either a local approximation, e.g.,
Taylor series and polynomial based response surface approach,
or a global approximation, e.g., Kriging modeling approach [6].
With the global approximation, all historical sampling points
are used to construct the surrogate model. One advantage of the
trust-region based variable fidelity optimization framework is
that the convergence of a local minimum using the surrogate
model is theoretically guaranteed [1]. However, the trust-region
based approach may fail to find the global optimum if multiple
local optimums exist.
In this work, a new variable fidelity optimization approach is
proposed based on the Bayesian surrogate modeling for model
fusion and the objective-oriented sequential sampling. There
are two fundamental differences between our method and some
traditional trust-region based methods. First, when using the
trust-region based variable fidelity optimization, the
optimization search is over the surrogate approximation that is
1 Copyright © 2007 by ASME
constrained within a so-called trust-region, the size of which is
adjusted depending upon the accuracy of the local
approximation in the previous iteration. In contrast, with our
approach, a global optimization is performed over the global
surrogate model that is updated using new sampling points
from the HF model. Second, for the trust-region based variable
fidelity optimization, the sampling point is identified at the site
of the optimum of the local approximation, whereas in our
approach the new sampling site is identified by the objective-
oriented sampling criteria.
How to best integrate LF and HF models in constructing a
surrogate model is one of the key issues in variable fidelity
optimization. Bayesian style modeling approaches have been
developed (e.g., Qian et al. [7]). A similar mathematical
framework has been followed to combine the results from
physical and computer experiments in our earlier work on
model validation [8]. We demonstrate in [9] that the combined
models always have a better accuracy than using the surrogates
of either computer or physical experiments alone. Similar
conclusion can be derived for combining results from HF and
LF simulations. Associated with surrogate model is the need for
quantifying the interpolation uncertainty, also known as
prediction uncertainty [10]. Interpolation uncertainty, which is
used to account for the lack of data and knowledge in
constructing a surrogate model, plays an important role in
sequential sampling, with more details provided in the next
paragraph. Extensive efforts have been made to quantify such
uncertainty. Among them, the Gaussian process (GP) models
(Kriging model as a special case) are widely used. Apley et al.
[10] developed an analytical approach to quantifying the impact
of interpolation uncertainty on a robust design objective. In our
proposed approach, the constrained linear scaling method is
first applied to match the LF model to the HF data, followed by
using a Bayesian Gaussian process to account for the remaining
discrepancy and to quantify the interpolation uncertainty.
The other important issue associated with a surrogate model
based design optimization is the sequential updating of the
surrogate model with additional data, or called sequential
sampling. One category of existing sequential sampling
techniques aims at global accuracy in a space-filling fashion,
e.g., based on criteria such as the Integrated Mean-Squared
Error, MaxiMin/MiniMax, and Maximum Entropy. Recent
studies [11-16] have proposed adaptive strategies based on the
knowledge obtained from the preceding surrogate model.
However, it is generally unaffordable to conduct enough
simulation runs to cover the entire input space when
simulations are expensive. A more efficient sequential
sampling strategy is the so-called objective-oriented approach
that brings the design objective into account. The initial
utilizations of the objective-oriented approach and the concept
of interpolation uncertainty could be seen in the Efficient
Global Optimization (EGO) algorithm developed by Jones et al.
[17] using the idea of expected improvement (EI) originally
proposed by Mockus et al. [18]. Although EI was demonstrated
to be well suited for the global deterministic optimization,
alternative sequential sampling criteria [19] [13] were
investigated and shown to have various merits in making the
trade-off between optimizing a design objective and reducing
the interpolation uncertainty. All of these existing objective-
oriented approaches were applied to surrogate models built
only using HF simulations. In this work, we extend the same
principle to surrogate models built based on both LF and HF
simulations. We also investigate a new periodical switching
criterion to effectively guide the sequential high-fidelity
simulations tailored towards both improving a design objective
as well as reducing interpolation uncertainty. Another
important feature of the proposed sequential procedure is the
use of the design confidence (DC) as a metric to assist
designers in making the decision regarding when to terminate
the sampling process so that the current optimum design can be
accepted as the ‘true optimal’ solution with sufficient
confidence.
2. PROPOSED METHODLOGY
2.1 The Proposed Variable Fidelity Optimization
Framework
Low fidelity model yl(x) High fidelity model yh(x)
Surrogate of yl(x) Sampling Objective-
oriented
sequential
Surrogate model ys(x) with interpolation
uncertainty quantification sampling
Optimization on yˆ s (x)
Optimum design x*
No
Meet design confidence?
Yes
Accept optimal design x*
Figure 1. Proposed Variable Fidelity Optimization Framework
The proposed variable-fidelity optimization framework in this
work is depicted in Figure 1. In contrast to the classical variable
fidelity optimization strategy, none of the LF and HF models is
directly invoked during optimization. Instead, a model fusion
technique is applied to combine information from both LF
model yl(x) and HF simulations yh(xi) (i=1,…,N) to yield a
surrogate model ys(x), over which optimization is performed.
The proposed model fusion approach follows a Bayesian
modeling framework, in which the surrogate model is a
combination of an augmented LF model with linear scaling plus
a bias function that characterizes the remaining difference with
the HF model. With the Bayesian approach, the uncertainty of
ys(x) in predicting HF models can be quantified. As previously
stated, such type of uncertainty is called the ‘interpolation
uncertainty’ due to the lack of sufficient HF simulations. When
a LF model is cheap to run, it will be directly used for model
fusion without fitting a surrogate model to replace it. When a
LF simulation is expensive, the surrogate of LF model (dashed
line box in Figure 1) could be used to replace the original LF
model. After the surrogate model ys(x) is obtained based on
model fusion, design optimization is performed using the
predictor (or posterior mean) of the Bayesian surrogate model
ys(x). The design confidence (DC) of the newly identified
optimum x* will then be assessed considering the interpolation
2 Copyright © 2007 by ASME
uncertainty associated with ys(x). If the design confidence
meets a satisfactory level, or when the computing resource has
been exhausted, the sequential sampling process is terminated.
Otherwise, an objective-oriented sequential sampling procedure
will be applied to pick new samples of HF simulations. In the
proposed framework, only one sampling point of HF simulation
is added at each iteration (or called ‘stage’).
2.2 Model Fusion and Quantification of Interpolation
Uncertainty
The goal of model fusion is to integrate the information from
both LF and HF models to build a surrogate for predicting the
design behavior. Existing model fusion techniques include, but
are not limited to, the difference or scaling approach [3], the
Taylor-series approach [6], and the space mapping approach
[20], etc. Due to the lack of expensive HF simulation data in
most practical problems, a Bayesian approach to model fusion
and quantification of interpolation uncertainty is considered in
this work. Among the existing Bayesian modeling approaches,
the Gaussian process (GP) models (e.g., Kriging as a special
case) have gained popularity due to their convenience and
flexibility in interpolating deterministic computer data. With
Gaussian process models, the mean and covariance functions
are specified to reflect prior knowledge about the unknown
function. The approach to combining LF and HF simulations
described bellow follows the similar mathematical framework
as combining computer and physical experiments in our earlier
work on model validation [8].
In this work, the mathematical relationship between the HF
model yh(x) and the LF model yl(x) is represented as
y h (x) = ysl (x) + δ (x) = [ ρ0 + ρ1 y l (x)] + δ (x) , (1)
where ρ0 and ρ1 are two scaling parameters that define the
‘scaled’ LF model ysl (x) = ρ0 + ρ1 y l (x) ; δ (⋅) is a bias
function used to account for the discrepancy between the
‘scaled’ LF model ysl (x) and the HF model yh(x). yl(x) is
simply regarded as either the original LF model or its surrogate
model yˆ l (x) .
The scaling parameters ρ0 and ρ1 help bring (or ‘scale’) the LF
model as close as possible to the HF model. The application of
similar linear scaling approach could be found in previous
works, e.g., [3] and [7], even though the details of Bayesian
modeling and its complexity vary. For determining the
Bayesian modeling parameters, different approaches (e.g.,
MLE estimation [7], Cross Validation [21]) could be employed.
In this work, ρ̂0 and ρ̂1 are identified by the least square
(LS) method, together with bounds constrained on both ρ̂0
and ρ̂1 in Eq. (2):
∑
N l h 2
min L( ρ0 , ρ1 ) = [ ρ + ρ1 y (xi ) − y (xi )] , (2)
i =1 0
st. l0 < ρ0 < u0 ; l1 < ρ1 < u1
where L (ρ0, ρ1) stands for the loss function in a square sense;
xi(i=1…N) are the sampling points of HF model. The bounds
(l0, u0) and (l1, u1) posed on parameter ρ0 and ρ1 reflect,
respectively, the prior belief of the global constant bias and
multiplicative scaling between HF and LF models. Due to the
use of bounds, we call our method the constrained linear
scaling (CLS) approach. Note that if the bound constraints on
ρ0 and ρ1 are both ignored, the solution to Eq. (2) will be
exactly the same as that from a regular regression.
To simplify the Bayesian modeling, the scaling parameters ρ0
and ρ1 are assumed to be unknown but fixed in this work. This
treatment is different from [7], where a linear function ρ(x) is
used for scaling instead of two constant parameters ρ0 and ρ1
used in our work. It is our belief that using ρ1will help better
preserve the ‘profile’ of a LF model and the term ρ0 will help
satisfy the assumption of ‘zero-mean’ prior associated with the
bias function δ(·), especially if a global bias exist between LF
and HF models.
The bias function δ(·) accounts for the remaining discrepancy
between the HF simulation data and the scaled LF model. For
modeling the bias function, it is assumed that δ(·) follows a
Gaussian process (GP) with mean function βT F(x) , and a
covariance function C(σ, θ), where σ and θ denote the variance
and correlation parameters, respectively. The sampling points
for δ(·) are evaluated by
δ (xi ) = y h (xi ) − yˆ sl (x) = y h (xi ) − [ ρˆ 0 + ρˆ y l (xi )] . (3)
It is observed that when data are far from sufficient to explore
the behavior of the true HF model performance, a common
Kriging approach [11] [7] would typically underestimate the
true interpolation uncertainty [22], because all Gaussian
process parameters (β, σ2, θ) are treated as unknown but fixed
and are determined through methods like the maximum
likelihood estimation (MLE) [23] and the cross validation (CV)
[21]. With the Bayesian approach, the prior knowledge of the
unknown bias function δ(·) can be expressed through the prior
distribution of (β, σ2, θ). The posterior distributions of these
parameters are quantified and propagated through the predictor.
In this work, a semi-Bayesian analysis is implemented while
treating the correlation parameters θ unknown but fixed. With
such a treatment, by imposing appropriate priors, we can derive
the close forms of the posterior process of the Gaussian process
model [8]. The hierarchical Bayesian modeling approach [24]
[25] is followed to choose the prior distributions of σ2 and β,
for example,
σ 2 ∼ IG (2,1), β | σ 2 ∼ N (0, σ 2 I ), (4)
where IG (⋅, ⋅) is the inverse Gamma distribution, N(·) is the
normal distribution. Note such prior specifications can be used
if no previous knowledge is available. Because the scaled LF
model ysl (x) has been pulled as close as possible to the HF
simulation data, it is reasonable to assume the prior mean for
δ(·) as a zero (i.e., βT F (x) =0) , thus β is centered on zero in
Eq. (4). Since we pose no uncertainty on ρ̂ , the resulting scaled
LF model ρˆ y l (x) is viewed as deterministic. As a result, the
surrogate model yˆ s (⋅) could be represented as the predictor or
the (posterior) mean
yˆ s (x) = [ ρˆ 0 + ρˆ y l (x)] + δˆ (x) . (5)
The interpolation uncertainty is only contributed by the
interpolation uncertainty of the bias function δ(·), i.e.,
Var[ y s (⋅)] = Var[δ (⋅)] , (6)
3 Copyright © 2007 by ASME
 Cov[ y s (x), y s (x′)] = Cov[δ (x), δ (x′)] , (7)
where Var[·] and Cov[·,·] denote the variance and covariance,
respectively. The mathematical close forms of the mean,
variance, and covariance of the posterior process δ(·) are not
provided here but can be found in [8].
2.3 Objective-Oriented Sequential Sampling
Sequential sampling is one critical step in the variable fidelity
optimization framework. Design decisions made via the
predictor (mean) of the Bayesian surrogate model can be
improved by reducing the interpolation uncertainty through the
inclusion of more data. The concept of Expected Improvement
(EI) was first applied in the Efficient Global Optimization
(EGO) algorithm proposed by Jones et al. [17]. The algorithm
selects the next sampling site to maximize the expected
improvement of a design objective function with respect to the
interpolation uncertainty. The EI function is formulated as
E[max{0,[ ymin − y s (x N +1 )]}] and further expanded as
ymin − µ y ymin − µ y
EI (x N +1 ) = [ ymin − µ y ]Φ ( ) + σ yφ (
σy σy
where ymin = yˆ (xi *) , xi* is the best design point based on the
current metamodel, y s (x N +1 ) stands for prediction (mean)
from surrogate model at site x N +1 , µ y = yˆ s (x N +1 ) and
σ y = Var1/ 2 [ y s (x N +1 )] . According to the study done in Sasena
[19] and Sekishiro et al. [27], although the EI method is
intended to balance between the local search and the global
search, it may not converge in some cases. Sometimes, it is
difficult to optimize the EI function because it may show
extremely ‘bumpy’ behavior at existing sampling points and
extremely ‘flat’ (close to zero) behavior in regions that are
largely inferior to the optimum.
In this work, we examine the use of the Statistical Lower
Bounding (SLB) criterion [26,28], which has the following
form:
Minimize SLB(x N +1 ) ≡ µ y s (x N +1 ) − kσ y s (x N +1 ) , (9)
s 1/ 2
where µ y s (x N +1 ) = yˆ (x N +1 ) , σ y (x N +1 ) = Var [ y (x N +1 )] ,
and k is a user-defined parameter. A larger k value implies the
emphasis on reducing interpolation uncertainty or the need for
global search. The SLB criterion is easier to be controlled and
interpreted than the EI criterion. By assigning k =0 and k = +∞ ,
respectively, we get two extreme values of the SLB criterion:
Maximizing Var (x N +1 ) ≡ σ 2 y s (x N +1 ) (‘Max Var’), (10)
Minimizing Mean(x N +1 ) ≡ µ y s (x N +1 ) (‘Max Mean’). (11)
‘Max Var’ focuses on the aspect of exploring regions with large
interpolation uncertainty; ‘Min Mean’ focuses on the aspect of
locating local optima. In the work of Sasena [19], it was
proposed to alternate the two criteria under a subjective
guideline. In the method described by Sekishiro et al. [27], two
samples, one based on ‘Max Var’ and the other on ‘Min Mean’,
are used at each stage. One possible drawback common to these
strategies is that consecutive sampling points may pile up at
extremes, causing ill-conditioning when calculating the inverse
of the covariance matrix used in Gaussian process modeling.
To overcome these difficulties, we propose a periodical
switching criteria (PSC) strategy as depicted in Figure 2. The
sequence of the periodical alternation of criteria begins with an
extreme global search ‘Max Var’, and ends up with an extreme
local search ‘Min Mean’, while using in-between three
compromising searches, i.e., ‘Min SLB’ with k=5.0, 3.0, and
1.0. By applying the compromise strategy with ‘Min SLB’ in
the middle, sampling points are far less likely to cluster around
the local minimum. This strategy is applied repeatedly
throughout the whole sequential sampling procedure. Our
proposed strategy is good at exploring both the local and global
regions in a systematic manner, as demonstrated through
examples in later sections.
Start Max Var
Min SLB (k=5.0)
Min SLB (k=3.0)
) , (8)
Min SLB (k=1.0)
Min Mean
Figure 2. The proposed periodical switching
criteria (PSC) strategy
2.4 Design Confidence and Stopping Criteria
Design confidence (DC), proposed in Chen et al. [8] for model
validation, is defined as a probabilistic measure [29] of a
chosen optimal design being better than other design choices
with the consideration of model uncertainty. In the proposed
variable fidelity optimization framework, design confidence
DC (x*) is defined as the probability of whether an optimal
design x* is the true optimum, in comparison with all designs
outside the indifferentiable region X0, considering the
interpolation uncertainty of using a surrogate model y s (x) , i.e.,
x∉ X
{ }
DC (x*) = min0 P[ y s (x*) < y s (x)] = min0 { P[ Z x* (x) > 0]} , (12)
x∉ X
s where x* is identified by optimizing the predictor or mean of
the surrogate model y s (x) , i.e,
x
{
x* = arg min yˆ s (x) .} (13)
Z x* (x) = y s (x) − y s (x*) is assumed to follow the Gaussian
distribution with mean
µ Z x* (x) = yˆ s (x) − yˆ s (x*) , (14)
and variance
σ Z2x* (x) = Var[ y s (x)] + Var[ y s (x*)] − Cov[ y s (x), y s (x*)] . (15)
Figure 3 shows an illustrative example of the uncertainty of
ys(x) at three design points, namely, x * , x a , and xb , where
x* is identified as the optimum based on the mean of
prediction. Correspondingly, the mean and variance of Z x* (x)
at the same three points are illustrated in Figure 4, where the
probabilistic distribution of Z x* (xi ) (xi = x*, x a , and xb ) is
calculated from Eqs. (14) and (15). Note that Z x* (x) has zero
mean and zero variance at x*.
4 Copyright © 2007 by ASME
 ys(x)
yˆ s (x*)
x models being artificially created for the illustrative purpose (see
xa x* xb
s
Figure 3. The uncertainty of y (x) at x*, xa, and xb
P{|zx*(x)|<H }
zx*(x) P{ zx*(x)>0 }
H noted that the optimal design (x*LF= 0.9150, marked with *)
µ z x* (x*) = 0 H obtained from the LF model is a sub-optimal solution, located
x 0.2307, marked with star) obtained from the HF model. After
xa x* xb
Figure 4. The uncertainty of zx*(x) at x*, xa, and xb
The assessment of DC in Eq. (12) involves the evaluation of
probability that x* is better than the most competing design x
outside the indifferentiable region X0. The concept of
indifferentiable region X0 is introduced because, with model
uncertainty, designs with performance values close to x* should
be considered as equally good. More strictly, X0 is defined as a
region within which the design points are claimed indifferent to
x* for a given design tolerance H with a confidence level
C X 0 =1- α (e.g. α =0.05 or 0.10), as follows,
{ }.
X 0 ≡ x P [| Z x* (x) |< H ] > C X0
Note that X0 may include not only the close neighborhood of
x*, but also regions not adjacent to x*. The evaluation of Eq.
(12) requires the search of the most competing design w.r.t. the
optimal design x*, called the most competing (MC) point xmc,
by minimizing the probability P and treating the
indifferentiable region X0 as the constraint. The obtained xmc is
then used to calculate the design confidence of x* through
DC (x*) = P[ Z x* (x mc ) > 0] . (17)
For sequential sampling in global optimization, the commonly
used stopping criteria are based on the convergence behavior in
either design space of x, or performance space of y [3] [27].
Although generally applicable, none of them provide
probabilistic measure regarding the validity of an optimal
design considering model uncertainty. In this work, we view
the sequential sampling as a process of reducing the
interpolation uncertainty of surrogate models, as well as
improving the confidence in accepting a design solution. We
propose to use the stopping criterion based on the design
confidence values of any two consecutive stages. If any two
consecutive design confidences DC(x*) meet (is higher than) a
desired confidence level (e.g., 90%, 95%) prescribed by
designers, the sequential sampling process can then be
terminated. Often times, once resources (time and cost, etc) are
exhausted, designers would have to accept the current best
design. With the information of design confidence, designers
are aware of the validity, or goodness, of the achieved design to
decide if the design is acceptable or not.
3. NUMERICAL EXAMPLES
3.1 Example 1: Single Dimensional Problem
A single dimensional problem is first studied, with HF and LF
Appendix for the mathematical equations). At the initial stage
(Stage 0), 3 uniform HF sampling points in region [0 1] are
generated. Figure 5 shows the plots of the true HF model, the
LF model, the scaled LF model, and the 3 HF sampling points
(dark solid circles). Note there are two local minima of the HF
model, and the LF model only roughly captures the general
trend of the HF model but provides a poor approximation. It is
in an area that is quite far from the true optimum (x*HF=
scaling, the scaled LF model are pulled close to the 3 HF data
points, with the scaling parameters estimated as [ ρˆ 0 ρˆ1 ] =
[0.1844 0.5371] based on Eq. (2). It should be noted that the
values of [ ρˆ 0 ρˆ1 ] change over different stages after more
samples are added.
Stage (0)
HF
1
LF
Scaled LF
HF samples
HF optimum
0.5 LF optimum
y
(16) 0
-0.5
-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
Figure 5. The HF model and LF model, with 3 initial HF
samples (Example 1)
Different from using a single fidelity model, in variable fidelity
optimization, very few data from HF simulations are available,
thus LF model may be used to capture the global trend of a HF
model. We note in Figure 5 that the scaled LF model is fairly
close to the HF model. The trend information provided by the
scaled LF model is integrated into the Bayesian surrogate
model to enhance the accuracy of prediction.
From Stages 1 to 5, additional 5 sampling points are
sequentially collected following the PSC strategy we proposed
in Section 2.3. Using the Bayesian Gaussian process modeling
approach described in Section 2.2, the posterior mean and the
interpolation uncertainty (95% prediction interval (PI)) of
y s (x) are shown in Figure 6, with all 5 sequential points
annotated with the stage number besides. It is noted that
although two local minimums exist in the HF model, only one
of the 5 samples is used to explore the secondary local
5 Copyright © 2007 by ASME
minimum region, the other four are all located around the the true optimal design (x*HF= [0.0970, 0.9344], marked with a
global minimum. Therefore the created surrogate model is star).
much more accurate in the neighborhood of the global
minimum than that of the secondary local minimum. In Stages Y HF
4 and 5, the sequential sampling points are very close to the Y LF
global minimum. The sequential sampling process is objective- 300
oriented, addressing both needs of global search and local 200
search.

y
100
In Figure 6, points in the local region (0.215~0.253) 0 1
surrounding x* (=0.2330) is identified as indifferentiable to x*
0 0.5
with certain design tolerance H (=0.023) and confidence 0.5
1 0
C X 0 (=99%). The design point x=0.746, shown with large x1
x2

interpolation uncertainty, is identified as the most competing Figure 7. The 3-D plots of HF and LF models
point xmc (marked with triangle). The xmc point is considered as
1 15
the most competing point to x* among all design points outside 0 0
15 3 0

70
10
90 170gloabl optimum

30
HF 1
the indifferentiable region X0 because its lower bound of the

-1 0
0.9 13 0
0 HF local optimum 11
90
prediction interval is very close to the performance at x*. 11

10
0.8 50 0
Stage (5) 70 90

-1 0
30
0.7
Ys hat 30 70 50
95% PI 0.6
30

10
1 50
initial HF samples
0.5

50
x2
sequential HF samples
10
x star from Ys

11 0
7090
0.4

10
10

30
most competing point
0.5

30

10
0.3
Ys hat

0.2

15 01711900
50
0

23 0 25
0.1

2
mc

0
x

13

30
70 0
11

10
10
0
0
3

9
1 0

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
2 x1
-0.5
5 4
Figure 8. The contour plot of HF model marked with x*HF
=[0.0912 0.9325], yh(x*HF)=-19.142.
-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Stage (0)
x 1
Initial HF samples

Figure 6. The plots of yˆ s (x) and the HF samples (Stage 5)

0.9 x star from HF
x star from Ys
0.8 Indifferentiable region
3.2 Example 2: Two Dimensional Problem 1 (the Modified 0.7
most competing point

Branin Function)
0.6
In this example, the effectiveness of the proposed sequential
0.5
x2

approach is demonstrated through a modified Branin function.

Optimizing the Branin function [31] is challenging because it 0.4

has three global minima with exactly equal performance values. 0.3
The problem has been studied in literature for various purposes. 0.2
In this work we modified the original Branin function by
0.1
adding an additional small ‘tip’ term so that it has only one
global minimum, while the other two become local minima but 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
stay competing to the global one. The modified Branin function x1

is used as the HF model, while we hypothetically construct the

LF model (see in Appendix for mathematical details). The 3-D
plots of the HF and the LF models are shown in Figure 7, and
the contour plot of the HF model with the global optimal point
indicated as x*HF is shown in Figure 8, with the two local
minima marked with * and the global minimum marked with a
star. Note the ranges of x1 and x2 are normalized to 0~1.
At the initial stage (Stage 0)(see Figure 9), only 5 sampling
points are available which are generated with the Optimal
Latin-Hypercube (OLH) algorithm [32]. The contour plot of the
obtained surrogate model at this stage is shown in Figure 9. It is
clear that the optimal design (x*= [0.6094, 0.3012], marked
with a square) from the surrogate model is very different from
Figure 9. The plot of yˆ s (x) at Stage 0 (5 points) with x*=
[0.694, 0.001]
In order to demonstrate the effectiveness of the proposed PSC
strategy in the sequential sampling, we compare the result from
using the EI criterion with that from the PSC strategy using the
same amount of sampling data. The plots of the resulted
surrogate models from EI and PSC at Stage 10 (i.e., after 10
sequential points are added) are shown in Figure 10 and Figure
11, respectively for comparison.
In Figure 10, where the EI is applied, it is found that the
optimal design x* (marked by square) is erroneously identified

6 Copyright © 2007 by ASME

at a sub-optimal region. The number marked beside each depicted by a collection of ‘+’ markers. xmc and X0 are
sequential point indicates the stage they belong to. It is found determined based on the definitions given in the previous
that the sequential points via EI criterion (marked by solid section. In essence, with the consideration of model
circles) fails to discover the region sufficiently before it uncertainty, the designs in the indifferentiable region are
converges to the local minimum. Similar observations about the considered as equivalent to the optimal x* within certain
shortcomings of the EI criterion are also made in [19] and [27]. tolerance. The design tolerance value H selected for this
Stage (10) example is 12, which is about 3% of the range of y. Note that in
1
1 Initial HF samples
Figure 10, since the local sub-optimal regions have similar y
0.9 Sequential HF samples values, the indifferentiable region X0 w.r.t. x* are located in
x star from HF
0.8
x star from Ys
both the neighborhood of x* and a disjoint region centered
0.7
Indifferentiable region around the other sub-optimal region extreme. In Figure 11,
most competing point
since x* is already correctly identified within the true global
0.6
optimal region, X0 is a continuous region surrounding x*. The
0.5 most competing point xmc w.r.t. x* is marked with a triangle in
x2

0.4 2 both Figures 10 and 11. It is noted that xmc is always located
0.3 54
outside of X0 and represents the most competing design w.r.t
8 10 x*. Based on Eqs. (17) and (18), the design confidence DC(x*)
0.2 6
9 3 achieved using the EI criterion and our proposed PSC strategy
0.1
7
is 87.12% and 99.99%, respectively.
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 15
x1 PSC strategy
10 EI criteria
s
Figure 10. The plot of yˆ (x) at Stage 10, with x*= [0.5276, One-shot sampling
ymin from HF
5
0.2132]: sequential sampling points generated by EI criterion
Stage (10) 0

ymin(k) from Ys
1
2 1
0.9 10 -5
4 Initial HF samples
0.8 5 Sequential HF samples
-10
x star from HF
0.7 x star from Ys
3 -15
Indifferentiable region
0.6
most competing point
-20
0.5
x2

8
0.4 -25
0 1 2 3 4 5 6 7 8 9 10
Stage k
0.3
9 7
0.2 Figure 12. The comparison of the history plots of y*
0.1 1
6 PSC strategy
0 0.95 EI criteria
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
One-shot sampling
x1
0.9

Figure 11. The plot of yˆ s (x) at Stage 10, with x*= [0.1011, 0.85

0.9156]: sequential sampling points generated by PSC strategy 0.8

DC

0.75
In Figure 11, where the PSC is applied, x* (marked by square)
0.7
is identified at [0.1011, 0.9156], fairly close to x*HF. From the
locations of the sequential points (marked by solid circles), it is 0.65

found that most of the sampling points are placed in the local 0.6
region around the global minimum, while the rest of them are 0.55
placed elsewhere to reduce the interpolation uncertainty of the
surrogate model. It is found that at the early stages when the 0.5
0 1 2 3 4 5 6 7 8 9 10
uncertainty of the surrogate model is large, the sampling Stage k

procedure explores the model space more rather than focusing
on any local promising region. After sufficient samples have
been accumulated and the uncertainty of a surrogate model is
reduced, more samples are used for local refinement of the
global surrogate model in the region that is in favor of the
design objective.
The most competing point (xmc) w.r.t. the optimal x* obtained
from the surrogate model, is marked as a triangle in both plots
in Figures 10 and 11, where the indifferentiable region X0 is
Figure 13. The comparison of the history plots of DC(x*)
(H = 12 (3% of yrange) and C X 0 = 0.95)
Figures 12 and 13 show the history plots of the response value
y* and design confidence value DC(x*) from the surrogate
models at different stages. The results from using the EI
criterion and the PSC strategy are compared in both figures. It
is observed that the DC level consistently increases with more
sample points when using the PSC strategy which yields a high

7 Copyright © 2007 by ASME

DC level (close to 100%) in Stages 9 and 10, much better than simulated using ADAMS/Flex, a finite element based multi-
that (87.4%) from the EI with the same amount of sampling body dynamics code. In this study, four geometric parameters
points. Besides, in terms of y*, from the true y*HF indicated by are considered as the design variables (see Table 1): skirt length
the horizontal dashed line in Figure 13, it is observed that the (SL), skirt profile (SP), skirt ovality (SO), and pin offset (PO),
PSC generates a more accurate value than the EI. These facts while the performance of interest is the slap noise (SN). The
imply that the proposed PSC strategy holds much advantage HF and LF models we use are two Kriging models based on,
over the EI approach. respectively, a larger set (200) of data and a smaller set (20) of
data. Using the HF model output to validate the LF model, the
From the history plot of y* in Figure 13, it is found that y* R-Square value of the LF model is obtained as 61.60%,
from both EI and PSC appears to be stabilized after a few indicating the LF model is not a satisfactory approximation to
stages. However, only examining the history of y* is not the HF model.
sufficient in determining if the sequential process should be
terminated, because the DC level could still be low like the case Table 1. Design variable in the Engine Piston design
with using the EI method. This implies that the use of the Variable Description Lower Upper Unit
design confidence (DC) as a termination criterion is more Bound Bound
effective which offers more information for design decision SL Skirt Length 21 25 mm
making than simply examining the convergence behavior of y*. SP Skirt Profile 1 3 N/A
1
Stage (0) SO Skirt Ovality 1 3 N/A
Initial HF samples PO Pin Offset 0.5 1.3 mm
0.9 x star from HF
x star from Ys
0.8
Indifferentiable region At Stage 0, we create 10 initial sampling points using the
0.7
most competing point
space-filling criterion (OLH). During the following 5 stages,
0.6
additional 5 sampling points are collected using the proposed
PSC strategy.
0.5
x2

1
0.4
0.95
0.3
0.9 Setting A: H=0.24 Cx0=0.95
0.2
Setting B: H=0.08 Cx0=0.99
0.85
0.1
0.8
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
DC

x1 0.75

Figure 14. The plots of yˆ s (x) with x*= [1.0000, 0.2103]

0.7

0.65
(One-shot sampling of 5+10=15 points)
0.6

To demonstrate the advantage of using sequential sampling 0.55

over a one-shot (single stage) sampling, we generate the same 0.5
amount of data (5+10=15 points) using the space-filling 0 1 2 3 4 5
Stage k
6 7 8 9 10

criterion (OLH) as a comparison to the sequential sampling

above. Figure 14 shows the settings of the evenly-spaced 15
points and the contour plot of the surrogate model yˆ s (x) built
using the HF data. Even though this surrogate model might be
more accurate in a global sense than those built based on
sequential sampling, the model fails to capture the local details
of the three local minimum regions, and the optimal design
x*=[1.000, 0.2103] (marked by square) is erroneously
identified at the local minimum region far away from the true
global minimum. It is found that DC(x*) (=72.81%) achieved
by the one-shot sampling is lower than the sequential sampling
by both the EI criterion and the PSC strategy.
3.3 Example 3: Engine Piston Design – An Engineering
Design Example
We use the vehicle engine piston design case study previously
analyzed in [33] as an illustrative engineering example. The
design goal of this problem is to optimize the geometry of the
engine piston to obtain the minimal piston slap noise (measured
in dB). Technical background of the engine piston design
problem can be found in [34]. Piston slap noise is the engine
noise resulting from piston secondary motion, which can be
Figure 15. History plot of DC(x*)
Figure 15 shows the history plot of DC(x*) measured with two
settings of values of design tolerance (H) and confidence of
indifferentiable region ( C X 0 ), namely, Setting A (H=0.24,
C X 0 =95%) and Setting B (H=0.08, C X 0 =99%). Note Setting
B is much stricter than Setting A. Stricter H and C X 0 usually
reflect the higher expectation from designers about the
accuracy and validity of the optimal solution. From Figure 15,
it is found that Setting B always achieves smaller DC values
from Stage 0 to Stage 4. However, the two DC curves begin to
‘merge’ after Stage 4 and stay above 99% in all later stages. If
we assume the desired confidence level prescribed by designers
are 95% for Setting A, and 99% for Setting B, applying the DC
based stopping criteria described in Section 2.4, the sequential
process could be terminated at Stage 5 for Setting A and Stage
6 for Setting B after the DC values of two consecutive stages
are above the specified level. This example shows that
designers’ preference does have an impact on the decision of
whether a solution can be accepted with sufficient confidence.

8 Copyright © 2007 by ASME

 approximation accuracy compared to only using HF simulation
3.4 Comparison of Surrogate Models from Different Model data or combining it with the LF model without scaling. With
Fusion Approaches Gaussian process modeling, interpolation uncertainty due to the
In our proposed model fusion approach as described in Section lack of sufficient HF simulations can also be quantified. As a
2.2, the constrained linear scaling (CLS) method is applied first part of the variability fidelity optimization framework, an
to correct the LF model, before using a bias function that objective-oriented sequential sampling strategy is investigated.
accounts for the remaining discrepancy based on Bayesian Compared with the Expected Improvement approach, the
modeling. To show the effectiveness of the approach, adopted periodical switching criteria (PSC) based strategy for
comparative results of different approaches applied over all sequential sampling achieves a better balance between
three examples discussed above are provided in Table 2. The optimizing the design objective and reducing the interpolation
Root Mean Square Error (RMSE) is utilized as the accuracy uncertainty. We also propose to use design confidence (DC), a
metric: the smaller RMSE the higher accuracy. Note that all the probabilistic measure of the confidence in employing the
sampling points are created in an one-shot space-filling surrogate model for making a specific design choice, as the
criterion (OLH), rather than with the sequential sampling. For stopping criterion for the proposed framework. This proposed
the same problem with the same different settings of sampling approach effectively facilitates decision making in engineering
sizes, the RMSE values from three modeling approaches, design, by taking into account the uncertainty associated with
namely, the ‘non-fusion’ approach, the ‘non-scaled’ approach, the use of surrogate models. Through examples, we show that
and the proposed CLS approach are compared. By ‘non- designers’ preference has an impact on the decision of whether
fusion’, we mean that the information from the LF model is not a solution can be accepted with sufficient confidence.
considered at all for constructing the surrogate model. The only
difference between the ‘non-scaled’ approach and the ‘CLS’ In this paper, we only work on examples of unconstrained
approach is that the former does not scale LF model before optimization. The methodology could be extended to more
applying the bias function. general constrained optimization problems, which would be
one future research topic. Another potential application of the
In Table 2, the best RMSE values are marked in bold. From proposed framework is to extend the approach to design
these data, we note that the ‘non-fusion’ approach without optimization using information from both computer simulations
exploiting the LF models always rank the worst among the and physical experiments, while the former and later will play
three with remarkably large RMSE values, which implies that a the similar roles of LF model and HF data, respectively.
surrogate model based on the fusion of LF and HF models is
much superior over a surrogate model based on only HF model. ACKNOWLEDGMENTS
Compared with the ‘non-scaled’ approaches, the CLS approach The grant support (DMI – 0522662) from National Science
we propose also yields higher accuracy in most of comparisons. Foundation (NSF) to this research is greatly appreciated.
Table 2. The accuracy (RMSE) comparison APPENDIX
Example 1 (1-D)
Sampling size
Example 1: y1h ( x) = 0.5sin[4π sin( x + 0.5)] + ( x + 0.5) 2 / 3 ,
5 6 7 8 10
non-fusion 0.326 0.297 0.313 0.336 0.338 y1l ( x) = y1h (1.1x − 0.1) − 0.2 , x ∈ [0,1]
non-scaled 0.221 0.114 0.060 0.044 0.003 Example 2: y2h ( x1, x2 ) = ybranin ( x1, x2 ) − 22.5 x2 ,
CLS 0.286 0.112 0.052 0.036 0.003
Example 2 (2-D, Modified Branin function) where, ybranin ( x1, x2 ) = 10 + [ x2 − 5.1x12 /(4π 2 ) + 5 /(π x1) − 6]2
Sampling size 7 8 9 10 15 +10cos( x1)[1 − 1/(8π )]
non-fusion 85.450 92.829 79.249 91.675 82.18 y2l ( x1, x2 ) = y2h (0.7 x1,0.7 x2 ) + 20 ⋅ (0.8 + x1 )3 − 50 , x1 ∈ [−5,10]
non-scaled 12.673 8.775 8.787 7.555 6.544 x2 ∈ [0,15]
CLS 8.939 7.074 9.186 7.358 2.350
Example 3 (4-D, Engine Piston problem)
REFERENCES
Sampling size 10 15 20 30 50 [1] Alexandrov, N. M., Lewis, R. M., Gumbert, C. R., Green,
non-fusion 55.475 55.365 55.406 55.409 55.415 L. L., and Newman, P. A., 1999, "Optimization with
non-scaled 0.026 0.022 0.023 0.014 0.011 variable-fidelity models applied to wing design," The 38th
CLS 0.024 0.036 0.022 0.020 0.011 AIAA Aerospace Sciences Meeting and Exhibit,
January10–13, 2000, Reno, NV.
4. CONCLUSION [2] Rodriguez, J. F., Pérez, V. M., Padmanabhan, D., and
Renaud, J. E., 2001, "Sequential Approximate
In this work, we propose a new variable fidelity optimization
Optimization Using Variable Fidelity Response Surface
approach from the perspective of reducing the uncertainty of
Approximations," Structural and Multidisciplinary
using surrogate models in engineering design. To effectively
Optimization, 22, 24-34.
incorporate the information from both HF and LF models, a
[3] Gano, S. E., Renuad, J. E., and Sanders, B., 2004,
model fusion approach based on linear scaling and Bayesian
"Variable Fidelity Optimization Using a Kriging Based
Gaussian process modeling is proposed. It has been shown
Scaling Function," 10th AIAA/ISSMO Multidisciplinary
through examples that the Bayesian surrogate model obtained
Analysis and Optimization Conference, 30 August - 1
from the proposed model fusion approach has a better
September 2004, Albany, New York.

9 Copyright © 2007 by ASME

[4] Panchal, J. H., Choi, H.-J., Shepherd, J., Allen, J. K.,
McDowell, D. L., and Mistree, F., 2005, "A Strategy for
Simulation-Based Design of Multiscale, Multi-Functional
Products and Associated Design Processes," ASME 2005
IDETC/CIE Conference, Long Beach, CA.
[5] Liu, W. K., Karpov, E. G., and Park, H. S., 2005, Nano-
Mechanics and Materials: Theory, Multiscale Methods and
Applications, Wiley, New York.
[6] Gano, S. E., Renaud, J. E., Martin, J. D., and Simpson, T.
W., 2006, "Update strategies for kriging models used in
variable fidelity optimization," Structural and
Multidisciplinary Optimization, 32(4), 287-298.
[7] Qian, Z., Seepersad, C. C., Joseph, V. R., Allen, J. K., and
Wu, C. F. J., 2006, "Building Surrogate Models Based on
Detailed and Approximate Simulations," Journal of
Mechanical Design, 128, 668.
[8] Chen, W., Xiong, Y., Tsui, K.-L., and Wang, S., 2006,
"Some Metrics and a Bayesian Procedure for Validating
Predictive Models in Engineering Design," ASME Design
Technical Conference, Design Automation Conference,
September 10-13, 2006, Philadelphia, PA.
[9] Wang, S., Chen, W., and Tsui, K., 2006, "Bayesian
Validation of Computer Models," submitted to
Technometrics.
[10]Apley, D., Liu, J., and Chen, W., 2006, "Understanding the
Effects of Model Uncertainty in Robust Design with
Computer Experiments," ASME Journal of Mechanical
Design, 128(945), 745-958.
[11]Sacks, J., Welch, W. J., Mitchell, T. J., and Wynn, H. P.,
1989, "Design and Analysis of Computer Experiments,"
Statistical Science, 4(4), 409-423.
[12]Martin, J. D., and Simpson, T. W., 2002, "Use of Adaptive
Metamodeling for Design Optimization," 9th AIAA/ISSMO
Symposium on Multidisciplinary Analysis and
Optimization, Atlanta, GA.
[13]Turner, C. J., Campbell, M. I., and Crawford, R. H., 2003,
"Generic sequential sampling for meta-model
approximations," ASME 2003 Design Engineering
Technical Conferences and Computers and Information in
Engineering Conference, September 2-6, 2003, Chicago,
Illinois.
[14]Jin, R., Chen, W., and Sudjianto, A., 2002, "On Sequential
Sampling for Global Metamodeling in Engineering
Design," The 2002 ASME IDETC Conferences, September
29-October 2, Montreal, Quebec, Canada.
[15]Lin, Y., Mistree, F., and Allen, J. K., 2004, "A Sequential
Exploratory Experimental Design Method: Development
of Appropriate Empirical Models in Design," ASME 2004
Design Engineering Technical Conferences and
Computers and Information in Engineering Conference,
September 28-October 2, Salt Lake City, Utah.
[16]Farhang-Mehr, A., and Azarm, S., 2005, "Bayesian Meta-
Modeling of Engineering Design Simulations: A [33]Jin, R., 2005, "Enhancements of Metamodeling Techniques
Sequential Approach with Adaptation to Irregularities in
the Response Behavior," International Journal for
Numerical Methods in Engineering, 62(15), 2104-2126.
[17]Jones, D. R., Schonlau, M., and Welch, W. J., 1998,
"Efficient global optimization of expensive black-box
functions," Journal of Global Optimization, 13(4), 455-
492.
View publication stats
[18] Mockus, J., Tiesis, V., and Zilinskas, A., 1978, "The
application of Bayesian Methods for Seeking the
Extremum," Towards Global Optimisation, 2, 117-129.
[19]Sasena, M. J., 2002, "Flexibility and Efficiency
Enhancements for Constrained Global Design
Optimization with Kriging Approximations," University of
Michigan.
[20]Bandler, J. W., Cheng, Q. S., Dakroury, S. A., Mohamed,
A. S., Bakr, M. H., Madsen, K., and Sondergaard, J., 2004,
"Space mapping: the state of the art," IEEE Transactions
on Microwave Theory and Techniques, 52(1), 337-361.
[21]Hastie, T., Tibshirani, R., and Friedman, J., 2001, The
Elements of Statistical Learning, Data Mining, Inference
and Prediction, Springer, New York.
[22]Smith, R. L., and Zhu, Z., 2004, "Asymptotic Theory for
Kriging with Estimated Parameters and Its Application to
Network Design," Preliminary version, available from
http://www.stat.unc.edu/postscript/rs/supp5.pdf.
[23]Li, R., and Sudjianto, A., 2003, "Analysis of Computer
Experiments Using Penalized Likelihood Gaussian Kriging
Model," ASME International DETC &CIE, September,
Chicago, IL.
[24]Santner, T. J., Williams, B. J., and Notz, W. I., 2003, The
Design and Analysis of Computer Experiments, Springer-
Verlag, New York.
[25]Qian, Z., and Wu, C. F. J., 2005, "Bayesian Hierarchical
Modeling for Integrating Low-Accuracy and High
Accuracy Experiments," Georgia Institue of Technology.
[26]Jones, D. R., 2001, "A Taxonomy of Global Optimization
Methods Based on Response Surfaces," Journal of Global
Optimization 21(4), 345-383.
[27]Sekishiro, M., Venter, G., and Balabanov, V., 2006,
"Combined Kriging and Gradient-Based Optimization
Method," 11th AIAA/ISSMO Multidisciplinary Analysis
and Optimization Conference, September 6 - 8,
Portsmouth, Virginia.
[28]Cox, D. D., and John, S., 1997, "SDO: A Statistical Method
for Global Optimization " Multidisciplinary design
optimization: state of the art, SIAM, Philadephia, 315-329.
[29]Hazelrigg, G. A., 2003, "Thoughts on Model Validation for
Engineering Design," ASME Design Technical Conference,
Sept. 2-6, 2003, Chicago, IL.
[30]Nocedal, J., and Wright, S. J., 1999, Numerical
Optimization, Springer Verlag.
[31]Leary, S. J., Bhaskar, A., and Keane, A. J., 2004, "A
Derivative Based Surrogate Model for Approximating and
Optimizing the Output of an Expensive Computer
Simulation," Journal of Global Optimization, 30, 39-58.
[32]Jin, R., Chen, W., and Sudjianto, A., 2005, "An Efficient
Algorithm for Constructing Optimal Design of Computer
Experiments," Journal of Statistical Planning and
Inference, 134(1), 268-287.
on Engineering Design," University of Illinois Chicago.
[34]Hoffman, R. M., Sudjianto, A., Du, X., and Stout, J., 2003,
"Robust Piston Design and Optimization Using Piston
Secondary Motion Analysis," SAE World Congress, Mar.
3-6, 2003, Detroit, Michigan.
10 Copyright © 2007 by ASME