16 views

Uploaded by Alexander

Sensitivity Method for kinetics models

- Practices Barriers and Benefits of Using Risk Management Approaches in Selected Hong Kong Industries 1997 International Journal of Project Management
- Parameters of Rock Mass
- 034
- 037
- Sensitivity Analysis for Noise MappingThe Department for Environment
- A2.pdf
- liu2017.pdf
- 3rdNSCportland3
- RISK OF EXTREME EVENTS IN A MULTIOBJECTIVE FRAMEWORK
- Uncertainty Analysis in Air Dispersion Modeling
- BA
- LP Sensitivity
- POWER MEASUREMENT AS SENSITIVITY ANALYSIS A UNIFIED APPROACH
- Sensitivity Analysis
- 132441-356985-1-PB
- saruman field
- Research Analytics
- Probability and Statistics in Engineering by William W. Hines, Douglas C. Montgomery, David M. Goldsman, Connie M. Borror
- Analysis HW #12
- Smith Et Al - Mental Mistakes

You are on page 1of 18

www.elsevier.com/locate/jfranklin

for parameter estimation

H. Sulieman, I. Kucuk

Department of Mathematics and Statistics, American University of Sharjah, P.O. Box 26666, Sharjah,

United Arab Emirates

Received 4 June 2009; received in revised form 14 May 2010; accepted 18 May 2010

Available online 2 June 2010

Abstract

In nonlinear parameter estimation local sensitivity assessment; conventionally measured by the

rst-order derivative of the predicted response with respect to a parameter of interest fails to provide

a representative picture of the prediction sensitivity in the presence of signicant parameter codependencies and/or nonlinearities. In this article we derive the prole-based sensitivity measure

developed by Sulieman et al. (2001, 2004) [1,2] in the context of model re-parameterization. In

particular, the so-called predicted response re-parameterization is shown to ultimately lead to the

prole-based sensitivity coefcient dened by the total derivative of the model predicted response

with respect to a parameter. Although inherently local, the prole-based measure is shown to handle

simultaneous perturbations in parameter values while accounting for their co-dependencies. Thus the

proposed measure possesses a central property of a global sensitivity measure and so it is considered

hybrid localglobal measure.

The global Fourier amplitude sensitivity test (FAST) is added to the analysis and compared with

both marginal and prole-based sensitivity methods. The Fourier sine amplitude is utilized here as a

rst-order sensitivity measure and shown to be directly linked to the local sensitivity coefcient

averaged over all ranges of parameter uncertainties and so it is also considered hybrid localglobal

measure. The comparisons are explained by three compelling model cases with different degrees of

parameter co-dependencies and nonlinearities.

& 2010 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

0016-0032/$32.00 & 2010 The Franklin Institute. Published by Elsevier Ltd. All rights reserved.

doi:10.1016/j.jfranklin.2010.05.014

1557

1. Introduction

Parametric sensitivity analysis in general describes the impact of perturbations in the

values of model input parameters on the model outputs. The objective of the sensitivity

assessment is to improve the quality of an existing model representing a physical system

perhaps by reducing complexity or by guiding further experiments to reduce uncertainty.

Through better understanding of the interplay between the model input parameters and

the relative importance they have on the model outputs, one can target specic parameters

for more detailed study to reduce model uncertainty.

This article investigates the sensitivity of the predicted responses from nonlinear

regression models to variations in parameter values. The numerical values of these

parameters are usually estimated using available experimental data. The resulting

uncertainties associated with these parameter estimates propagate into the model

predictions via sensitivities. Results of the sensitivity analysis applied to an existing model

of a physical system can be used to strengthen the knowledge base by guiding subsequent

research in order to increase the condence in the model and its predictions.

In the literature, there are two different schools of thought for parametric sensitivity

methods. Local methods are derivative-based methods and generate sensitivity information that is valid over small ranges of parameter uncertainties where linear approximation

to the model function is assumed adequate. The conventional measure of local parametric

sensitivity is dened by the rst-order partial derivatives of the model response function

with respect to the parameters.The resulting sensitivity coefcients measure the marginal

impact of the parameter of interest on model predictions since only the value of the

parameter of interest is perturbed while all other parameters are held xed at their nominal

values. Therefore, co-dependencies among parameter estimates are ignored by these

measures, Sulieman et al. [1] called them marginal sensitivity coefficients. Global sensitivity

methods assess the inuence of a parameter on the model response by exploring the entire

parameter space. Simultaneous changes in parameter values across their respective ranges

of uncertainty are incorporated in an overall measure of sensitivity. Majority of global

sensitivity techniques are variance-based measures that apportion the output uncertainty

to the uncertainty in the parameters. Some of these methods are model independent and

can be applied to nonlinear and non-monotonic models.

To surmount the drawbacks of the local sensitivity assessment, Sulieman et al. [1,2]

proposed an alternative assessment procedure in which simultaneous perturbations in the

values of all model parameters are achieved using the proling scheme introduced by Bates

and Watts [3] for nonlinearity assessment of regression models. The prole-based

sensitivity measure, dened by the total derivative of the model function with respect to

parameter of interest, was shown to account for both nonlinearity within the parameter

estimation problem and parameter estimate co-dependencies. Like any derivative measure,

prole-based sensitivity is inherently local, it provides, however, a more comprehensive

picture of the prediction sensitivity in the presence of parameter co-dependencies and

model nonlinearity. Detailed discussion of the prole-based sensitivity measures is

presented in Section 2.

Recently, Sulieman et al. [4] conducted comparative study of prole-based sensitivity

approach versus Fourier amplitude sensitivity test (FAST) for parameter estimation in

multi-response regression models. FAST was developed by Cukier et al. [57] and Schaibly

and Shuler [8] to assess sensitivity of model outputs to input parameters in complex model

1558

systems. It is one of the most widespread techniques in the sensitivity literature. FAST

main essence is the simultaneous perturbations of all parameter values using a single set of

computational runs. The convenience of FAST perturbation scheme manifests itself

because all the terms in the Fourier expansion of the model function are mutually

orthogonal. In contrast to prole-based approach, classical FAST is shown to ignore the

parameter estimates co-dependencies in the model while it accounts for the full model

nonlinearity. Description of FAST and its features are given in Section 3.

In this article, we present prole-based sensitivity procedure using the notion of model

re-parameterization in single-response nonlinear regression models. In particular, we

adopt predicted value re-parameterization in which the expression for the predicted

response at a selected design point is dened as one of the parameters in the new system.

Such a re-parameterization allows the proling-based sensitivity measure to be calculated

automatically with the calculations of the prole vector when tting the newly formulated

model, and the inferential results about the model prediction to be obtained in a more

straightforward and computationally efcient manner. We also explore comparisons

between the measures of the conventional local sensitivity, the prole-based sensitivity and

conventional FAST method. The comparisons are demonstrated by the implementation of

the three measures to three model cases with different model structures and different

degrees of parameter estimates nonlinear co-dependencies.

In Section 2 we describe the development of the prole-based sensitivity measure for the

nonlinear parameter estimation using the predicted response re-parameterization of the

model function. In Section 3 we review FAST, consider the most recent developments in

the application of this analysis technique and discuss its link to marginal and prole-based

sensitivity measures. Illustrative model cases are presented in Section 4, and conclusions

are summarized in Section 5.

2. Prole-based sensitivity analysis

Consider the general mathematical form of a single response nonlinear regression model

y fX; Y e

where y is an n-element vector of observed values of the response variable for particular

values of the regressor variables X={x1, x2,y,xn}, Y is a p-element vector of unknown

parameters, f is an n-element vector of predicted values of the response variable for given X

and Y, fX; Y ff x1 ; Y; f x2 ; Y; . . . f xn ; Yg, and e is an n-element vector of

independent random errors with a specied joint distribution. In most cases, including

the case here, e is assumed to have a spherical normal distribution, with Ee 0 and

0

vare Eee s2 I.

Sulieman et al. [1] developed a prole-based sensitivity measure to assess the sensitivity

of the predicted responses from model (1). The motivation for the prole-based approach

arises from the proling algorithm developed by Bates and Watts [3] for constructing

likelihood intervals for individual parameters in single response nonlinear regression

models. The proling algorithm was implemented to a reformulated model function using

special re-parameterization in which the predicted response is one of the parameters in the

new formulation. The slope of the prole trace of the predicted response parameter has led

to the denition of prole-based sensitivity coefcient. The resulted sensitivity coefcient

quanties the net change in the predicted response due to perturbations in a parameter of

1559

interest, after the remaining parameters are updated to their estimates conditioned on the

perturbed value of the parameter.

The model in Eq. (1) can be re-expressed as

y gY e

2

0

predicted responses dened in Eq. (1), i.e.,

Zj Y f xj ;Y;

j 1;2; . . . ;n

parameters Y. The predicted response parameterization is dened as follows:

f1 Z0 Y f Y;x x0

f2 y2

^

fp yp

should be mentioned that the above form (4) does not necessarily suggest that only f1

should be used to dene the predicted response at x0, or that only y1 should be used as its

inverse f1

1 . Any other two parameters from F and Y parameters can be chosen for these

purposes. For readers convenience, however, we will denote the predicted response

parameter by f1 throughout the paper, whereas the inverse transformation f1

1 will be

dened by one of the least nonlinear behaving parameters in gY.

The idea of using this particular class of transformations was rst suggested by Ross

[9,10]. Bates and Watts [11] showed that when the regression model has only one nonlinear

parameter, the transformation (4) will reduce parameter nonlinearity to zero for all Y.

Clarke [12] referred to this class of transformation as optimal parameter transformations since it produces zero or small parameter nonlinearities in the model and minimum

estimation bias. However, as shown by Clarke [12], the effectiveness of such

re-parameterizations depends substantially on the choice of the points x0 at which the

predicted responses are to be estimated.

In using this class of transformations, our purpose is not to reduce either the parameter

nonlinearities in the model but rather to characterize the parameter sensitivities of the

predicted response at a predetermined point x0. Therefore, the re-parameterization dened

in Eq. (4) is more functional here since it denes the identity transformation for (p1)

parameters, so that neither the nonlinearities of these parameters nor the biases in their

estimates are changed by the transformation. The only parameter for which the

nonlinearity or the bias may change is the one dened by the inverse transformation

f1

1 , say y1 , where y1 is chosen to be one of the most linearly behaving parameters in the

model.

Let gnew F be the new formulation of the model function in terms of F and suppose that

fi is the parameter of interest. In proling algorithm, the vector of parameters F is

partitioned as F fi ; Fi which is, in terms of Y, equivalent to Y yi ; Yi . fi yi is

then varied across its range of uncertainty and for each value of fi , let F~ i fi be the

conditional least squares estimates of Fi . The joint behavior of the predicted response

parameter and each of the remaining parameters in gnew can be understood through the

prole traces, and so sensitivity information can be extracted from the prole trace plots.

1560

Using the studentized parameters, the prole trace of the predicted response parameter f1

~

~

versus fi , i=2,3,y,p, is the curve consisting of the points dfi ; df

1 , where df1 is the

studentized conditional estimate of f1 given a xed value of fi , i.e,

f~ 1 f^ 1

~

df

1

sef^ 1

and

dfi

fi f^ i

sef^

where the parameter estimates f^ 1 and f^ i and their associated standard errors sef^ 1 and

sef^ i are obtained from the unconditional least squares tting of gnew to the data.

~

The slope of df

1 with respect to dfi is equal to the prole-based sensitivity coefcient

[1]. The mathematical derivations of prole-based sensitivity coefcients are detailed in the

following subsection.

An argument against the use of the predicted value transformation is the difculty with

the inverse transformation from F to Y. In practice this need not be serious. Even when no

explicit inverse transformation exists, an iterative solution can be made almost trivial

because of the ready availability of good starting values.

Example 2.1. MichaelisMenten Model. MichaelisMenten model is commonly used in

enzymatic kinetics with well-known formulation:

f x;Y

y1 x

y2 x

where y is the measured initial velocity of an enzymatic reaction and x is the substrate

concentration. The unknown parameters y1 and y2 represent maximum conversion rate

and MichaelisMenten constant, respectively. Several types of linearization parameterization in which the model terms are re-arranged so that it becomes linear-in-parameters have

been proposed for the MichaelisMenten model [13]. The purpose of these linearization

parameterizations is to obtain global minimum when using least squares estimation

method. Alcazar and Ancheyta [14] argued that the nonlinear estimation techniques must

be used in order to avoid large estimation bias and signicant error distortion by the

linearization transformation. To overcome the drawbacks of the linearization process,

Doeswijk and Keesman [15] used a bias compensated total least squares method in which

bias compensation is incorporated into the calculation of the parameter estimate

covariance matrix in an ordinary least squares estimation strategy.

For the model formulation in Eq. (7), y1 appears linearly and is known to have

insignicant nonlinear behavior in the least squares tting of the model. Hence, y1 is used

for the inverse transformation f1

1 . The transformation (4) is applied to the two-parameter

MichaelisMenten model as follows:

f1

y1 x0

y2 x 0

and

f 2 y2

1561

gnew F

f1 xf2 x0

x0 f2 x

The data set used by Bates and Watts [3] for tting model (7) is used here and the

estimation results are reproduced in Table 1: The data set consists of six prediction (design)

points, each is replicated twice. Using the new formulation in Eq. (8), sensitivity of the

predicted response at each of the six prediction points is assesses with respect to f2 y2 .

The range of uncertainty for parameter y2 is considered to be the 95% condence interval,

i.e., within two standard errors of y^ 2 0:06411. Twenty values of y2 were generated in the

prescribed range and for each value, f1 is conditionally estimated using model (8). The

resulting six prole traces are depicted in Fig. 1.

In Fig. 1 the six prole traces intersect at the point (0,0) corresponding to

y^ 1 212:68; y^ 2 0:06411. Different sensitivity behaviors are clearly seen for y2 f2 at

the six prediction points. For the prediction at x0=0.02, 0.06, 0.11, y2 possesses negative

impact with sharpest slope at x0=0.02. The slope of the prole trace at x0=0.22 is

insignicant indicating small inuence of y2 at the predicted response at this design point.

At x0=0.56, 1.10 the prole traces have positive slopes indicating positive sensitivity

behavior of the corresponding predicted responses to y2 . Among the six prediction points,

the largest sensitivity is exhibited at x0=0.02 with the most nonlinear trend.

The sensitivity analysis procedure described in the above example along with the visual

summary in Fig. 1 has prompted the development of a prole-based sensitivity measure

dened by the slope of the prole trace of f1 versus a parameter of interest. The resulting

measure is named prole-based sensitivity coefcient (PSC).

2.1. Profile-based sensitivity coefficient

For given model formulation, parameterization and design space, the sensitivity of the

predicted response at x0 represented by f1 to a parameter of interest yi fi is measured by

the sensitivity coefcient:

PSC i x0

~

~

@df

sef^ i @f

1

1

where df1 and dfi are the studentized parameters dened in Eqs. (5) and (6).

Expressing Eq. (9) in terms of Y parameters yields

PSC i x0

~ i

sey^ i @Z~ 0 Y sey^ i DZ0 yi ;Y

seZ^ 0 @yi

seZ^ 0

Dyi

10

Table 1

Summary of parameter estimates for the MichaelisMenten model.

Parameter

Estimate

St. error

y1

y2

212.68

0.06411

6.94

0.008

1562

x0=0.02

(

1)

x0=0.06

x0=0.11

x0=0.22

x0=0.56

x0=1.10

8

2

1.5

0.5

0

(

2)

0.5

1.5

Fig. 1. Prole trace plots for the studentized predicted response parameter f1 at the six prediction points

x0=(0.02, 0.06, 0.11, 0.22, 0.56, 1.10) for the MichaelisMenten model.

~ i with respect to yi

where the operator DZ0 yi ; Y

[1]. Expressing the total derivative in terms of partial derivatives gives the following result:

~ i

DZ0 yi ;Yi yi @Z0

@Z0 @Y

11

Dyi

@yi @Yi Y~ i @yi

~ i =@yi

Using least squares estimation criterion, Sulieman et al. [1] showed that the term @Y

is given by

(

)

1 2

~ i

@Y

@2 Syi ;Yi yi

@ Syi ; Yi yi

12

0

~

@yi @Yi

@yi

@Yi @Yi

Y i

Pn

where Syi ; Yi yi i1 yi Zi yi ; Yi yi is the conditional least squares function.

Substituting Eqs. (11) and (12) into Eq. (10) yields

8

9

1

sey^ i <@Z0

@Z0

@2 S

@2 S =

13

PSC i x0

0

seZ^ 0 : @yi @Yi @Yi @Yi

@yi @Yi ~ ;

Y i

1563

Using rst and second order derivative information of the model function gY, Eq. (13)

can be shown to equal

PSC i x0

sey^ i

0

0

0

0

0

fv0 v V Vi e Vii 1 Vi vi D eg

seZ^ 0 i 0i i

14

where v0i is the ith component of the rst derivative vector v0 evaluated at x0; V i is an

n p1 matrix consisting of rst derivative vectors of gY with respect to Yi ; v0i is a

(p1) dimensional vector consisting of the elements in the row of V i which corresponds

to x0; V i i is the n (p1) (p1) array of the second derivatives of gY with respect

to Yi ; D is the n (p1) matrix of the second derivatives of gY with respect to Yi and

yi , and e is the n-element residuals vector. The quantities in Eq. (14) are evaluated at

~ i y^ i .

y^ i ; Y

The rst term in Eq. (14), sey^ i =seZ^ 0 v0i gives a scaled (studentized) measure of the

conventional marginal sensitivity coefcient. The second term in the equation,

0

0

0

0

0

sey^ i =seZ^ 0 fv0i Vi Vi e Vii 1 Vi vi D eg, represents an adjustment term

containing two components of information. The rst is the scaled marginal effects of Yi

on the predicted response at x0 through the derivative vector v0i . The second component of

information is about the co-dependency structure of the parameters Y measured by the

reaming part of the second term. It includes two sets of scaled co-dependencies: the pairwise

~ i via the term V 0 Vi e0 Vii 1 and correlations

correlations among the elements of Y

i

~ i via the term V 0 vi D0 e. Because second-order derivatives of gY are

between y^ i and Y

i

included in the co-dependency terms, model nonlinearity is accounted for by the prole-based

sensitivity coefcient, PSCi( x0), up to second-order derivatives.

The adjustment term in Eq. (14) incorporates the simultaneous changes in the parameter

values making PSCi(x0) global sensitivity measure while it is inherently local since it is

derivative-based. It is only when the parameter co-dependency structure and model

nonlinearity are insignicant that PSCi(x0) becomes equivalent measure to local sensitivity

coefcient. Under these facts, PSCi(x0) is called hybrid localglobal sensitivity measure.

Following the same philosophy, Sulieman et al. [2] extended the prole-based sensitivity

approach to the parameter estimation in multi-response regression models. A multiresponse regression model is expressed as

ykj fj xk ;Y zkj ;

k 1; . . . ;n j 1; . . . ;J

15

where now there are J response variables for the n experimental settings, zkj is the

disturbance term assumed to have a normal distribution with zero mean, for all k and j,

independent for different k but having a xed variancecovariance matrix R of dimension

J J when k is common. Using the determinant criterion by Box and Draper [16],

Sulieman et al. [2] dened the following sensitivity measure:

1 2

0

0

@g0

@g0 @2 dyi ;Yi yi

@ dyi ;Yi yi

PSCi x0

16

0

@yi @Yi ~

@yi @Yi

@Yi @Yi

Y i

where dY jZ Zj is the determinant function, the vector @g0 =@yi has J elements of the

rst-order partial derivatives giving marginal sensitivity coefficients of the J model

0

functions gY with respect to yi evaluated at x0. @g0 =@Yi is an J p1 matrix

consisting of the marginal sensitivity coefcients of the J model functions with respect to

0

Yi evaluated at x0. The matrix @2 d=@Yi @Yi is the (p1) (p1) sub-matrix of the

1564

Hessian matrix of dY, and @2 d=@yi @Yi is a ( p1)-element vector of the Hessian terms

corresponding to yi and Yi . These Hessian terms are computed using rst and second

order derivatives of model function with respect to parameters. The (r,s)th component is

given by

@2 d

0

jZ ZjtrUs trUr trUs Ur

@ys @yr

0

17

where

0

Ur Z Z1 Z Zr Zr Z;

Zsr

@2 Z

@2 gY

;

@ys @yr

@ys @yr

Zr

@Z

@gY

@yr

@yr

r;s 1; . . . ;i1;i 1; . . . ;p

The (p1) elements of the vector @2 d=@yi @Yi in Eq. (16) are obtained by taking s= i

and r=1,y,i1, i1,y,p in Eq. (17). Because the Hessian matrix is symmetric, there are

only (p1)p/2 distinct Hessian terms in the (p1) (p1) sub-matrix. This gives a total

number of Hessian terms to be evaluated as (p1)(p1)p/2=(p1)(p2)/2.

As in the single-response model case, Eq. (16) suggests that

PSCi x0 Marginal Sensitivity of yi adjustment term

18

The marginal sensitivity of yi , MSCi, in Eq. (18) is measured by the local sensitivity dened

by partial derivatives of model functions with respect to yi . The adjustment term,

measuring nonlinear co-dependencies among parameters up to second-order derivative

information, accounts for the overall impact of the simultaneous changes in the parameter

space. The derivative-based sensitivity measure, PSCi, stems its global property from the

adjustment term which in turns depend on the degree of parameter co-dependencies and/or

model nonlinearity.

3. Fourier amplitude sensitivity test (FAST)

FAST was rst formulated by Cukier et al. [5,6] and Schaibly and Shuler [8] to

investigate the sensitivity of the solutions of large scale chemical reaction systems to

uncertainties in rate coefcients. Cukier et al. [7] reformulated the procedure to facilitate its

applications to systems other than chemical reaction systems. The core feature of FAST

method is to reduce the dimension of the problem from p-dimensional parameter space to

one dimensional parameter space by suitably dening a search variable s using the

transformations:

yi gi ui ;

i 1; . . . ;p

19

ui Gi sinwi s;

i 1; . . . ;p

20

1565

P

wis are integer incommensurate frequencies, i.e., pi1 zi wi a0 for any integers zi . With

appropriate choice of G transformation, variation of s over the range prsrp generates a

curve which traverses the p-dimensional parameter space in a manner consistent with the

assumed probability distribution of Y. The set of frequencies, w=( w1, w2,y,wp) cannot be

truly incommensurate causing the Fourier coefcients associated with the ith parameter yi

to reect sensitivity of the model function not to yi only, but rather to possible

combination of yi and other parameters. To avoid or minimize this interference error, an

interference factor M is judiciously for a suitable nite set of integer frequencies. The

factor M postpones the interference to harmonics of order higher than M and therefore,

harmonics up to order M are included in the analysis. The order M is generally taken as

4 or higher in order to obtain sufcient accuracy of the calculations [17,18]. Cukier et al. [6]

tabulated frequency sets free of interferences to 4th order for up to 50 parameters.

The predicted response at x0, Z0 , becomes periodic in s and so it is expanded by a Fourier

series giving

Z0 s

1

X

Aq x0 sinqs Bq x0 cosqs

21

q1

where

Z

Arwi x0 1=2p

Z0 ssinrwi s ds

r 1;2; . . .

22

Z0 scosrwi s ds

r 0;1; . . .

23

p

Brwi x0 1=2p

p

Arwi x0 and Brwi x0 are the Fourier coefcients for the fundamental frequency wi and its

harmonics, r=1,2,yat x0.

Arwi x0 and Brwi x0 are computed numerically using nite summations. The number of

model evaluations Ns required for the analysis is at least 2 Mwmax1 [17].

The transformations ui in Eq. (20) are selected so that the two integrals (22) and (23) in

the s space are equal to the corresponding ones in Y space. Several transformations have

been proposed in the literature. The transformation proposed by Saltelli et al. [17] and used

by Haaker and Verheijen [24] for uniform distribution is applied here. It is given by

!

2 yui yli

ui Gi sinwi s

24

arcsinsinwi s

p yui yli

where yui and yli are the upper and lower bounds of the uniform parameter distribution.

The function gi in Eq. (19) is appropriately chosen to be

gi ui y^ i eui ;

1oui o1

25

y^ i is the unconditional estimate of the parameter yi using least squares criterion in singleresponse model or determinant criterion in multi-response model.

In the original formulation of FAST which is used here, uis are considered to be

independent random variables with respective probability distributions. Most recently Xu

and Gertner [19] proposed, using rank correlations, a procedure to extend the application

of FAST to models with correlated parameters.

1566

The authors of FAST centered their efforts on the rst-order Fourier sine coefcients,

Awi , corresponding to r=1 as sensitivity measures. They showed that

Awi x0

2

/@Z0 u=@ui S

bi

26

where bi is a distributional parameter for ui, and /@Z0 u=@ui S is the expected value of

@Z0 u=@ui over the u-space probability distribution. Cukier et al. [6] established a more

direct relation between the sensitivity measures Awi 0 , and the marginal sensitivity

coefcients @Z0 u=@ui . They showed, using moments representation of the probability

distribution and Taylor series expansion of model function, that

Awi x0

2

2

/@Z0 u=@ui S @Z0 u=@ui ju0 R

bi

bi

27

where @Z0 u=@ui ju0 is the value of the marginal sensitivity coefcient (MSC) when u=0

^ in Eq. (19)), and R is a remainder term containing the expected

(corresponding to Y Y

values of the higher-order terms in the Taylor series expansion of @Z0 u=@ui .

The remainder R is small only if the higher-order terms are essentially small, implying a

linear model in parameters. Thus Eq. (27) indicates that Awi are nonlinear sensitivity

measures that can be valid over broad ranges of parameter uncertainty.

Eq. (27) can be expressed as

Awi x0 Marginal Sensitivity of yi R

28

The parallel between Eqs. (28) and (18) is quite clear and indicates that the prole-based

sensitivity coefcient PSCi(x0) and Awi x0 are directly related to the marginal local

sensitivity coefcient while globally and simultaneously perturb parameter values and have

broader ranges of validity than the local analysis due to their nonlinear nature. Both are

hybrid localglobal measure that provide adjustment factors to the local sensitivity

assessment. The PSC measure adjusts the marginal sensitivity coefcient MSC for the

nonlinear co-dependencies among parameter estimates. Using second-order derivative

information of the model function, shown in Eqs. (14) and (17), PSC accounts for model

nonlinearity in two parts. The rst part lies in the second-order derivatives with respect to

the p1 parameters in the conditional estimation problem. The second part lies in the

second-order derivatives with respect to the parameter of interest and the remaining

parameters. The only second-order derivative information that is not incorporated in the

analysis is the one corresponding to the parameter of interest. This is because the PSC is a

rst-order sensitivity coefcient. On the other hand, the Fourier amplitude Awi adjusts the

marginal sensitivity coefcient for the full nonlinearity in the model function up to all

orders but not for the dependencies among parameters. It incorporates second-order

derivative information with respect to the parameter of interest which is not incorporated

by the PSC.

In examining the differences between the three sensitivity measures, the difference

between MSC and PSC values is directly attributed to the magnitude of the correction

term measuring the nonlinear dependencies among parameters. The difference between

MSC and Awi is attributed to the magnitude of R involving higher-order terms in

the Taylor series expansion without identication of particular terms that contribute the

most to the difference. On the other hand, the difference between PSC and Awi is not easily

1567

co-dependencies or because of model nonlinearity. PSC accounts for nonlinear

correlations among parameter estimates using second-order derivative information only.

It also does not account for second-order derivative information with respect to the

parameter of interest. Awi accounts for model nonlinearity using higher-order derivative

terms with respect to all parameters including interactions terms. It does not, however,

account for parameter estimate correlations. It should be pointed out that signicant

interaction terms in Awi are not necessarily associated with signicantly correlated

parameters. In addition, insignicant parameter correlations do not necessarily exhibit

insignicant interaction terms [20]. Separation of the effects of dependence and

nonlinearity can only be achieved if an assessment of the magnitudes of the secondorder terms in yi and higher orders in other parameters is carried out.

For sensible comparisons with MSC and PSC, Awi is scaled by the factor sey^ i =seZ^ 0 .

In the next section the three sensitivity measures, MSC, PSC and Aw are applied to

nonlinear parameter estimation of three model cases. Discussions and comparisons of the

sensitivity results obtained by the three measures are included.

4. Illustrative examples

Example 4.1. MichaelisMenten model. For each of the design points discussed in Example

2.1, the prole-based and marginal sensitivity coefcients for y1 and y2 in the Michaelis

Menten model in Eq. (7) calculated using Eq. (14). For the FAST numerical calculations, the

frequency set w=(11, 13) used by Cukier et al. [7] is utilized here. The order of interference for

w is M=6 giving a minimum number of model evaluation Ns to 157. We carried out 201

evaluations. The two Fourier sine coefcients, Aw1 and Aw2 , at each x0, are calculated using

201 model evaluations. The results are summarized in Fig. 2 where the dashed, solid and

dotted lines join, respectively, the values of Awi , MSCi and PSCi. For concise graphical

representation and convenience in comparing the three measure of sensitivities, the scaled

values of Awi are re-scaled by a factor of 101 for y1 and by 102 for y2 .

The values of MSCi (solid curves) and Awi (dashed curves) are very close to each other

and they are larger in magnitude than the PSCi values (dotted curves) of both parameters

for the majority of the data points. The nonlinear behaviors of y1 and y2 in this Michaelis

Menten model formulation is mild as demonstrated by Bates and Watts [3]. This mild

nonlinearity implies insignicant nonlinear effects in Awi causing FAST results to be very

close to the marginal analysis results. The parameter estimate correlations accounted for

by the PSCi exhibit substantial impact on the sensitivity behavior of the predicted

responses at low levels of concentration for y1 and at high levels of concentration for y2 .

According to the prole-based sensitivity curves, the largest value of PSC1 is observed at

the largest value of x (observation number 12), indicating that y1 is most inuential on the

predicted responses at large x values in the observed range. This result is coherent because

y1 is the horizontal asymptote for the MichaelisMenten equation (7), and therefore the

predicted responses at large values of the regressor variable depend largely on y1 . An

analogous result is obtained for the effect of y2 at small values of x, suggesting that y2 is

strongly inuential when predicting at small x values in the data range. y2 represents the

half-concentration, i.e., the value of x at which the velocity is one-half its ultimate value.

This usually occurs at low values of x.

1568

1 sensitivity

1.5

1

0.5

0

0.5

1

6

7

Observation Number

10

11

12

6

7

Observation Number

10

11

12

2 sensitivity

0.5

0

0.5

1

1.5

2

Fig. 2. Parametric sensitivities for the MichaelisMenten model. The dashed, solid and dotted lines join the values

of Awi , MSCi and PSCi, respectively.

Table 2

Summary of parameter estimates for the double exponential model.

Parameter

Estimate

St. error

y1

y2

y3

y4

y5

s2 1:95 106 with 28 degrees of freedom

0.375

1.936

0.013

1.465

0.022

0.002

0.220

0.0004

0.221

0.0009

Obviously these relations between the parameters and the predicted responses are not

possible to uncover using either MSCi or Awi .

Example 4.2. Osborne [21] tted the model

Ey y1 y2 expy3 x y4 expy5 x

29

to data supplied by A.M. Sargeson of the Research School of Chemistry in the Australian

National University. The results of model tting using least squares estimation are given in

Table 2. A common feature of tting linear combination of exponentials is the pronounced

parameter estimate co-dependencies that are induced by model formulation. In this

1569

addition to severe parameter correlations, model (29) suffers from pronounced parameter

nonlinearities as shown by Sulieman et al. [1]. Using Eq. (14) prole-based and marginal

sensitivity coefcients for the ve parameters were calculated for each of the 33 cases

considered in the data.

The Fourier sine coefcients Awi x0 were calculated for each parameter at all design

points using the frequency set w=(11, 21, 27, 35, 39) reported in Schaibly and Shuler [8].

The order of interference is M=4 and the corresponding minimum number of model

evaluations is Ns=313. We used Ns=600. The resulting values of the three scaled measures

are displayed in Fig. 3.

While the MSC values suggest some strong sensitivity relations for all parameters at

most of x0, the PSC values (dotted curves) show no distinguishing strong effects of any of

the parameters on the predicted response at any x0 [23]. FAST results for the parameters

y1 , y3 and y5 shown in the plots of the rst column are substantially similar to the

corresponding PSC values. Aw1 , Aw3 and Aw5 are also nearly zero at all x0. This to say that

the nonlinearity effects of these three parameters on the prediction sensitivity are more

6

200

150

100

50

0

5

10

15

20

25

30

10

15

20

25

30

10

15

20

25

30

200

20

100

0

40

5

10

15

20

25

30

10

15

20

25

30

100

40

30

20

10

0

Fig. 3. Parameter sensitivities for the exponential model. The dashed, solid and dotted lines join the values of Awi ,

MSCi and PSCi, respectively. The plots in the rst column represent sensitivities with respect to y1 , y3 and y5 . The

second column represents sensitivities with respect to y2 and y4 .

1570

dominant than their co-dependencies accounted for by PSC analysis. On the other hand,

the values of Aw2 and Aw4 show some signicant sensitivities for small x0. According to Awi

and PSCi values, signicant relationships between the parameters and the predicted

responses are not easily determined due to the extreme levels of parameter nonlinearities

and/or co-dependencies. Signicant relationships, if any, can be made clearer when these

parameter nonlinearities and/or co-dependencies are reduced by using appropriate

parameter transformations.

Example 4.3. The model here is a multi-response regression model taken from Bates and

Watts [22]. The three expected responses are linear functions in two parameters,

f1 xk1 ;b b0 b1 xk1 ;

where xkj is the value of the jth regressor variable at the kth experimental settings,

k=1,2,y, n=8 and j=1,2,3. The interesting feature of this simple linear multi-response

model is that there is no nonlinearity associated with the parameters, however, the use of

the determinant criterion produces a nonlinear estimation problem. Using the data given

by Bates and Watts [11], the parameters were estimated by minimizing dY in Eq. (16).

The resulting values of the parameter estimates are b^ 0 ; b^ 1 0:408; 2:555 with an

optimum determinant value of 569 and corrb^ 0 ; b^ 1 0:968. The estimated variance

covariance matrix is

2

3

0

0:962 1:130 1:231

^ Z

^

Z

6

7

2:464 1:166 5

30

S^

4

8

2:660

The vector-valued prole-based and marginal sensitivity coefcients for the two

parameters were calculated for each of the eight cases using Eq. (17). The frequency set

used for the FAST analysis is the same as the one used in Example 4.1, namely, w=(11, 13)

with Ns=201. Aw2 was re-scaled by a factor of 102 for the graphical t. The results are

shown in Fig. 4. Because the model function is linear in b0 and b1 , the marginal sensitivity

calculations and FAST produce equal or nearly equal sensitivity value at all design points

for both parameters. The insignicant differences in the values of the two sensitivity

measure can be partly attributed to the error by the numerical integrations used in FAST

method. When the calculations are adjusted for the correlations between b^ 0 and b^ 1 , prolebased sensitivity analysis produces smaller values than corresponding Awi and MSCi at all

design points for both parameters. The PSC results also suggest different sensitivity

relations of the three predicted responses to the two parameters from those indicated by

FAST and marginal analysis. For example, according to the marginal sensitivity

coefcients and FAST, the predicted response Z^ 1 (top row in Fig. 4) exhibits high

sensitivity with respect to both parameters at large x0. For the same x0, the prole-based

sensitivity coefcients decline to almost zero indicating very minimal sensitivity effects.

Similar conclusions can be drawn for the sensitivities of Z^ 2 and Z^ 3 to both parameters.

5. Conclusions

The prole-based sensitivity coefcient introduced by Sulieman et al. [1] has been

derived in this paper using the concept of model re-parameterization. In particular, the

predicted response re-parameterization has been shown to directly lead to the formulation

6

1 (x0)

2

0

2

0

0

2 (x0)

6

4

2

0

2

0

0

3 (x0)

1571

2

6

4

2

0

2

0

0

2

8

0

Observation Number

Fig. 4. Parametric sensitivities for the multi-response linear model. The dashed, solid and dotted lines join the

values of Awi , MSCi and PSCi, respectively. Plots in the rst column correspond to the sensitivities to b0 and in the

second column to the sensitivities to b1 .

coefcient, dened by the total derivative of the predicted response with respect to a

parameter of interest, is the ability to handle simultaneous changes in parameter values

while accounting for parameter co-dependencies. The resulting derivative-based sensitivity

measure is considered hybrid localglobal measure because of the global perturbation

scheme used to vary the parameter values.

Comparisons between prole-based sensitivity, conventional marginal sensitivity and

classical Fourier amplitude sensitivity test (FAST) have also been conducted in this paper.

The measures provided by these three methods are rst-order sensitivity measures that

have different degrees of accuracy and validity. The marginal sensitivity coefcient is a

local measure that fails to account for model nonlinearity and parameter dependencies.

The classical FAST is a global method that accounts for model nonlinearity up to all

derivative orders but fails to account for parameter estimate dependencies. This article

focuses on the Fourier sine amplitude as a rst-order sensitivity measure. The results show

that the measure is directly linked to the local marginal sensitivity coefcient averaged over

all ranges of parameter uncertainties and so it can also be considered hybrid localglobal

1572

sensitivity measure. The prole-based sensitivity method has been shown to provide a

more reective picture of the sensitivity behavior because it accounts for both parameter

estimate correlations and model nonlinearity up to second-order derivative information.

Three illustrative model examples with different degrees of nonlinearity and parameter

co-dependencies have been used to demonstrate the comparisons.

The comparison exercise carried out in this work has shown that using an appropriate

analysis procedure, computing the sensitivity coefcients in the nonlinear parameter

estimation and plotting them against the design points provides a concise visual way to

investigate the sensitivity behavior of the parameter estimation problem.

Acknowledgment

The authors gratefully acknowledge the nancial support of the American University of

Sharjah, United Arab Emirates.

References

[1] H. Sulieman, P.J. McLellan, D.W. Bacon, A prole-based approach to parametric sensitivity analysis of

nonlinear regression models, Technometrics 43 (4) (2001) 425433.

[2] H. Sulieman, P.J. McLellan, D.W. Bacon, A prole-based approach to parametric sensitivity in

multiresponse regression models, Computational Statistics and Data Analysis 45 (2004) 721740.

[3] D.M. Bates, D.G. Watts, Nonlinear Regression Analysis and its Applications, Wiley, New York, 1988.

[4] H. Sulieman, I. Kucuk, J. McLellan, Parametric sensitivity: a case study comparison, Computational

Statistics and Data Analysis 53 (7) (2009) 26402652.

[5] R.I. Cukier, C.M. Fortuin, K.E. Shuler, A.G. Petschek, J.H. Schaibly, Study of the sensitivity of coupled

reaction system to uncertainties in rate coefcients. I, Theory, Journal of Chemical Physics 59 (8) (1973)

38733878.

[6] R.I. Cukier, J.H. Schaibly, K.E. Shuler, Analysis of the approximations, III, Journal of Chemical Physics 63

(3) (1975) 11401149.

[7] R.I. Cukier, H.B. Levine, K.E. Shuler, Nonlinear sensitivity analysis of multiparameter model systems:

review, Journal of Computational Physics 26 (1978) 142.

[8] J.H. Schaibly, K.E. Shuler, Study of the sensitivity of coupled reaction system to uncertainties in rate

coefcients. II. Applications, Journal of Chemical Physics 59 (8) (1973) 38793888.

[9] G.J.S. Ross, Exact and approximate condence regions for functions of parameters in nonlinear models, in:

COMPSTAT 1978, Physica-Verlag, Wien, 1978, pp. 110116.

[10] G.J.S. Ross, Uses of nonlinear transformations in nonlinear optimisation problems, in: COMPSTAT 1980,

Physica-Verlag, Wien, 1980, pp. 381388.

[11] D.M. Bates, D.G. Watts, Parameter transformation for improved approximate condence regions in

nonlinear least squares, Annals of Statistics 9 (1981) 11521167.

[12] G.P.Y. Clarke, Approximate condence limits for a parameter function in nonlinear regression, Journal of

the American Statistical Association 82 (397) (1987) 221230.

[13] P.M. Doran, Bioprocess Engineering Principles, Elsevier Academic Press, London, UK, 1995.

[14] L.A. Alcazar, J. Ancheyta, Sensitivity analysis based methodology to estimate the best set of parameters for

heterogeneous kinetic models, Chemical Engineering Journal 128 (2007) 8593.

[15] T.G. Doeswijk, K.J. Keesman, Linear parameter estimation of rational biokinetic functions, Water Research

43 (2009) 107116.

[16] G.E.P. Box, N. Draper, The Bayesian estimation of common parameters from several responses, Biometrika

52 (1965) 355365.

[17] A. Saltelli, S. Tarantola, K. Chan, A quantitative model-independent method for global sensitivity analysis

of model output, Technometrics 41 (1999) 3956.

[18] A. Saltelli, K. Chan, E.M. Scott, Sensitivity Analysis, Wiley, New York, USA, 2000.

[19] C. Xu, G. Gertner, Extending a global sensitivity analysis technique to models with correlated parameters,

Computational Statistics and Data Analysis 51 (12) (2007) 55795590.

1573

[20] A. Saltelli, M. Ratto, S. Tarantola, F. Campolongo, Sensitivity analysis practices: strategies for model-based

inference, Reliability Engineering and System Safety 91 (2006) 11091125.

[21] M.R. Osborne, Some Aspects of Non-linear Least Squares Calculations, Numerical Methods for Nonlinear

Optimization, Academic Press, London, 1972.

[22] D.M. Bates, D.G. Watts, A generalized GaussNewton procedure for multi-response parameter estimation,

SIAM Journal on Scientic and Statistical Computing 7 (1) (1987) 4955.

[23] H. Sulieman, Improved local sensitivity measures for regression models with correlated parameters, in:

Proceedings in Computational Statistics 18th Symposium, Porto, Portugal, 2008.

[24] M.P.R. Haaker, P.J. Verheijen, Local and global sensitivity analysis for a reactor design with parameter

uncertainty, Chemical Engineering Research and Design 82 (A5) (2004) 591598.

- Practices Barriers and Benefits of Using Risk Management Approaches in Selected Hong Kong Industries 1997 International Journal of Project ManagementUploaded byluisbmwm6
- Parameters of Rock MassUploaded byManmohit Singh
- 034Uploaded byMohammed Almusawi
- 037Uploaded byalvamu3354
- Sensitivity Analysis for Noise MappingThe Department for EnvironmentUploaded byDavid Benavides
- A2.pdfUploaded byMarcela Muñoz Pinzon
- liu2017.pdfUploaded byadriansyah170982
- 3rdNSCportland3Uploaded byGere Lu
- RISK OF EXTREME EVENTS IN A MULTIOBJECTIVE FRAMEWORKUploaded bySusana
- Uncertainty Analysis in Air Dispersion ModelingUploaded byIbnu Riswan
- BAUploaded byChanukya Griddaluru
- LP SensitivityUploaded byRoshan Mohammad
- POWER MEASUREMENT AS SENSITIVITY ANALYSIS A UNIFIED APPROACHUploaded byAna Rivera Castañon
- Sensitivity AnalysisUploaded byMarcin Kot
- 132441-356985-1-PBUploaded byJamiu Ariremako
- saruman fieldUploaded byalex9and9ru9
- Research AnalyticsUploaded byDreamtech Press
- Probability and Statistics in Engineering by William W. Hines, Douglas C. Montgomery, David M. Goldsman, Connie M. BorrorUploaded byShivaRoonval
- Analysis HW #12Uploaded byBlake Rupard
- Smith Et Al - Mental MistakesUploaded bySean Nanivadekar
- Decision Support Systems and Executive Support Systems (1)Uploaded bySakshi Jindal
- High Low MethodUploaded byCharles Kioko Mulonzi
- calculus - Find$_lim_limits_{x_to 0+} x_ln(_sin(x)) =__$ - Mathematics Stack ExchangeUploaded bybalajiseshikanthvarma
- Tutorial MinitabUploaded byjc141168
- 0063-03-0006-0004-3Uploaded bySalma Chakroun
- Taylor Chap 3 AnswersUploaded byms_1226
- OPTIMAL MACHINE REPLACEMENT STRATEGIES USING DYNAMIC PROGRAMMING RECURSIONS: A CASE STUDY OF NASCO HOUSEHOLD PRODUCTS LTD., JOSUploaded byMohammed Hamed
- SEG-1989-0593Uploaded bySaed
- DEM_CEP_2012Uploaded byVenkat Reddy Yedulla
- The Major Role Accountants PlayUploaded byZlatan Mesanovic

- Dialnet-EstrategiasPracticasParaModeladoYControlDeUnSepara-3196517Uploaded byAlexander
- 8 Principio Activo1Uploaded bysam884
- Circular tarifas cannabis 2019.pdfUploaded byAlexander
- decreto-162-de-2004Uploaded byEugenia Garcia
- Science of Botanicals.pdfUploaded byAlexander
- diseñoUploaded byDiana Libertad Castro Salvador
- ROFA2009EuroDistUploaded byAlexander
- 3 Material de EmpaqueUploaded byAlexander
- 2013empresaserviciospublicosdecalotoesp.pdfUploaded byAlexander
- 26944075 Manual Basico Del Cultivo de La MarihuanaUploaded byHidro Alegre
- 26944075 Manual Basico Del Cultivo de La MarihuanaUploaded byHidro Alegre
- 1.Plan de Manejo AmbientalUploaded byPaolo Torres
- Entrega-y-uso-de-EPI-1.3.docxUploaded byFabian Esteban Gutierrez Gonzalez
- gth-p-17.docUploaded bynohemi
- Presion Vapor PentanoUploaded byAlexander
- Liu 2013Uploaded byAlexander
- Settling Tanks PptUploaded bypandeysekhar
- Explicación Molibdeno y HierroUploaded byAlexander
- InstruccionesUploaded byAlexander
- 20110224 Thompson JeffUploaded byAlexander
- FLUENT Tutorial 1 - Developing Flow In a Channel.pdfUploaded byAlexander
- FLUENT Tutorial 1 - Developing Flow In a Channel.pdfUploaded byAlexander
- 004660 Cme 73 2014 Opc Petroperu BasesUploaded byAlexander
- D445.4712 (17a)Uploaded byAlexander
- 44-74448bro_lambda.pdfUploaded byAlexander
- Lube Oil Re-RefiningUploaded byAbdul Wahab
- 5991-4304EN_Cary8454_BrochureUploaded byAlexander
- 5991-4304EN_Cary8454_BrochureUploaded byAlexander
- FLUENT Tutorial 2 - Mixing ElbowUploaded byAlexander
- iso_17025Uploaded byLorena Molina Calderon

- Unani Pharmacopoeia of India Part II Vol 3Uploaded byazeem dilawar
- b.inggUploaded byElisa Kartika
- B. L. S. LL. B. Notes on Evidence - Hearsay EvidenceUploaded byeaglelegal
- Curso de japonesUploaded byDanna MoCa
- Writing a Successful Paragraphan Extended Paragraph and a Five- Paragraph CompositionUploaded byJesuv Cristian Clete
- CloudForms 1.1 Cloud Engine User Guide en USUploaded byVijay Pratap Singh
- Cordillera ArchitectureUploaded bySaw Hack
- edtpa lesson plan 1Uploaded byapi-400202890
- ديوان الصحابي حسان بن ثابت الأنصاري, طبعة 1910Uploaded byQmralzman
- Fixed Pros CaseUploaded byAmniAzmi
- Program & Presentation ScheduleUploaded byShaidatul Fareha
- Activate Document SplittingUploaded byVincentvke
- Design of Wind Turbine Tower and Foundation Systems- OptimizationUploaded byAman Sharma
- document.pdfUploaded byOjank16 Labala
- Stamping Die Solution Overview Updated 01142011Uploaded byHerberPeraza
- ChristmasUploaded byJilpa Dedhia
- 1.5 vs 3 vs 7 Tesla in Abdominal MRI.pdfUploaded byFrancesco Magoga
- Data Integrity Proofs in Cloud Storage AbstractUploaded bysukanyaraju
- 71542622-00-Crimpro-Bail-Compiled-Digest.docUploaded byErika Mariz Sicat Cunanan
- 360 Degree Feedback Booklet-finalUploaded bykhaledhaddad
- NTE Int AchievementTest1Uploaded byFernandoCóndorTapia
- Desert Labyrinth Lines, Landscape and Meaning Nazca PERUploaded byAlberto Navas
- 100 Winning Resumes for 100000 Jobs ~ RaGa@TMTUploaded byaravindyellow
- Hart H. L. a., The Concept of LawUploaded byAdelina Alexandra
- observation report lesson 1 descriptive writing revUploaded byapi-143111550
- The World as a Reality and Representation Rodrigo AlonsoUploaded byOrneb
- Status and Trend of HD in N.E. Region of IndiaUploaded byPinak Deb
- Harvard Government 90 SyllabusUploaded byJ
- UserGuide EnUploaded byAlex Pinhal
- Pascal's Triangle The Royal Society Western History the Nobel Foundation the Fields Institute A New YearUploaded byMark Richard Hilbert(Mark Richard Rossetti)