Nonlinear Regression

Non Linear Regression Yi = f(Fxi) + Ii f(Fx
Marco Lattuada
Swiss Federal Institute of Technology - ETH Institut fr Chemie und Bioingenieurwissenschaften ETH Hnggerberg/ HCI F135 Zrich (Switzerland) E-mail: lattuada@chem.ethz.ch http://www.morbidelli-group.ethz.ch/education/index
Puromycin
Description: Puromycin is an antibiotic used by scientists in bio-research to select cells modified by genetic engineering. Mechanism of action: This is described by the Michaelis-Menten model for enzyme kinetics, which relates the initial velocity on an enzymatic reaction to the substrate concentration x trough the equation:
U1 x f x, ! U2 x
Marco Lattuada Statistical and Numerical Methods for Chemical Engineers Nonlinear Regressions Page # 2
Puromycin Kinetics
Raw Data
22 0 20 0
Reaction Velocity [counts/min ]
18 0 16 0 14 0 12 0 10 0 80 60 40 20 0 0 0.2 0.4 0.6 0.8 1 1.2
The model:
U1 x f x, ! U2 x
S ubs tra te C o nc e ntr a tion [ppm ]
Model Linearization
Puromycin Kinetics:
U1 x f x, ! U2 x
Model Rearrangement:
1 f x,
U2 x 1 U2 1 ! ! U1 x U1 U1 x
Linearized Model:
f y,
! F1 F2 y
Model Linearization
L ine a rize d Ra w D a ta
0 .0 2 2 0 .0 2 0 .0 1 8 0 .0 1 6 0 .0 1 4
1 /f
0 .0 1 2 0 .0 1 0 .0 0 8 0 .0 0 6 0 .0 0 4 0
Regression Line F1 = 0.0051072 F2 = 0.00024722

5 10 15 20 25 30 35 40 45 50 55
1 /x
Model Linearization
220 200
R e a ctio n V e lo c ity [c o u n ts /m in2 ]
180 160 140 120 100 80 60 40 20 0 0 0.2 0.4 0.6 0.8 1 1.2
Regression from linearized model
U1 = 195.8 U2 = 0.048407
S ubs tra te C onc entra tion [ppm ]
Puromycin Kinetics
Raw Data
22 0 20 0
Reaction Velocity [counts/min ]
18 0 16 0 14 0 12 0 10 0 80 60 40 20 0 0
U The model:
U1 x f x, ! U2 x
U
0.2 0.4
Linearized model is needed to estimate U2

0.6 0.8 1 1.2
S ubs tra te C o nc e ntr a tion [ppm ]
Nonlinear Regression
Object To minimize the objective function
S ( ) ! yi f xi ,
i !1
where n is the number of observations, yi the responses, xi is the vector of the observations, U the vector of the parameters and f(xi,U) the nonlinear model function. It is possible to plot the objective function S(U) as a function of the parameter values, in order to reveal the presence of a minimum.
Objective Function S(U) S(U

Contour plot of S(q)
0.1
50 00
40
50
0 30 0
25 00
00
20 0
0 25 0
0.09
12
75
00
10
00
18 0 0
25 0
16 00
20 00
50
00
40
0 1 4 013 50
00
13 5
00
25
U2
50
14
16
00
16
00
00
40 50 00
25 00 30 00
0.07
18
75
30
0.06
50 00
00
12
13
50 13 14 00
00
20
18 00
00
00
20
0.05
16 0 0
18 0 0
0 40
50 00
Minimum
75 00
10 00 0
40
00
20 0 0
25 0
0
3
000
0.04
30 00
Estimated value of U from linearization

40 0 0
75 0
15
000
50 0
40 00
0.03 18 0
10 0
00
12 5
00
17 5
00
20 0
00
19 0
20 0
21 0
U1
22 0
23 0
00
00
00 13 0 12 5
1200
00
18 0 0
30
0.08
30
00
20
00
0
00
25 00
24 0
Minimization of S(U) S(U

Model linearization:
f i xi ,
where:
} fi xi , 0
j !1
xfi xi , xU j
U0, j
xfi xi ! xU1 U 2 xi
so the residuals are:
xfi U1 xi ! 2 xU 2 U2 xi
! f y } f 0 y J 0
S( ) !
T
! 0 J 0(
J0 = Jacobian
Search for minimum with Gauss-Newton method:
J 0T J 0 ( ! J 0T
GaussGauss-Newton Method Applied to S(U) S(U

0 .1
50 00
40
50
0 30 0
25 00
00
20 0
25 00
0 .0 9
12
75
00
18 0 0
25 0
16 00
10
00
20 00
50
00
40
0 1 4 0 13 50
00
13 5
00
25
U2
50
14
16
00
16
00
00
40 50 00
25 00 30 00
0 .0 7
18
75
30
0 .0 6
50 00
00
12
13
50 13 14 00
00
20
18 00
00
00
20
0 .0 5
16 0 0
18 0 0
20 0 0
25 0
0 30 0
0 .0 4
30 00
40 0 0
75 0
50 0
0
10 0 00
Convergence path of GaussNewton Method (U1)opt = 212.66 (U2)opt = 0.064091

0 40
00
40
75
00
00
50
10
00
15
000
0 .0 3 1 80
40 00
12 5
00
17 5
00
20 0
00
1 90
2 00
2 10
U1
2 20
2 30
00
00
00 13 0 12 5
1200
00
18 0 0
30
0 .0 8
30
00
20
00
0
00
25 00
2 40
Nonlinear Regression
2 20 2 00
R e a c tio n Ve lo c ity [c o u n ts /m in2 ]
1 80 1 60 1 40 1 20 1 00 80 60 40 20 0 0 0 .2 0 .4 0 .6 0 .8 1 1 .2
Nonlinear Regression Regression from linearized model
S ubs tra te C onc e ntra tion [ppm ]
Ellipsoidal Confidence Region

The ellipsoidal confidence region can be evaluate from the linearized model around the point , which is the vector of the parameters for which the objective function has a minimum. In practice, every vector of the parameters U which satisfies the following condition:
J J e ps F ( p, n p, E)
T 2
is within the confidence interval, where n is the number of observations, p the number of parameters and s the standard deviation:
s 2 ! S / n p
Ellipsoidal Confidence Region

0 .1 Ucap 0 .0 9 9 9% 9 5% 9 0%
0 .0 8
0 .0 7
U2
0 .0 6 0 .0 5
E
0 .0 4 0 .0 3 1 80 1 90 2 00 2 10 2 20 2 30 2 40
U1
True Confidence Region for Parameters

The real confidence region can be estimated by plotting the region of space for which: p F p, n p, E S e S 1 n p
0.12 0.11 0.1 0.09 0.08 Uca p 95% 90% 99% C on tours
U
0.07 0.06 0.05 0.04 0.03 180
190
200
210
220
1
230
240
250
Matlab Nonlinear Regression Routine

First, create a function providing the residuals for the n observation as a function of the parameter values:
Then, use the routine 'nlinfit';
TukeyTukey-Ancombe Plot
30 25 20 15
R e s id u a ls
10 5 0 -5 -10 -15 40
60
80
1 00
1 20
1 40
1 60
1 80
2 00
2 20
F itte d y
Normal Plot
N o rm al Prob a bility Plot
0 .9 5 0 .9 0
0 .7 5 P rob a bility
0 .5 0
0 .2 5
0 .1 0 0 .0 5 -10 -5 0
>> normplot(r)
5 D ata 10 15 20 25
Matlab Estimation of Parameter CI

Parameter confidence interval can be estimated by Matlab as follows:
The confidence interval can be estimated using the following Matlab GUI:
t-Profiles
Goal: verify weather one parameter can have a certain value, as compared to the optimum value Method: let be Uk the parameter to be investigated. Let us fix one value of Uk, and then optimize for all the other parameters. Let us then compute:
X U k ! sign U k U k
S
n p
where 5k is the set of 5 values with Uk kept constant. The probability distribution of X(Uk) is a t-Student distribution with n-p degrees of freedom
t-Profiles
t-Profile for U1
0 .8
-5 1 80
1 90
2 00
2 10
U1
2 20
2 30
2 40
0 .6 2 50
Probability
X (U1)
t-Profiles
t-Pro file for U 2 10 1
0 .9
0 .8
-5
0 .7
-10 0 .0 3
0 .0 4
0 .0 5
0 .0 6
0 .0 7
U2
0 .0 8
0 .0 9
0 .1
0 .6 0 .1 1
P ro b a b ility
X (U2 )
Combined t-Profiles t0 .1 1 0 .1 0 .0 9 0 .0 8
U2
0 .0 7 0 .0 6 0 .0 5 0 .0 4 0 .0 3 1 80
1 90
2 00
2 10
U1
2 20
2 30
2 40
2 50
BOD Regression
Biochemical Oxygen Demand (BOD) refers to the amount of oxygen that would be consumed if all the organics in one liter of water were oxidized by bacteria and protozoa (ReVelle and ReVelle, 1988). The model considered here is:
y ! U1 1 e U2 x
The response variable (y) is biochemical oxygen demand in mg/l. The predictor variable (x) is incubation time in days
Raw Data
30
25
Expected behavior
20
B O D [m g /m l]
15
10
y ! U111 eUx2 x y ! U1 e U2 y (0) ! 0 y (g) ! U1
y d U1U2 e U2 x ! (0) y d ! U1U2 (g yd ) ! 0

8
Tim e [days]

Contour plot of S(U)
5
1000 1250 500 400 750 300 1750
400
30 0
1250
20 00
20 0
4
20 0
1000
75 0
50 0
3
10 0
10 00 12 50
15 00
500
400
30 0
17 5 0
750
40 0
U2
300
90 80
10 0
1250
20 0 0
20 0
75 0
60 50
70
10 00
20 0
50
30
40
10 0
60 40 10 90 20 00 50 0 00 40 0 75 0 10 1500 1750 2000
15 0
90
40 0
70
75 0
50 0
30 0
80
50
8090
30 0 20 0
60 30 0
40 0
70 80 12 50
10 0 0
10 0 8010 0 20 0 75 0
12 5 0
70
0
17 50 00 15 20 0 0
12 50
10 00 1500 1750 2000
12 50
50 0 90 50 0 00 40 0 10 1500 1750 2000
75 0
Minimum
5 10 15
-1 0
U1 Estimated value of U
20
25
30
35
40

Contour plot of S(U)
5
1000 1250 500 400 750 300 1750
400
30 0
1250
20 00
20 0
4
20 0
1000
75 0
50 0
3
10 0
10 00 12 50
15 00
500
400
750
30 0
17 5 0
40 0
U2
300
90 80
10 0
1250
20 0 0
20 0
75 0
60 50
70
10 00
20 0
50
30
40
10 0
50 8090 60 40 10 90 20 00 50 0 00 40 0 75 0 102000 1500 1750 30 0 20 0
60 30 0
15 0
90
40 0
70
75 0
50 0
30 0
80
40 0
70 80 12 50
10 0 0
10 0 8010 0 20 0 75 0
12 5 0
70
0
17 50 00 15 20 0 0
12 50
10 00 1500 1750 2000
12 50
50 0 90 50 0 00 40 0 10 1500 1750 2000
75 0
-1 0
10
15
U1
Path followed by the 2 0Gauss-Newton method 4 0 25 30 35
Regressed Model
30
Regressed Experimental Points

25
Experimental Points
20
B O D [m g /m l]
15
10
Regressed Model U1 = 19.1; U2 = 0.531
Tim e [days]
Confidence Region for the Parameters

5 Ucap 4 9 9% 9 5% 9 0%
True Confidence Region
U2
Estimated Minimum of S
Confidence-1Region 0 5 10 from Model Linearization
15
U1
20
25
30
35
40
Confidence Region for Response

30
25
20
B O D [m g /m l]
15
10
1-E confidence region for the response

0 1 2 3 4 5 6 7 8
Tim e [days]
Matlab Regression
30
25
20
B O D [m g /m l]
15
10
Regressed Experimental Points by Matlab U1 = 19.1; U2 = 0.531

0 1 2 3 4 5 6 7 8
Tim e [days]
TukeyTukey-Ancombe Plot
4
R e sid u a ls
-1
-2
-3 6
10
12
14
16
18
20
F itted y
Normal Probability Plot

N orm al P robability Plot
0 .9 5 0 .9 0
0 .7 5
P ro b a b ility
0 .5 0
0 .2 5
0 .1 0 0 .0 5 -2 -1 0 1 2 3
D ata
t-Profiles
10
t-Profile for U1
0 .8
-1 0 10
15
20
U1
25
30
35
0 .6 40
P ro b a b ility
X (U1 )
2 .5
t-Profile for U2
2 2 0 .9
U2
0 .8
-2 0 .5
0 .7
0 10
15
20
U1
25
30
35
40
-4
0 .5
U2
1 .5
0 .6 2 .5
P ro b a b ility
1 .5
X (U )

Nonlinear Regression

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nonlinear Regression

Uploaded by

Copyright:

Available Formats

Non Linear Regression Yi = f(Fxi) + Ii f(Fx

Reaction Velocity [counts/min ]

18 0 16 0 14 0 12 0 10 0 80 60 40 20 0 0 0.2 0.4 0.6 0.8 1 1.2

S ubs tra te C o nc e ntr a tion [ppm ]

Regression Line F1 = 0.0051072 F2 = 0.00024722

R e a ctio n V e lo c ity [c o u n ts /m in2 ]

Regression from linearized model

S ubs tra te C onc entra tion [ppm ]

Reaction Velocity [counts/min ]

Linearized model is needed to estimate U2

S ubs tra te C o nc e ntr a tion [ppm ]

Objective Function S(U) S(U

Estimated value of U from linearization

Minimization of S(U) S(U

Search for minimum with Gauss-Newton method:

GaussGauss-Newton Method Applied to S(U) S(U

Convergence path of GaussNewton Method (U1)opt = 212.66 (U2)opt = 0.064091

R e a c tio n Ve lo c ity [c o u n ts /m in2 ]

Nonlinear Regression Regression from linearized model

S ubs tra te C onc e ntra tion [ppm ]

Ellipsoidal Confidence Region

Ellipsoidal Confidence Region

True Confidence Region for Parameters

Matlab Nonlinear Regression Routine

Then, use the routine 'nlinfit';

Matlab Estimation of Parameter CI

y ! U111  eUx2 x y ! U1  e U2 y (0) ! 0 y (g) ! U1

y d U1U2 e U2 x ! (0) y d ! U1U2 (g yd ) ! 0

Objective Function S(U) S(U

10 00 1500 1750 2000

50 0 90 50 0 00 40 0 10 1500 1750 2000

Objective Function S(U) S(U

10 00 1500 1750 2000

50 0 90 50 0 00 40 0 10 1500 1750 2000

Path followed by the 2 0Gauss-Newton method 4 0 25 30 35

Regressed Experimental Points

Regressed Model U1 = 19.1; U2 = 0.531

Confidence Region for the Parameters

True Confidence Region

Confidence-1Region 0 5 10 from Model Linearization

Confidence Region for Response

1-E confidence region for the response

Regressed Experimental Points by Matlab U1 = 19.1; U2 = 0.531

Normal Probability Plot

You might also like

Non Linear Regression Yi = f(Fxi) + Ii f(Fx

y ! U111 eUx2 x y ! U1 e U2 y (0) ! 0 y (g) ! U1

y d U1U2 e U2 x ! (0) y d ! U1U2 (g yd ) ! 0