Professional Documents
Culture Documents
Colorado School of Mines CHEN403: F y Au
Colorado School of Mines CHEN403: F y Au
Introduction
Procedures to minimize the error between given data points and the calculated values from
an equation. The value that is to be minimized (the value of the objective function) is
typically the sum of the errors squared.
Linear Regression
All functional forms of the independent variables must be linearly combined:
N
i =1
i =1
y = ai ui ( x1 , x2 , , x M ) = ai ui
With linear regression we would like to minimize the square error of K data points (where
K N ). The objective function is:
2
F yk ai ui ,k .
k =1
i =1
The unconstrained minimum will occur at the extremum point all partial derivatives with
respect to the regression parameters ( a1 , a2 , etc.) will be zero. The partial derivatives are:
a j
Data Fitting
N
K
N a
2 yk ai ui ,k i ui ,k
i =1
i =1 a j
k =1
.
K
N
= 2 yk ai ui ,k ( u j ,k )
k =1
i =1
-1-
k ai ui ,k ( u j ,k ) = 0
k =1
i =1
K
N
yk u j ,k ai ui ,k u j ,k = 0
k =1
i =1
( y
k =1
k =1 i =1
( y
k u j ,k ) ( ai ui ,k u j ,k ) = 0
N
u j ,k ) ( ai ui ,k u j ,k ) = 0
k =1
i =1 k =1
N
K
y
a
( k j ,k ) i (ui ,k u j ,k ) = 0
k =1
i =1
k =1
K
u
)
i ,k
j ,k = ( yk u j ,k ) .
k =1
a (u
i
i =1
k =1
This is a system of N equations that can be expressed in the following matrix form:
Ua = Y
where the elements of the coefficient matrix are:
K
Ui , j = ( ui ,k u j ,k )
k =1
the elements of the unknown parameter vector are ai , and the elements of the right-handside vector are:
K
Yi = ( yk ui ,k )
k =1
and the system of equations to find the parameters in matrix form is:
Data Fitting
-2-
K
xk
k =1
K 2
xk
k =1
K
xN
k
k =1
k =1
K
xk2
k =1
K
xk3
k =1
N +1
k
k =1
3
k
4
k
k =1
K
x
k =1
k =1
yk
k =1
k =1
K
a
0
N +1
x k yk x k
a
k =1
1 k =1
K
K
=
a
.
x kN +2 2 yk x k2
k =1
k =1
aN
K
2N
y xN
x
k
k k
k =1
k =1
K
2
k
N + 22
k
k =1
N
k
This is a system of N equations that can be expressed in the following matrix form:
Ua = Y
where the elements of the coefficient matrix are:
K
Ui , j = ( ui ,k u j ,k )
k =1
the elements of the unknown parameter vector are ai , and the elements of the right-handside vector are:
K
Yi = ( yk ui ,k )
k =1
t
y = y 1 exp
t
where , , and possibly y are the unknown parameters that are to be determined from
a fit to data. The equation in this form is obviously non-linear the unknowns & are in
the exponential term. However, if we algebraically manipulate the equation we can get:
Data Fitting
-3-
y
t
t
= exp
y
y 1
t t .
ln 1
=
y
Y = a0 + a1t .
Where:
y
Y ln 1
1
1
a1 =
a1
a0 = a0 = 0 .
a1
The following Figures 1 & 2 show how the data points can be linearized. The curve in
Figure 2 shows that a straight line can be obtained, but this requires that we have a good
value for y . From the data it is pretty easy to see that y = 1 . However, it is not always
possible to come up with a good value for y before the regression is done. If a poor value
of y is used then the resulting transformed data will not form a straight line Figure 3
shows this.
Data Fitting
-4-
1.2
1.0
0.8
y'(t) 0.6
0.4
0.2
0.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Figure 1. Data Points for FOPDT Process With Step Change Forcing Function
0.0
-1.0
-2.0
-4.0
-5.0
-6.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Data Fitting
-5-
0.0
-1.0
y = 1.2
-2.0
y = 1.1
-4.0
y = 0.8
-5.0
y = 0.9
-6.0
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Non-Linear Regression
Much more general because the regression equation does not have to be in a specific form.
Set up an objective function of errors that should be minimized. If the regression equation
has a form of:
y = f ( x1 , x2 ,, x M , a0 , a1 ,, aN )
then to be consistent with linear regression the objective function should be of the form:
K
k =1
k =1
F 2k = yk f ( x1,k , x 2,k , , x M ,k , a0 , a1 , , aN ) .
2
Here the parameters a0 , a1 , , aN are the model parameters & they are to be adjusted to
minimize the value of this objective function.
For example, non-linear regression could be performed directly on the response curve of
an FOPDT driven by a step change by creating an objective function of the form:
Data Fitting
-6-
t
F = yk y 1 exp k
k =1
y*
t <
y (t ) = *
t
y + y 1 exp t
where , , and y are the unknown parameters that are to be determined from a fit to
data.
Note that this data was generated from a 3rd order process models response to a
step change:
y =
10
s (5s + 1 )( 4s + 1 )( 0.4s + 1)
Note that the Taylor series approximation FOPDT model would be:
1
10
10e 4.4 s
=
y =
.
s (5s + 1)( 4s + 1)( 0.4s + 1 ) 5s + 1
and the Skogestad FOPDT model would be:
1
10
1 10e (4/2+0.4)s
10e 2.4 s
y =
.
=
=
s (5s + 1)( 4s + 1)( 0.4s + 1 ) s 5 + 1 4 s + 1 7s + 1
Data Fitting
-7-
t
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
t
0.9
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
y (t )
50.08
50.11
50.28
50.53
50.83
51.17
51.54
51.93
52.33
t
5
6
7
8
9
10
15
20
25
y (t )
52.74
53.55
54.32
55.04
55.70
56.29
58.34
59.30
59.72
t
30
35
40
45
50
55
60
y (t )
59.89
59.96
59.98
59.99
60.00
60.00
60.00
The following figure plots this data. Note that the values at the higher times have steadied
out to a value of 60. So, a reasonable ultimate value is y = 60 y = 10 .
62
60
58
y(t) 56
54
52
50
0
10
20
30
40
50
60
Data Fitting
-8-
Data Fitting
-9-
0
-1
-2
-3
ln( 1 - y'/y' ) -4
-5
-6
-7
-8
0
10
20
30
40
50
60
To use the Regression add-in it must first be installed. This can be done by the commands:
Tools, Add-Ins
Place a check mark in front of the Analysis TookPak in the list presented.
Start the regression process with the command Tools, Data Analysis Chose the option
Regression from the list presented. You will then get a window like the one shown below.
The important items to fill out are:
What cells have the transformed function data? Here the values are in the cells
denoted as Input Y Range.
What cells have the time data? Here the values are in the cells denoted as Input X
Range.
Where do you want your calculated values reported? The upper right hand cell is
specified as Output Range.
Data Fitting
- 10 -
The following shows the report that is generated by this procedure. There are various
statistics reported (most of which well ignore for right now) what were most interested
in are the values in the Coefficients column of the last table. From these values the
regression equation will be:
y
ln 1
= a0 + a1t = 0.263884 0.15702 t .
y
From this we can determine the time constant & the time delay:
1
1
=
= 6.37
a1 0.15702
a 0.263884
= 0 =
= 1.67 .
a1 0.15702
=
Data Fitting
- 11 -
Regression Statistics
Multiple R 0.994951
R Square 0.989928
Adjusted R Square
0.989581
Standard Error
0.205303
Observations
31
ANOVA
df
Regression
Residual
Total
SS
MS
1 120.1397 120.1397
29 1.222333 0.042149
30 121.3621
Coefficients
Standard Error t Stat
Intercept 0.263884 0.04553 5.795897
X Variable 1 -0.15702 0.002941 -53.3885
Significance F
2850.33
1.64E-30
-2
ln( 1 - y'/y' ) -4
-6
-8
-10
0
10
20
30
40
50
60
The chart shows the regressed line compared to the original data. It does not look very
good. Why? All of the points were specified, including the early time values that are going to
be approximated with the time delay. It is probably better to not use those points in the
regression. So, lets re-run the regression only using the points for t 10 . When you re-
Data Fitting
- 12 -
2
0
-2
-4
ln( 1 - y'/y' )
-6
-8
-10
-12
0
10
20
30
40
50
60
Now the fit looks much better. Now we get the results:
Data Fitting
- 13 -
y
ln 1
= a0 + a1t = 0.759653 0.17388 t .
y
From this we can determine the time constant & the time delay:
1
1
=
= 5.75
a1 0.17388
a 0.759653
= 0 =
= 4.37 .
a1 0.17388
Note that these parameters match those of the Taylor series approximation quite
well.
Data Fitting
- 14 -
We will use the Solver tool to do the non-linear regression. Start the regression process
with the command Tools, Solver The important items to fill out are:
What cell represents the objective function? Specify this cell in the Set Target Cell
box & that is to be minimized by clicking Min.
Data Fitting
- 15 -
What cells are to be modified? The cells for & are specified in the By Changing
Cells box.
After clicking Solve, we get the following result. The fit is very good for times t 10 (the
data used for the regression) & the associated parameters are also shown. The figure
suggests that we can add in more of the shorter time values, too, & still get a good fit to the
large time values.
= 6.029618
= 4.038509
y' =
10
12
10
y'(t) 6
0
0
10
20
30
40
50
60
Data Fitting
- 16 -
The following set of results makes use of the data for t 7 . The fit still looks very good for
these values used for the regression. The associated parameters have changed slightly.
With some experimentation one finds that including more short time data makes the fit
near the maximum (for times around 20) worse. This is probably the best FOPDT fit to the
data.
= 6.442579
= 3.463511
y' =
10
12
10
y'(t) 6
0
0
10
20
30
40
50
60
Data Fitting
- 17 -