Nonlinear Regression

1
 Nonlinear Regression
Some popular nonlinear regression models:

1. Exponential model: ( y = aebx )

2. Power model: ( y = axb )
 ax 
3. Saturation growth model:  y = 
 b + x 
4. Polynomial model: ( y = a0 + a1x + ... + amx m )
 Nonlinear Regression

Given n data points ( x1, y1), ( x 2, y 2), ... , ( xn, yn) best fit y = f (x)
to the data, where f (x) is a nonlinear function of x .

( xn , y n )

( x2 , y 2 )
y = f (x)
( xi , yi )
yi − f ( xi )
( x1 , y1 )

Figure. Nonlinear regression model for discrete y vs. x data

 Regression
Exponential Model
 Exponential Model

Given ( x1 , y1 ), ( x2 , y2 ), ... , ( xn , yn ) best fit y = ae to the data.

bx

( x1 , y1 )
y = aebx

yi − aebxi
( xi , yi )

( x2 , y 2 )
( xn , y n )

Figure. Exponential model of nonlinear regression for y vs. x data

 Finding Constants of Exponential
Model
The sum of the square of the residuals is defined as

( )
n
Sr =  yi − ae
bxi 2

i =1
Differentiate with respect to a and b

S r
( )( )
n
=  2 y i − ae − e = 0
bxi bxi

a i =1

S r
( )( )
n
=  2 y i − ae bxi
− axi e = 0
bxi

b i =1
 Finding Constants of Exponential
Model
Rewriting the equations, we obtain

n n
−  yi e bxi
+ a  e 2bxi = 0
i =1 i =1

n n
 y i xi e
bxi
− a  xi e 2bxi
=0
i =1 i =1
Finding constants of Exponential Model
Solving the first equation for a yields
n
bxi
 yi e
i =1
a= n
2bxi
 e
i =1

Substituting a back into the previous equation

n
bxi
n
 yi e n
i =1
 y i xi e
bxi
− n
 i
x e 2bxi
=0
i =1 2bxi i =1
e
i =1
The constant b can be found through numerical
methods such as bisection method.
 Example 1-Exponential Model
Many patients get concerned when a test involves injection of a
radioactive material. For example for scanning a gallbladder, a
few drops of Technetium-99m isotope is used. Half of the
techritium-99m would be gone in about 6 hours. It, however,
takes about 24 hours for the radiation levels to reach what we
are exposed to in day-to-day activities. Below is given the
relative intensity of radiation as a function of time.

Table. Relative intensity of radiation as a function of time.

t(hrs) 0 1 3 5 7 9
 1.000 0.891 0.708 0.562 0.447 0.355
Example 1-Exponential Model cont.

The relative intensity is related to time by the equation

t
 = Ae
Find:
a) The value of the regression constants A and 
b) The half-life of Technium-99m
c) Radiation intensity after 24 hours
Plot of data
 Constants of the Model
 = Ae t

The value of λ is found by solving the nonlinear equation

n
ti
n

 i e n
f ( ) =   i t i e ti
− i =1
n
 i
t e 2ti
=0
i =1 2ti i =1
 e
i =1
n

 i
 e t i

A= i =1
n

 e 2 t i

i =1
 Setting up the Equation in MATLAB
n
ti
n
 ie n
f ( ) =   i t i e ti
− i =1
n
 ti e
2ti
=0
i =1 2ti i =1
e
i =1

t (hrs) 0 1 3 5 7 9
γ 1.000 0.891 0.708 0.562 0.447 0.355
Setting up the Equation in MATLAB
n
ti
n
 ie n
f ( ) =   i t i e ti
− i =1
n
 i
t e 2ti
=0
i =1 2ti i =1
e
i =1

 = −0.1151
t=[0 1 3 5 7 9]
gamma=[1 0.891 0.708 0.562 0.447 0.355]
syms lamda
sum1=sum(gamma.*t.*exp(lamda*t));
sum2=sum(gamma.*exp(lamda*t));
sum3=sum(exp(2*lamda*t));
sum4=sum(t.*exp(2*lamda*t));
f=sum1-sum2/sum3*sum4;
Calculating the Other Constant

The value of A can now be calculated

6

 e i
ti

A= i =1
6 = 0.9998
 e 2 ti

i =1

The exponential regression model then is

 = 0.9998 e −0.1151t
Plot of data and regression curve

 = 0.9998 e −0.1151t
Relative Intensity After 24 hrs
The relative intensity of radiation after 24 hours
−0.1151(24 )
 = 0.9998  e
−2
= 6.3160  10
This result implies that only

6.316  10−2
 100 = 6.317%
0.9998
radioactive intensity is left after 24 hours.
 Polynomial Model

Given ( x1, y1), ( x 2, y 2), ... , ( xn, yn) best fit y = a + a x + ... + a x m
0 1 m
(m  n − 2) to a given data set.

( xn , y n )

( x2 , y 2 )

( xi , yi ) y = a0 + a1 x +  + am x m
yi − f ( xi )
( x1 , y1 )

Figure. Polynomial model for nonlinear regression of y vs. x data

 Polynomial Model cont.

The residual at each data point is given by

Ei = yi − a0 − a1 xi − . . . − a m xim
The sum of the square of the residuals then is
n
S r =  Ei2
i =1

( )
n
=  y i − a 0 − a1 xi − . . . − a m xim
2

i =1
 Polynomial Model cont.

To find the constants of the polynomial model, we set the derivatives

with respect to ai where i = 1, m, equal to zero.
S r
=  2.( yi − a0 − a1 xi − . . . − am xim )(−1) = 0
n

a0 i =1
S r
=  2.( yi − a0 − a1 xi − . . . − am xim )(− xi ) = 0
n

a1 i =1

   
S r
=  2.( yi − a0 − a1 xi − . . . − am xim )(− xim ) = 0
n

am i =1
 Polynomial Model cont.

These equations in matrix form are given by

   
   n 
 n  n   n m 
  xi  .  xi  a
   yi
. .  
  i =1   i =1    0 
 n    ni =1
   xi   n 2  n m+1  a1 =

  xi  . . .  xi   xi yi 
  i =1   i =1   i =1  . . 
.  i =1 
. . . . . . . . . . .  a  . . . 
  m   
 n m   n m+1   
n
 xim yi
n

  xi    xi  . . .  xi2 m  
 i =1 
 i =1   i =1   i =1 

The above equations are then solved for a0 , a1 ,  , am

 Example 2-Polynomial Model

Regress the thermal expansion coefficient vs. temperature data to

a second order polynomial.

Table. Data points for

temperature vs α
7.00E-06

Thermal expansion coefficient, α

Temperature, T Coefficient of 6.00E-06
(oF) thermal
expansion, α 5.00E-06
(in/in/oF)
80 6.47×10−6 (in/in/o F) 4.00E-06

40 6.24×10−6
3.00E-06
−40 5.72×10−6
−120 5.09×10−6 2.00E-06

−200 4.30×10−6 1.00E-06

−280 3.33×10−6 -400 -300 -200 -100 0 100 200

−340 2.45×10−6 Temperature, o F

Figure. Data points for thermal expansion coefficient vs

temperature.
 Example 2-Polynomial Model
cont.
We are to fit the data to the polynomial regression model
α = a0 + a1T + a 2T 2
The coefficients a0 ,a1 , a2 are found by differentiating the sum of the
square of the residuals with respect to each variable and setting the
values equal to zero to obtain

  n   n 2   n 
 n   Ti    Ti     i 
  i =1   i =1   a   i =1 
0
 n   n 2  n 3     n 
  i 
 T   Ti    Ti   a1  =  Ti  i
 i =n1   i =1   i =1      i =1 
 n 
 n 4    2   T 2
a
 T 2   n 3
  i   Ti    Ti  


 i i 

 i =1   i =1   i =1   i =1
 Example 2-Polynomial Model
cont.
The necessary summations are as follows
7
Table. Data points for temperature vs.
Temperature, T Coefficient of
α
T
i =1
i
2
=2.5580 105
(oF) thermal expansion,
7

T
α (in/in/oF)
i
3
= − 7.0472  10 7
80 6.47×10−6
i =1
40 6.24×10−6 7
−40 5.72×10−6 T
i =1
i
4
= 2.1363 1010
−120 5.09×10−6
7


−200 4.30×10−6
i = 3.3600  10 −5
−280 3.33×10−6
i =1
−340 2.45×10−6 7

T 
i =1
i i = − 2.6978  10 −3
7

T
i =1
i
2
 i =8.5013  10 −1
 Example 2-Polynomial Model
cont.
Using these summations, we can now calculate a0 ,a1 , a2
 7.0000 − 8.6000  10 2 2.5800  10 5  a0   3.3600  10 −5 
   
 − 8.600  10 2
2.5800  10 5 − 7.0472  10 7   a1  = − 2.6978  10 −3 
 2.5800  10 5 − 7.0472  10 7 2.1363  1010  a 2   8.5013  10 −1 

Solving the above system of simultaneous linear equations we have
a0   6.0217  10 
−6

 a  =  6.2782  10 −9 
 1  
a 2  − 1.2218  10 
−11

The polynomial regression model is then

α = a0 + a1T + a 2T 2
= 6.0217  10 −6 + 6.2782  10 −9 T − 1.2218  10 −11 T 2
 Transformation of Data

To find the constants of many nonlinear models, it results in solving

simultaneous nonlinear equations. For mathematical convenience,
some of the data for such models can be transformed. For example,
the data for an exponential model can be transformed.
As shown in the previous example, many chemical and physical processes
are governed by the equation,
y = aebx
Taking the natural log of both sides yields,
ln y = ln a + bx
Let z = ln y and a0 = ln a
We now have a linear regression model where z = a 0 + a1 x
(implying) a = e ao with a1 = b
 Linearization of data cont.

Using linear model regression methods,

n n n
n xi z i −  xi  z i
a1 = i =1 i =1 i =1
2
n
  n
n xi2 −   xi 
i =1  i =1 
_ _
a 0 = z − a1 x
Once ao , a1 are found, the original constants of the model are found as
b = a1
a = e a0
 Example 3-Linearization of data

Many patients get concerned when a test involves injection of a radioactive

material. For example for scanning a gallbladder, a few drops of Technetium-
99m isotope is used. Half of the technetium-99m would be gone in about 6
hours. It, however, takes about 24 hours for the radiation levels to reach what
we are exposed to in day-to-day activities. Below is given the relative intensity
of radiation as a function of time.
1
Table. Relative intensity of radiation as a function

Relative intensity of radiation, γ

of time
t(hrs) 0 1 3 5 7 9
 1.000 0.891 0.708 0.562 0.447 0.355 0.5

0
0 5 10
Time t, (hours)

Figure. Data points of relative radiation intensity

vs. time
 Example 3-Linearization of data cont.

Find:
a) The value of the regression constants A and 
b) The half-life of Technium-99m
c) Radiation intensity after 24 hours
The relative intensity is related to time by the equation
 = Ae t
Example 3-Linearization of data cont.

Exponential model given as,

 = Ae t
ln( ) = ln( A) + t
Assuming z = ln  , ao = ln( A) and a1 =  we obtain
z = a0 + a1t
This is a linear relationship between z and t
 Example 3-Linearization of data cont.

Using this linear relationship, we can calculate a0 , a1 where

n n n
n  ti zi −  ti  zi
a1 = i =1 i =1 i =1
2
n
 n

n t12 −   ti 
i =1  i =1 
and
a0 = z − a1t
=a 1
a
A=e 0
 Example 3-Linearization of Data cont.

Summations for data linearization are as follows

Table. Summation data for linearization of data model With n = 6

6

i ti i zi = ln  i ti zi t 2 t
i =1
i = 25.000
i
6

z
1 0 1 0.00000 0.0000 0.0000
2 1 0.891 −0.11541 −0.11541 1.0000 i
= −2.8778
3 3 0.708 −0.34531 −1.0359 9.0000 i =1
4 5 0.562 −0.57625 −2.8813 25.000 6
5
6
7
9
0.447
0.355
−0.80520
−1.0356
−5.6364
−9.3207
49.000
81.000 t z
i =1
i i
= −18.990


6
−2.8778 −18.990
t
25.000 165.00
i
2
= 165.00
i =1
Example 3-Linearization of Data cont.

Calculating a0 , a1
6(− 18.990) − (25)(− 2.8778)
a1 = = −0.11505
6(165.00) − (25)
2

− 2.8778
− (− 0.11505)
25 = −2.6150  10 −4
a0 =
6 6
Since
a0 = ln( A)
A = e a0
−2.615010−4
=e = 0.99974
also
 = a1 = −0.11505
Example 3-Linearization of Data cont.

Resulting model is  = 0.99974  e

−0.11505t

1
 = 0.99974  e −0.11505t

Relative
Intensity
0.5
of
Radiation,

0
0 5 10
Time, t (hrs)

Figure. Relative intensity of radiation as a function of

temperature using linearization of data model.
Example 3-Linearization of Data cont.

The regression formula is then

 = 0.99974  e −0.11505t
1
b) Half life of Technetium 99 is when  = 
2 t =0

0.99974  e −0 .11505t = (0.99974)e −0 .11505 (0 )

1
2
e −0 .11508t = 0.5
− 0.11505t = ln(0.5)
t = 6.0248 hours
Example 3-Linearization of Data cont.

c) The relative intensity of radiation after 24 hours is then

 = 0.99974e −0.11505(24)
= 0.063200
6.3200 10−2
This implies that only 100 = 6.3216% of the radioactive
0.99983
material is left after 24 hours.
 Comparison
Comparison of exponential model with and without data linearization:
Table. Comparison for exponential model with and without data
linearization.

With data linearization Without data linearization

(Example 3) (Example 1)
A 0.99974 0.99983
λ −0.11505 −0.11508
Half-Life (hrs) 6.0248 6.0232
Relative intensity
6.3200×10−2 6.3160×10−2
after 24 hrs.

The values are very similar so data linearization was suitable to

find the constants of the nonlinear exponential model in this
case.
KORELASI dan REGRESI
REGRESI DAN KORELASI

39
 REGRESI LINEAR

• y= a + bx

40
REGRESI LINEAR SEDERHANA
Interpretasi persamaan :

Setiap perubahan x sebesar 1

satuan maka y akan berubah
sebesar b
Dengan :

Dengan : n = banyaknya data

xi= peubah bebas ke-i
yi=peubah respon ke -i
41
 ANALISIS REGRESI
• Pendugaan terhadap koefisien regresi:
→ b0 penduga bagi 0 dan b1 penduga bagi 1

( x)( y )
 xy − n Metode
b1 =
x − n 2 (  x ) 2 Kuadrat Terkecil

b0 = y − b1 x

42
 REGRESI
→ analisis yang menyatakan hubungan antara peubah
respon Y dengan satu atau lebih peubah bebas Xi, yang
dinyatakan dalam bentuk

43
Estimasi persamaan regresi linear

Meminimumkan error dengan Least square error atau

(Sum Square Error)

44
Persamaan untuk mencari parameter a, b1 dan b2

45
 KORELASI
Analisis yang digunakan untuk menelaah tingkat
keeratan hubungan 2 variabel (x dan y)

X : Variabel bebas
Y : Variabel tidak bebas (Peubah Respon)

→ tidak menggambarkan hubungan sebab akibat

→ nilainya berkisar antara -1 dan 1
→ Pearson’s Coef of Correlation linear relationship
→ Spearman’n Coef of Correlation (rank correlation) trend
relationship

46
Nilai koefisien korelasi (r) untuk regresi linear :

n xi yi −  xi  yi
r=
n xi − ( xi ) n  y i − (  yi )
2 2 2 2

Dengan : n=banyaknya data

xi= peubah bebas ke-i
yi=peubah respon ke-i
Koefisien Determinasi (R):
Besarnya kontribusi x terhadap naik turunnya y

R=r 2
47
Untuk Regresi Linear Ganda

48
Nilai koefisien korelasi (r) untuk regresi linear berganda :

49
 BENTUK HUBUNGAN
Hubungan + :
Jika kenaikan /penurunan x pada umumnya diikuti oleh kenaikan /
penurunan y

Hubungan - :
Jika kenaikan / penurunan x pada umunya diikuti oleh penurunan //
kenaikan y

50
POLA HUBUNGAN X DAN Y

51
Y .
. Korelasi Linear:
. . Jika semua titik (X,Y) pd diagram pencar
.. ..
. .. mendekati bentuk garis lurus.
. X
Y .. Korelasi Non-linear:
. . .
.. .. Jika semua titik (X,Y) pd diagram pencar
. . . . tidak membentuk garis lurus.
.
X
Y .
. Korelasi Positif:
. . Jika arah perubahan kedua variabel
.. ..
. .. sama  If X naik, Y juga naik.
. X
Y
. .
. .
.. ..

Korelasi Negatif:
Jika arah perubahan kedua variabel
. ..

tidak sama  If X naik, Y turun.

.

X
52
KOEFISIEN KORELASI

53
Contoh :

54
55
56
57
58
59
Koefisien Korelasi regresi Ganda :

60
 Regresi

Makna 0 & 1 ?

61
Regresi

62
 UJI MODEL REGRESI

Bagaimana Pengujian terhadap model regresi ??

• parsial (per koefisien) → uji-t
• bersama → uji-F (Anova)
Bagaimana menilai kesesuaian model ??
R2 (Koef. Determinasi: % keragaman Y yang mampu
dijelaskan oleh X)

63
64
a. Plot data tersebut pada koordinat kartesian
b. Cari regresi linear pad plot hasil regresi linear pada soal a
c. Hitung koefisien korelasi

65
 Tahun y x
1995 38.8 4.15
1996 56.7 10.38
1997 59.0 10.89
1998 62.2 11.96
1999 62.6 12.46
2000 85.7 17.72
2001 119.1 28.07
2002 133.3 36.11
2003 150.4 38.70
2004 167.8 39.60

a. Plot data tersebut pada koordinat kartesian

b. Cari regresi linear pad plot hasil regresi linear pada soal a
c. Hitung koefisien korelasi

66
y xi xi2 yi2 xiyi
38.8 4.15
56.7 10.38
59.0 10.89
62.2 11.96
62.6 12.46
85.7 17.72
119.1 28.07
133.3 36.11
150.4 38.70
167.8 39.60
 yi  xi

67
68
 TUGAS D3

toksin y x
1 23.00 39.00
2 22.30 37.50
3 9.40 22.20
4 9.70 17.50
5 0.15 0.64
6 0.28 0.45
7 0.75 2.26
8 0.51 2.36
9 28.00 32.00
10 0.39 0.77

a. Plot data tersebut pada koordinat kartesian

b. Cari regresi linear pad plot hasil regresi linear pada soal a
c. Hitung koefisien korelasi

69
y xi xi2 yi2 xiyi
23.00 39.00
22.30 37.50
9.40 22.20
9.70 17.50
0.15 0.64
0.28 0.45
0.75 2.26
0.51 2.36
28.00 32.00
0.39 0.77
 yi  xi

70
 TUGAS

Membangkitan data yang berdistribusi normal dengan rata-rata

dan standar deviasi (0,1) dn rata-rata dan standar deviasi tertentu
(dalam satu kelas tidak boleh sama).
→ Buat histogram
→ Plot pdf dan cdf dari kedua pembangkitan.

71