Multiple-Choice Test

Nonlinear Regression
Regression
COMPLETE SOLUTION SET

1. When using the transformed data model to find the constants of the regression model y = ae bx
to best fit ( x1 , y1 ), ( x 2 , y 2 ),........( x n , y n ), the sum of the square of the residuals that is minimized is

( )
n
(A) ∑ y i − aebxi
2

i =1
n

∑ (ln( y ) − ln(a ) − bx )
2
(B) i i
i =1
n

∑ ( y − ln(a ) − bx )
2
(C) i i
i =1
n
(D) ∑ (ln( y i ) − ln (a ) − b ln( xi ) )
2

i =1
Solution
The correct answer is (B).

Taking the natural log of both sides of the regression model

y = ae bx
gives
ln ( y ) = ln (a ) + bx
The residual at each data point xi is
Ei = ln( y i ) − ln(a ) − bxi
The sum of the square of the residuals for the transformed data is
n
S r = ∑ Ei2
i =1
n
= ∑ (ln ( y i ) − ln (a ) − bxi )
2

i =1
2. It is suspected from theoretical considerations that the rate of water flow from a firehouse is
proportional to some power of the nozzle pressure. Assume pressure data is more accurate. You
are transforming the data.
Flow rate, F (gallons/min) 96 129 135 145 168 235
Pressure, p (psi) 11 17 20 25 40 55
b
The exponent of the nozzle pressure in the regression model F = ap most nearly is
(A) 0.49721
(B) 0.55625
(C) 0.57821
(D) 0.67876

Solution
The correct answer is (A).

The transforming of the above data is done as follows.

F = ap b
ln( F ) = ln(a ) + b ln( p )
z = a 0 + bx
where
z = ln (F )
x = ln( p)
a0 = ln (a )
implying
a = e a0
There is a linear relationship between z and x.
Linear regression constants are given by
n n n
n∑ xi zi −∑ xi ∑ zi
b= i =1 i =1 i =1
2
n
  n
n∑ x −  ∑ xi 
2
i
i =1  i =1 
n n n n

∑ xi2 ∑ zi − ∑ xi ∑ xi zi
a0 = i =1 i =1 i =1 i =1
2
n
  n
n∑ xi2 −  ∑ xi 
i =1  i =1 
Since
 n=6
6

∑x z
i =1
i i = ln(11) × ln(96) + ln(17) × ln(129) + ln(20) × ln(135)

+ ln(25) × ln(145) + ln(40) × ln(168) + ln(55) × ln(235)

= 96.208
6

∑x
i =1
i = ln(11) + ln(17) + ln(20) + ln(25) + ln(40) + ln(55) = 19.142
6

∑z
i =1
i = ln(96) + ln(129) + ln(135) + ln(145) + ln(168) + ln(235) = 29.890
6

∑x
i =1
2
i = (ln(11)) 2 + (ln(17)) 2 + (ln(20)) 2 + (ln(25)) 2 + (ln(40)) 2 + (ln(55)) 2 = 62.779

then
6 × 96.208 − 19.142 × 29.890
b=
6 × 62.779 − 19.1422
577.25 − 572.15
=
376.67 − 366.41
= 0.49721
Can you now find what a is?
3. The transformed data model for the stress-strain curve σ = k1εe − k 2ε for concrete in
compression, where σ is the stress and ε is the strain, is
(A) ln (σ ) = ln (k1 ) + ln (ε ) − k2ε
σ 
(B) ln  = ln (k1 ) − k2ε
ε 
σ 
(C) ln  = ln (k1 ) + k2ε
ε 
(D) ln (σ ) = ln(k1ε ) − k2ε
Solution
The correct answer is (B)

σ = k1εe − k ε 2

The model can be rewritten as

σ
= k1 e − k ε
2

ε
To transform the data, we take the natural log of both sides
σ 
ln  = ln (k1e − k ε )
2

ε 
(
= ln (k1 ) + ln e − k 2ε )
= ln (k1 ) − k 2 ε
4. In nonlinear regression, finding the constants of the model requires solving simultaneous
nonlinear equations. However in the exponential model y = ae bx that is best fit
to ( x1 , y1 ), ( x2 , y2 ),........, ( xn , yn ), the value of b can be found as a solution of a single nonlinear
equation. That nonlinear equation is given by
n n n
(A) ∑ y i xi e bxi − ∑ y i e bxi ∑ xi = 0
i =1 i =1 i =1
n

n ∑ye i
bxi
n
(B) ∑ yi xi e bxi − i =1
n ∑x e i
2 bxi
=0
i =1
∑e
i =1
2 bxi i =1

n ∑ye i
bxi
n
(C) ∑ yi xi e bxi − i =1
n ∑e bxi
=0
i =1
∑e
i =1
2 bxi i =1

n ∑ye i
bxi
n
(D) ∑ y i e bxi − i =1
n ∑x e i
2 bxi
=0
i =1
∑e
i =1
2 bxi i =1

Solution
The correct answer is (B).

Given ( x1 , y1 ), ( x2 , y2 ),........, ( xn , yn ), best fit y = ae bx to the data. The variables a and b are the
constants of the exponential model. The residual at each data point xi is
Ei = yi − ae bxi (1)
The sum of the square of the residuals is
n
S r = ∑ Ei2
i =1

( )
n
= ∑ yi − aebxi
2
(2)
i =1

To find the constants a and b of the exponential model, we find where S r is a local minimum or
maximum by differentiating with respect to a and b and equating the resulting equations to
zero.

∂S r
( )( )
n
= ∑ 2 y i − ae bxi − e bxi = 0
∂a i =1

∂S r
( )( )
n
= ∑ 2 y i − ae bxi − axi e bxi = 0 (3a,b)
∂b i =1
or
n n
− ∑ yi ebxi + a ∑ e 2bxi = 0
i =1 i =1
n n

∑ y xe
i =1
i i
bxi
− a ∑ xi e 2bxi = 0
i =1
(4a,b)

Equations (4a) and (4b) are simultaneous nonlinear equations with constants a and b . This is
unlike linear regression where the equations to find the constants of the model are simultaneous
but linear. In general, iterative methods (such as the Gauss-Newton iteration method, Method of
Steepest Descent, Marquardt's Method, Direct search, etc) must be used to find values of a and
b.
However, in this case, from Equation (4a), a can be written explicitly in terms of b as
n

∑ye i
bx i

a= i =1
n
(5)
∑e
i =1
2 bx i

Substituting Equation (5) in (4b) gives

n

n ∑ye i
bxi
n

∑ yi xiebxi − i =1
n ∑xe i
2 bxi
=0
i =1
∑e
i =1
2 bxi i =1

This equation is still a nonlinear equation in terms of b , and can be solved best by numerical
methods such as the bisection method or the secant method.

You can now show that these values of of a and b, correspond to a local minimum, and since the
above nonlinear equation has only one real solution, it corresponds to an absolute minimum.
5. There is a functional relationship between the mass density p of air and the altitude h above
the sea level.
Altitude above sea level, h (km) 0.32 0.64 1.28 1.60
Mass Density, ρ (kg/m3) 1.15 1.10 1.05 0.95
In the regression model ρ = k1e − k 2 h , the constant k 2 is found as k 2 = 0.1315 . Assuming the
mass density of air at the top of the atmosphere is 1 / 1000 th of the mass density of air at sea level.
The altitude in kilometers of the top of the atmosphere most nearly is
(A) 46.2
(B) 46.6
(C) 49.7
(D) 52.5
Solution
The correct answer is (D).

Note to the student: See the alternative answer given later as that is quite a bit shorter.
Since
k 2 = 0.1315
is given, the sum of the square of the residual is
2

( )
n
S r = ∑ ρ i − k1e −0.1315 hi
i =1

First we need to find the value of the constant k1 .

∂S r
( )( )
n
= ∑ 2 ρi − k1e − 0.1315 hi − e − 0.1315 hi = 0
∂k1 i =1
n n
− ∑ ρi e − 0.1315 hi + k1 ∑ e − 2×0.1315 hi = 0
i =1 i =1

Thus,
n

∑ρ e i
− 0.1315 hi

k1 = i =1
n

∑e i =1
− 0.263 hi

Since
n=4
n

∑ρ e
i =1
i
− 0.1315 hi
= 1.15e − 0.1315×0.32 + 1.10e − 0.1315×0.64 + 1.05e − 0.1315×1.28 + 0.95e − 0.1315×1.60

= 1.15 × 0.95879 + 1.10 × 0.91928 + 1.05 × 0.84508 + 0.95 × 0.81026

= 3.7709
 n

∑e
i =1
− 0.263 hi
= e − 0.263×0.32 + e − 0.263×0.64 + e − 0.263×1.28 + e − 0.263×1.60

= 0.91928 + 0.84508 + 0.71417 + 0.65652

= 3.1351
the value of the constant k1 is
3.7709
k1 =
3.1351
= 1.2028
Hence
ρ = k1e − k 2 h
= 1.2028e − 0.1315 h kg m3
ρsea − level =1.2028e −0.1315×0
= 1.2028 kg m3
1
ρ top = ρsea − level
1000
1
= × 1.2028
1000
= 0.0012028 kg m3

−0.1315× htop
ρ top = k1e
0.0012028
− 0.1315× htop
e =
1.2028
ln(0.001)
htop =
− 0.1315
= 52.530 km

Alternative Answer:
Note to the student: Do we really need to find k1 for this problem?

ρ = k1e −0.1315 h
ρ sea −level = k1e −0.1315×0
= k1
− 0.1315 htop
ρ top = k1e
ρ sea −level k1
=
ρ top k1 e
− 0.1315 h top
 ρsea −level 1
=
1 −0.1315 htop
ρsea −level e
1000
 1 
ln 
1000 
htop = 
− 0.1315
= 52.530 km
6. A steel cylinder at 80° F of length 12" is placed in a commercially available liquid nitrogen
bath (−315° F) . If the thermal expansion coefficient of steel behaves as a second order
polynomial function of temperature and the polynomial is found by regressing the data below,

Temperature, Thermal expansion

T (°F) Coefficient, α
( µ in/in/°F)
− 320 2.76
− 240 3.83
− 160 4.72
− 80 5.43
0 6.00
80 6.47

the reduction in the length of the cylinder in inches most nearly is

(A) 0.0219
(B) 0.0231
(C) 0.0235
(D) 0.0307

Solution
The correct answer is (C).

We are fitting the above data to the following polynomial.

α = a0 + a1T + a 2 T 2
(
S r = ∑ α i − a0 − a1Ti − a2Ti 2 )
2

There is a quadratic relationship between the thermal expansion coefficient and the temperature,
and the coefficients a0 , a1 , and a2 are found as follows

∂S r n
( )
= ∑ 2 α i − a 0 − a1Ti − a 2Ti 2 (− 1) = 0
∂a 0 i =1
∂S r n
( )
= ∑ 2 α i − a 0 − a1Ti − a 2Ti 2 (− Ti ) = 0
∂a1 i =1
∂S r n
( )( )
= ∑ 2 α i − a 0 − a1Ti − a 2Ti 2 − Ti 2 = 0
∂a 2 i =1

which gives
   n   n 2   n 
 n  ∑ Ti   ∑ Ti   ∑α i
  i =1   i =1   a   i =1 

0
 n   n 2  n    n

 ∑ Ti   ∑ Ti   ∑ Ti 3   a1  =  ∑ Ti α i 
 i =n1   i =1   i =1      i =1 
a  n 
 T 2   n 3  n 4    2   T 2α
 ∑ i  ∑ Ti  ∑ i  
T ∑ i i
 i =1


 i =1   i =1   i =1  

Table 1 Summations for calculating constants of model.

i T (oF) α (in/in/oF) T2 T3
1 80 6.4700 × 10–6 6.4000 × 103 5.1200 × 105
2 0 6.0000 × 10–6 0.0000 0.0000
–6 3
3 − 80 5.4300 × 10 6.4000 × 10 –5.1200 × 105
4 − 160 4.7200× 10–6 2.5600× 104 –4.0960× 106
5 − 240 3.8300 × 10–6 5.7600 × 104 –1.3824 × 107
6 − 320 2.7600 × 10–6 1.0240 × 105 –3.2768 × 107
6

∑
i =1 − 7.2000 × 102 2.9210 × 10–5 1.9840 × 105 − 5.0688 × 107

Table 1 (cont)
i T4 T ×α T 2 ×α
1 4.0960 × 107 5.1760 × 10–4 4.1408 × 10–2
2 0.0000 0.0000 0.0000
3 4.0960 × 107 –4.3440 × 10–4 3.4752 × 10–2
4 6.5536 × 108 –7.5520 × 10–4 1.2083 × 10–1
5 3.3178 × 109 –9.1920 × 10–4 2.2061 × 10–1
6 1.0486 × 1010 –8.8320 × 10–4 2.8262 × 10–1
6

∑ 1.4541 × 1010 –2.4744 × 10–3 7.0022 × 10–1

i =1

We have
 6 − 7.2000 × 10 2 1.9840 × 105  a0   2.9210 × 10 −5 
   
− 7.2000 × 10
2
1.9840 × 105 − 5.0688 × 107  a1  = − 2.4744 × 10 −3 
 1.9840 × 105 − 5.0688 × 107 1.4541× 1010  a2   7.0022 × 10 −1 

Solving the above system of simultaneous linear equations, we get
 a 0   6.0238 × 10 
−6

 a  =  6.3319 × 10 −9 
 1  
a 2  − 1.1965 × 10 −11 
The polynomial regression model is
α = a0 + a1T + a2T 2
= 6.0237 × 10 −6 + 6.3375 × 10 −9 T − 1.1942 × 10 −11T 2
Since
Tfluid

∆L = L0 × ∫ α dT
Troom
−315

∫ (6.0237 ×10 )
−6
= 12 × + 6.3375 × 10−9 T − 1.1942 × 10 −11T 2 dT
80
−315
 6.3375 × 10 −9 2 1.1942 × 10 −11 3 
= 12 × 6.0237 × 10 −6 T + T − T 
 2 3  80
[ ]
= 12 × 6.0237 × 10 −6 T + 3.1687 × 10 −9 T 2 − 3.9807 × 10 −12 T 3
−315
80

= 12 × [6.0237 × 10 (− 315) + 3.1687 × 10 (− 315 ) − 3.9807 × 10 (− 315 )]

−6 −9 2 −12 3

− 12 × [6.0237 × 10 (80) + 3.1687 × 10 (80 ) − 3.9807 × 10 (80 )]

−6 −9 2 −12 3

= 12 × [− 1.4586 × 10 − 5.0014 × 10 ]
−3 −4

= 0.023505"