6.

Least-Square Method
Class 12. Numerical Analysis
Least square and regression

Dr. Josept Revuelta

Least Square Method
In this section we are going to discuss the least method as an
approximate solution of a system of equations. Later, we will use to
perform regression analysis. But first, we need to introduce the method
and understand the math behind the method.

Please watch the this video before staring the class. You can activate
the captions on using the settings button. “Activa los Subtitulos”

https://www.youtube.com/watch?v=MC7l96tW8V8
Least Square Method
Now that have the theory, let’s do a basic example to extend the
concept:
Find the quadratic equation through the origin that a best fit for the
points (1,1), (2,5), and (-1,-2). The first step is to choose the model or
equation to fit the points or the data, in this exercise we use:
2
ct + dt = y
Next, we need a matrix A:

x̂ = [ ]
1 1 c 1
[−1 1] [−2]
A= 2 4 b= 5
d
We cannot solve A x ̂ = b because there is no quadratic equation
through the origin that contains these three points.
 Least Square Method
As we saw in the video, the system A x ̂ = b is the same as:
T T
A Ax ̂ = A b
Then,

1 1
[1 4 1 ] [ ] [8 10]
T 1 2 −1 6 8
A A= 2 4 =
−1 1

And

1
[1 4 1 ] [ ] [19]
T 1 2 −1 13
A b= 5 =
−2
Least Square Method

[8 10] [19]
6 8 ̂ 13
x=

Finally, we can back-solve using Gaussian Elimination:

[0 −2] [5]
6 8 13
x̂ =

The roots are:

c = 11/2
d = − 5/2
Thus, the final equation is: 11 5 2
y= t− t
2 2
More examples:
In engineering or science, we often use linear regression to estimate
variables that linearly depend on a second variable. For example let’s
considere the following data:
We can fit a linear model for the cancer death rate using excel:

y = a + bx
P P P
n xy x y
r= p P P P P
(n x2 ( x)2 )(n y 2 ( y)2 )

sy
b=r a = ȳ bx̄
sx
Exercises:
Perform linear regression for the following set of data:
Ice cream sales data
30 year Day Temp Sales
Median home Years Tui@on
X Y Year mortgage
price since in $ per 1 65 20
interest rate
2000 credit
3 1 1988 10.3 183800 2 68 22
0 38 3 66 21
4 4 1989 10.3 183200
4 75 23
1990 10.1 174900 1 39
5 6
5 81 25
1991 9.3 173500
6 6 2 43 6 76 26
1992 8.4 172900 7 78 28
8 10 3 50
1993 7.3 173200 8 85 30

4 54 9 88 30
1994 8.4 173200
10 95 35
1995 7.9 169700 5 56
11 90 29
1996 7.6 174500
12 82 28
6 65
1997 7.6 177900 13 80 26

1998 6.9 188100 14 76 26

15 74 24
1999 7.4 203200
16 73 24
2000 8.1 230200
17 73 20
2001 7 258200
18 77 18
2002 6.5 309800 19 68 19

2003 5.8 329800 20 60 20