A tutorial on the total least squares method for
fitting a straight line and a plane

© All Rights Reserved

16 views

A tutorial on the total least squares method for
fitting a straight line and a plane

© All Rights Reserved

- Seed sharing network properties and agroecosystem resilience - Presentation
- Principle Component Analysis
- Report Project
- j03. 1999_IEEE_TAC
- Notes1
- MATH461 Final Exam Spring 2009
- 34_ftp
- M. Bertola, M. Gekhtman and J. Szmigielski- Peakons and Cauchy Biorthogonal Polynomials
- Bộ PSS
- HW
- 9-estimation paramétriques pour les modèles de réacteurs
- Michelsen the Isothermal Flash Problem 1 Stab
- Differential Equations an Operational Approach
- Student Lecture 30 Special Matrices
- NM-Ang
- Li_The Effects of Distributed Masses on Acoustic
- Optimum Self-stress Design of Cable–Strut Structures
- Ackerman Proof Caylay
- Physical Interpretation of Mode Shapes and Oscillations W_ Examples
- Cleaning Correlation Matrices

You are on page 1of 6

fitting a straight line and a plane

Leonardo Romero Muoz, Moiss Garca Villanueva and Cuauhtmoc Gmez Surez,

Facultad de Ingeniera Elctrica, UMSNH, Mexico

AbstractThe classic least squares regression fits a line to data

where errors may occur only in the dependent variable, while the

independent variable is assumed to have no errors. The total least

squares regression fits a line where errors may occur in both

variables. This tutorial provides an introduction to the method of

total least squares supplementing a first course in statistics or

linear algebra.

I. INTRODUCTION

Detecting geometric features (lines, circles, surfaces, etc.)

from data points is a fundamental task in several fields of

science and engineering; for instance, metrology, computer

vision, mobile robotics, etc.

Let = { , , } be a set of measurements or points

where each point = , is represented by its rectangular

coordinates. A linear relation between

and

is usually

written as

(1)

=

+

where is the slope of the straight line and is the y-axis

intersection. In the classic Least Squares (LS) the abscissa

data ( , = 1, , ) are assumed to be known exactly while

the uncertainties of the ordinate data ( ) are used as weights

for fitting the line , , given by (1), to the set of

measurements .

The solution to fit a line using the least squares regression,

appears with complete derivations in textbooks at many levels:

calculus, linear algebra, numerical analysis, probability,

statistics, and others.

However, measured data are never free of uncertainty. This

means, in order to determine a best fit to a line, a method is

and

data

required which takes the uncertainties of the

into account [3]. The Total Least Squares regression (TLS)

was introduced by Golub and Van Loan [2] to deal with both

uncertainties. Despite its usefulness and its simplicity, TLS

has not yet appeared in numerical analysis, statistics or linear

algebra texts.

Introducing students to TLS is the purpose of this tutorial,

and it may complement the usual courses in numerical

analysis, statistics or linear algebra, or serve as a transition

from such courses to a more advanced and specialized course.

L. Romero Muoz, Facultad de Ingeniera Elctrica, Ciudad Universitaria,

Universidad

Michoacana,

58000,

Morelia,

Mxico

(e-mail:

lromeromunoz@gmail.com).

M. Garca Villanueva. Facultad de Ingeniera Elctrica, Ciudad

Universitaria, Universidad Michoacana, 58000, Morelia, Mxico (e-mail:

moigarciav@gmail.com).

C. Gmez Surez. Facultad de Ingeniera Elctrica, Ciudad Universitaria,

Universidad

Michoacana,

58000,

Morelia,

Mxico

(email:

temocgs@gmail.com)

Yves Nievergelt [5]; an overview of the TLS methods, by Ivan

Markovsky and Sabine Van Huffel [4]; or the book by Sabine

Van Huffel and Joos Vandewalle, about the TLS problem [7].

II. PRELIMINARIES

A. Normal form of a line

There is a disadvantage of using equation (1) to represent a

line: vertical lines can not be represented, because . To

avoid this problem, a line in the plane is represented by its

normal form,

(2)

= ,

where and are the length and the angle of inclination of

the normal, respectively. As shown in Figure 1, the normal is

the shortest segment between the line and the origin of a given

coordinate frame. Using this form, points = , that are

on the line = , satisfy

= x cos

+ sin

(3)

Fig. 1. Line parameters in the normal form. The shortest distance from the

.

origin to the line is =

representation , given by eq. (1) can be obtained dividing

eq. (3) by sin (while 0, avoiding vertical lines) and

reordering terms:

cos

+

=

sin

sin

1

(4)

=

+

tan

sin

From this last equation, the parameters of the line , are

given by:

=

,

=

(5)

with respect to the parameters of the line ( and ) vanish:

The shortest distance from a given point = , to a

( , ), is easily computed as

line = , , denoted by

follows.

A line through the point , parallel to line , is given

by:

= cos + sin

=0.

cos

sin

( cos

with parameters

, and the line with parameters , is the difference

= , because both lines have the same (see Figure

2). So the desired distance, called orthogonal distance, is

( , ) =

= 0 first

Lets do

( cos

of

=0

)=0

( )

( )=0

( )

1

( ) + sin

) )

sin

( ) + sin

cos

cos

sin

( ) + sin

cos

(6)

=0

( ) =0

(10)

and , defined as follows

=

(11)

cos

Fig. 2. Orthogonal distance

III.

from point

to line .

In the literature, the problem of fitting a straight line to

data with errors in both coordinates was first formulated by

Pearson as early as in 1901 [3]. Deming in 1943 [1] suggests

to minimize the sum

( )

( )

(, , , ) =

+

(7)

,

uncertainties ( , , , ) and ( , ) denote its corresponding

point of the straight line . The best line minimizes . In the

case , = , = , = 1, , , the problem is reduced to

the so-called total least-squares problem and minimizing (7) is

equivalent to minimizing the orthogonal distance of the

measured points ( , ) to the fitting line. Therefore, this is

also often referred to as orthogonal regression [3]. In this case,

the best line minimizes

( , )

(; ) =

(8)

B. Finding the best line

Replacing eq. (6) into (8), the best line minimizes

1

(; ) =

( cos + sin )

+ sin

=0

= cos + sin

(12)

result:

The centroid of points given by , is a point of the line

with parameters , which minimizes eq. (9).

Replacing eq. (12) into (9), (; ) can be expressed as

1

(; ) =

(13)

[( ) cos + ( ) sin ]

In eq. (13) the only unknown parameter is

= 0, in order to find the right .

1

( [( ) cos + (

1

) cos

2 [(

) sin

[(

(

2(

) 2 cos

)(

sin

+(

) sin

(9)

2 cos

2(

sin

)(

) sin ] )

+(

+(

+ 2(

. So, lets do

=0

) sin ]

) cos ] = 0

)(

) cos

) 2 cos

sin

[(

=0

) ( ) ] +

)(cos

sin

)=0

(14)

(

Using the follo

owing trigonom

metric identitiees

sin 2 = 2 co

os

sin

, co

os 2 = cos

D. M

Matrix form to obtain the anggle

sin

(15)

(

sin 2 [((

2(

)(

[((

sin 2

2 cos 2

) (

1

(; ) =

(17)

) ] +

) cos

c 2 =0

) (

) ] +

[(

)=0

)(

) )]

sin 2

2 [( )(

2

=

[( ) ( ) ]

cos 2

1

2 [( )( )]

= arctan

[(

2

[ ) ( ) ]

tan 2 =

(16)

(

g the desired

d line parametters

, . In practicce, equation (16) uses the four

f

quadrant arc

tanngent (atan2).

c

arctaan( ) but uses

u

2( , ) computes

thee signs of both

h and to determine

d

the quadrant

q

in wh

hich

thee resulting angle

a

lies. Fo

or example

2(2, 2)) =

1

135, whereaas

2(2,2)) = 45, a distinction

d

wh

hich

woould be lost with

w

a single-aargument arc tangent functiion.

Annother practicaal consideratio

on must be don

ne when eq. (12)

(

givves an < 0. In that case, the

t line , , where =

annd

= + , represents th

he same line , , but in this

t

reppresentation

0.

Fig. 3. Linne fitting minimiziing orthogonal disstances from pointss to line (TLS).

C. Example

Consider the data given in Table I. We want

w

to determ

mine

thee line of total least squares fo

or these points.

TAB

BLE I

AN EXAMPLE WITH

W

7 POINTS.

pooint

3

7

4

7

5

11

6

11

7

15

5

8

16

9

19

1

meters, we usse equations (1

11),

(166) and (12),

=7

3 + 4 + + 8 + 9

=

=6

7

7 + 7 + + 16

1 + 19

=

= 12..286

7

1

1

116

= arctan

= 0.43

361

2

97.4

429

= 0.24889

ws the line ass well as the points and th

heir

Figure 3 show

ortthogonal distances. For refeerence, Figure 4 shows the line

l

obbtained using th

he classical Leeast Square Meethod. In the saame

Figgure the vertical distances fro

om points to th

he line are show

wn.

Fig. 4. Linne fitting minimiziing vertical distances from points to line (LS).

where

2,

is a matrix of dimension

is a vecctor,

cos

sin

coordinaates [

] , defined ass

=

+ +

(18)

with

(19)

(17). In other words, a vector which minimizes the norm of

the linear map:

. Note that is a unit vector, because

= cos

+ sin

= 1.

To achieve this goal, the Euclidean norm also can be

expressed using de inner product of the vector and itself,

(20)

where

denotes the transpose of vector . Using the inner

product to compute the norm, eq. (17) can be written as

(

(;Z)=

)(

(

)(

(23)

,

,

=

=

,

,

0

,

,

and second

Let

be a matrix which first column is

column is ; and let be a diagonal matrix with elements

and . Using these matrices we can write last equation in a

simpler form

(26)

=

The orthonormal matrix has an interesting property: its

inverse is its transpose (

=

= , where is the

identity matrix). Using this property and equation (26), the

matrix can be expressed in terms of and ,

(27)

=

Replacing the matrix , given by the equation (27), into

equation (22),

,

{ (

)

0

0

) +

) }

(29)

and

and are the coordinates of unit vectors and

orthogonal vectors, we have

= cos

= cos( 2) = sin

where is the angle between vector

results into equation (28),

1

1

,

are

(30)

(31)

cos

sin

cos

+ (1 cos

)}

(32)

( = 0). Using

= 1 when vector is identical to vector

this new variable in Equation (32), we have finally

1

(; ) =

{

+ (1 )}

1

(; ) =

{( ) + }

(33)

The expression ( ) +

corresponding to a line

, with a negative slope =

(because

< )

and y-axis intersection

= . The maximum value of

(, ) is

when = 0, and the minimum value is

when = 1. Therefore,

,

,

= cos

(; ) =

(28)

To see the maximum and minimum value of , suppose

< . Taking into account that the inner product of

that

two vectors with coordinates and is defined as

)(

(; ) =

(; ) =

0 for 0), matrix

) and is positive semidefinite (

0,

0; and two

has two real eigenvalues:

orthonormal eigenvectors (unit vectors with inner product

equal to zero).

Let

=[ ,

and

=[ ,

be the

, ]

, ]

coordinates of eigenvectors

and

. Eigenvalues and

eigenvectors are related by

=

=[ ,

(24)

, ]

=

=[ ,

(25)

, ]

Equations (24) and (25) can be joined into a single relation

,

(; ) =

, of dimension 2 2,

(; ) =

(21)

and the associative law for the matrix product, we get a

quadratic form

1

(; ) =

(

)( )

1

(; ) =

(

)

1

(; ) =

(22)

(; ) =

of matrix

, minimizes

minimum eigenvalue

(, ).

From

= [u , , u , ] and

desired angle ,

= arctan(

= [cos

,

(34)

(eq. (34)), we can get the parameter of the line using eq.

(12),

= , +

(35)

,

Using equation (3) and previous results, we can see an

,

interesting property of vector

= cos + sin

+

, =

, +

0 = ( ) , + ( )

=( )

highest and the lowest point. The final steps are to undo the

rotation and translation made,

,

,

(36)

where = [

] , and = [ ] is the centroid of points.

must be orthogonal

Note that vector with coordinates

to vector , in order to satisfy equation (36).

E. Example (cont.)

Continuing the example from Section III-C, we can

compute the matrix ,

28.000 58.000

(37)

=

58.000 125.429

= ( )

= +

,

,

= ( )

= +

(44)

(45)

As an example, Figure 5 shows five line segments

computed from a set of points given by measurements of an

Infrared sensor of a small mobile robot, using the Line

Tracking algorithm [6] (with TLS). The robot rotates 360

taking measurements.

the octave program):

= 0.97076,

= 152.45781

(38)

= [0.90641 0.42241]

= [0.90641 0.42241]

(39)

From eigenvector

0.42241

(40)

= 2.7055

= arctan

0.90641

value for . In this case

= + , and we get the same

result for as in section III-C.

IV.

Fig. 5. Finding the best lines of a set of points given by an IR sensor of a small

mobile robot.

to the set of points , instead of only the infinite line

expressed by the parameters , . This section addresses this

problem.

First we move the origin of coordinates to the centroid of

points

= [ ] . For each point

=[

] , the

translated point is defined by

=

(41)

V. SOME EXTENSIONS

A. Weighted total least squares

In section III-A we consider the same uncertainty for all

for point

=

points. If we consider an uncertainty

1, , , the best line minimizes

( , )

(; ) =

(46)

Following a procedure similar to section III-B, we can get

the solution

( ) = cos

sin

sin

cos

(42)

counter-clockwise through an angle about the origin of the

Cartesian coordinate system. To perform the rotation using a

rotation matrix , the position of each point must be

represented by a column vector , containing the coordinates

of the point. A rotated vector is obtained by using the matrix

multiplication .

If we rotate all points an angle , the rotated points

follows a vertical line,

= ( )

= cos

1

=

2

(43)

Let

and

be the maximum and minimum values

respectively of all coordinates

, in

=[

], =

1, , . The points

= [0

] and

= [0

] are the

+ sin

2

[(

( )( )

) ( ) ]

(47)

(48)

where

=

(49)

(50)

= 1

are the weighted means; with individual weights

for each measurement. This approach is known as weighted

least squares.

B. Fitting a set of points to a plane

The method to find the best line in the total least squares

points in three dimensions.

A plane

is represented by four parameters =

, , , , where , , and are the length of the normal,

and the angle between the normal and the x-axis, y-axis and zaxis respectively. The normal is the shortest line segment

between the plane and the origin.

A point = , , that is on the plane satisfy

=

cos

+ cos

+ cos

(51)

to plane

is given

cos

(52)

by

( , )=

cos

cos

(53)

centroid of points given by , , is a point of the plane

which minimizes eq. (53),

+ cos

Squares to fit a line, developing a closed formula for

calculating the line parameters; and also the matrix

formulation, which can be easily extended to the case of fitting

a Cartesian plane to points in three dimensions.

It is written in a simple form which should be easy to

understand, even for students with a basic knowledge of

calculus and linear algebra. Also, the closed formulation is

useful for low computational resources, such as

microcontrollers for small mobile robots.

VII. REFERENCES

[2]

( , )

Doing

= cos

VI. CONCLUSION

[1]

( ; )=

points.

+ cos

[3]

[4]

[5]

[6]

(54)

where

[7]

Sciences. New York: Wile, 1943.

Gene H Golub. Some modified matrix eigenvalue problems. Siam

Review, 15(2):318-334, 1973.

Michael Krystek and Mathias Anton. A weighted total least-squares

algorithm for fitting a straight line. Measurement Science and

Technology, 18(11)_3438, 2007.

Ivan Markovsky and Sabine Van Huffel. Overview of total least-squares

methods. Signal processing, 87(10):2283-2302, 2007.

Yves Nievergelt. Tottal least squares: State-of-the-art regression in

numerical analysis. SIAM Rev., 36(2):258-264, June 1994.

L. Romero Muoz, M. Garcia Villanueva, and C.A. Lara Alvarez. An

extended line tracking algorithm. In 2013 IEEE International Autumn

Meeting on Power; Electronics and Computing (ROPEC), pages 1-5,

Now 2013.

Sabine Van Huffel and Joos Vandewalle. The total least squares

problem: computational aspects and analysis, volumen 9. Siam, 1991.

(55)

VIII. BIOGRAPHIES

the matrix formulation. Replacing equations (52) and (54) into

(53),

1

(56)

( ; )=

where

3,

is a matrix of dimension

is a vector,

cos

cos

=

cos

(57)

Note

that

is

a

unit

vector,

because

|| ||= cos + cos + cos = 1.

The best plane is given by =

=[ ,

,

, ] , the

of matrix

eigenvector associated to the smallest eigenvalue

. From =

we can obtain , using equation (54),

=

(58)

=

where

=[

+

=(

,

] , and

=[

+

)

(59)

(60)

] is the centroid of

Michoacan, Mexico. He studied electrical engineering in the

Universidad Michoacana. He got a master's and Ph.D. in

Computer Science at ITESM Campus Morelos, Mexico, in

1990 and 2002 respectively. He is currently a Professor and

Researcher at the Faculty of Electrical Engineering of the

Universidad Michoacana. His research interests are

computer vision, robotics and probabilistic reasoning.

Moises Garcia Villanueva was born in Patzcuaro,

Michoacan, Mexico. He received the degree of Electrical

Engineer and Master in Electrical Engineering with option

in Computational Systems at the Faculty of Electrical

Engineering of the Universidad Michoacana in 1999 and

2001 respectively. He is currently a Professor and

Researcher in the same Faculty. His areas of interest

include pattern recognition, computer vision, robotics and data mining.

Cuauhtemoc Gomez Suarez received the degree of

Electronic Engineering and Master in Electrical

Engineering, with option in Computational Systems in the

Universidad Michoacana, Mexico. He currently serves as

a teacher of some courses at the Faculty of Electrical

Engineering of the same university. His areas of interest

are robotics and three-dimensional reconstruction.

- Seed sharing network properties and agroecosystem resilience - PresentationUploaded byHumanCoral
- Principle Component AnalysisUploaded bymatthewriley123
- Report ProjectUploaded bynvrajesh007
- j03. 1999_IEEE_TACUploaded byDimitris Iracleous
- Notes1Uploaded byMichael Garcia
- MATH461 Final Exam Spring 2009Uploaded bymosalshehhi2
- 34_ftpUploaded byjopiter
- M. Bertola, M. Gekhtman and J. Szmigielski- Peakons and Cauchy Biorthogonal PolynomialsUploaded byPomac232
- Bộ PSSUploaded byMua Gio Bien
- HWUploaded byApam Benjamin
- 9-estimation paramétriques pour les modèles de réacteursUploaded byarfpower
- Michelsen the Isothermal Flash Problem 1 StabUploaded byjhosimar97
- Differential Equations an Operational ApproachUploaded bycessahector
- Student Lecture 30 Special MatricesUploaded byuploadingperson
- NM-AngUploaded byNaca Maja
- Li_The Effects of Distributed Masses on AcousticUploaded byshariefsm2
- Optimum Self-stress Design of Cable–Strut StructuresUploaded byNguyen Tan Tien
- Ackerman Proof CaylayUploaded byzulqar8228
- Physical Interpretation of Mode Shapes and Oscillations W_ ExamplesUploaded byRicardo Colosimo
- Cleaning Correlation MatricesUploaded bydoc_oz3298
- Damage assessment of structure using incomplete measured mode shapes.pdfUploaded byCamilo Manrique
- 03-MatricesDeterminants-2Uploaded byAria Carino
- Linear Algebra Done RightUploaded by88915334
- Tsang 1984Uploaded byAnonymous yjLUF9gDTS
- T1_MatrizesUploaded byJoana
- Mat Lab Notes for ProfessionalsUploaded byconrado
- 21-electroscienceUploaded byeddisonfh
- Tutorial 4Uploaded byMelchizedek Mashambo
- cdc-zai8Uploaded byRampyare Yadav
- HW12Uploaded by123chess

- Philips Essential CatalogUploaded byAllison Saw
- Linear AntennaUploaded byARN18
- ByJack UngerUploaded byranny17
- TR_A320 MLG - ServicingUploaded byJohn Detrick
- afrpowUploaded byTosin Omonijo
- System Analysis and DesignUploaded byAhamed Nishadh
- M.S.thesis Kumar_Analytical Solution for IHCPUploaded byJinsoo Kim
- 1922 Bangwa AssessmentUploaded byFranklin Ngosong Tougwa
- Safe Port Issues in the Context of Chittagong Port Captain Mak CiltUploaded byMohiuddin Abdul Kadir-Razia Sultana
- Biologia celular de la vasopresina.pdfUploaded byramosmarquez
- Single Duct UnitsUploaded byMike Arista
- Predictive Astrology EBookv1Uploaded bySasho Spasovski
- Document.pdfUploaded byGeorgian Georgescu
- 2 Rotor ShaftUploaded byatulyajyoti123
- CX310-510_SMUploaded byChristiano Dos Santos
- Volume 5 Issue 10Uploaded bythestuyvesantstandard
- Guelph Community Energy PlanUploaded byRyan
- PL-2303 Mac OS X Driver Installation GuideUploaded bycorneliusflavius7132
- v2aa01Uploaded byMohamed Selim
- The Last of the Legions and Other TalesUploaded byatom33
- 10Uploaded bysathish14singh
- Lab Write UpUploaded byAdam Graham Erickson
- 2FRM 6Uploaded byRomly Mech
- Forbidden City Assignment (1)Uploaded byArif Ahmed
- Durthu Rules PDFUploaded bydangerbunny23
- Surgical Treatment of Perforated Peptic UlcerUploaded byAWw Lieya
- Summer TrainingUploaded bySiddharth Garkhail
- THE CRITIQUE OF ARAB THOUGHT- MOHAMMED ARKOUN’S DECONSTRUCTION METHODUploaded byYasser Munif
- 2017 Exhibiting CompaniesUploaded byUmasankar Chakravorti
- Akai-AM2200 ampUploaded byRinaldo Rak

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.