8 views

Uploaded by adi.s022

Matrix Class Lecture at the UofM

- Eigenaxes - an aid to understanding eigenvalues and eigenvectors
- dsp file 2003
- Fuzzy Eigenvalues and Fuzzy Eigen Vectors for Fuzzy Matrix
- Acet May2016 Sol
- Eigenvalues and Eigenvectors Note2
- Convex Functions Handout
- Linear Systems Errata
- Matrices and System of Equation
- Home12
- ES 21 First Exam Notes
- CE GATE-2018 Paper 11-FEB-2018 Afternoon Session
- Variationl Principal of Eigenvalues
- An Introduction to Optimization 4th Edition Solution Manual
- Unitary and Hermitian Matrices
- Further Partial Differential Equations
- Inner Product Space
- MOHAMMED_2017_J._Phys.__Conf._Ser._783_012017
- 2010 Using Orientation Zernike Polynomials to predict the imaging performance of optical systems with birefringent and partly polarizing components.pdf
- Design of Sparse Filters for Channel Shortening.pdf
- cs229-notes10.ps

You are on page 1of 38

**COMPUTATIONAL ASPECTS OF MATRIX THEORY
**

Class time : 2:30 – 3:45 pm MW

Room : MechE 212

Instructor : Daniel Boley

URL : www-users.itlabs.umn.edu/classes/Fall-2014/csci5304/

September 3, 2014

Thanks to Yousef Saad for the use of his class notes.

CSci 5304: header– 0-1

GENERAL INTRODUCTION

• Linear algebra and numerical linear algebra

• Types of problems to be seen in this course

• Mathematical background - matrices, eigenvalues, ...

• Quick review of Matlab

• Vector norms, matrix norms

CSci 5304: BG– 1-1

Introduction

® This course is about Matrix algorithms or “matrix com-

putations”

® It involves: algorithms for standard matrix computations

(e.g. solving linear systems) - and their analysis (e.g., their

cost, numerical behavior, ..)

® Matrix algorithms pervade most areas of science and

engineering.

® In computer science: recent increase of interest in matrix

algorithms for data mining, information retrieval, search

engines, pattern recognition, graphics, ...

CSci 5304: BG– 1-2

Examples

® Ancient Chinese Problem (3rd cent BC) [Babylonians

had similar problem]:

There are three types of corn, of which three bundles of

the ﬁrst, two of the second, and one of the third make 38

measures. Two of the ﬁrst, three of the second and one of

the third make 33 measures. And one of the ﬁrst, two of

the second and three of the third make 26 measures. How

many measures of corn are contained in one bundle of each

type?

CSci 5304: BG– 1-3

® Pagerank of Webpages (21st cent AD)

If one were to do a random walk from web page to web

page, following each link on a given web page at random

with equal likelihood, which are the pages to be encoun-

tered this way most often?

CSci 5304: BG– 1-3a

® Vibrations in mechanical systems. See:

www.cs.umn.edu/

~

saad/eig_book_2ndEd.pdf

Problem: Determine the vibration

modes of the mechanical system [to

avoid resonance]. See details in

Chapter 10 (sec. 10.2) of above

reference.

m

1

m

2

l

1

l

2

k

1

k

2

CSci 5304: BG– 1-4

Examples (cont.)

® Method of least-squares (inspired by ﬁrst use of least

squares ever, by Gauss around 1801)

A planet follows an elliptical orbit according to ay

2

+bxy+

cx + dy + e = x

2

in cartesian coordinates. Given a set of

noisy observations of (x, y) positions, compute a, b, c, d, e,

and use to predict future positions of the planet. This

least squares problem is nearly rank-deﬁcient and hence

very sensitive to perturbations in the observations.

Read Wikipedia’s article on planet ceres:

http://en.wikipedia.org/wiki/Ceres_(dwarf_planet)

CSci 5304: BG– 1-5

Background in linear algebra

® Review vector spaces. Read section 1.1 of text on vector

notation.

® A vector subspace of R

n

is a subset of R

n

that is also

a real vector space. The set of all linear combinations of

a set of vectors G = {a

1

, a

2

, . . . , a

q

} of R

n

is a vector

subspace called the linear span of G,

® If the a

i

’s are linearly independent, then each vector of

span{G} admits a unique expression as a linear combina-

tion of the a

i

’s. The set G is then called a basis.

Recommended reading: Sections 1.1 – 1.6 of

www.cs.umn.edu/

~

saad/eig_book_2ndEd.pdf

CSci 5304: BG– 1-6

Matrices

® A real m×n matrix A is an m×n array of real numbers

a

ij

, i = 1, . . . , m, j = 1, . . . , n.

The set of all m×n matrices is a real vector space denoted

by R

m×n

.

® Complex matrices deﬁned similarly.

® A matrix represents a linear mapping between two vector

spaces of ﬁnite dimension n and m.

CSci 5304: BG– 1-7

Operations:

Addition: C = A + B, where A, B, C ∈ R

m×n

and

c

ij

= a

ij

+ b

ij

, i = 1, 2, . . . m, j = 1, 2, . . . n.

Multiplication by a scalar: C = αA, where

c

ij

= α a

ij

, i = 1, 2, . . . m, j = 1, 2, . . . n.

CSci 5304: BG– 1-7a

Multiplication by another matrix:

C = AB,

where A ∈ R

m×n

, B ∈ R

n×p

, C ∈ R

m×p

, and

c

ij

=

n

k=1

a

ik

b

kj

.

CSci 5304: BG– 1-8

Transposition: If A ∈ R

m×n

then its transpose is a matrix

C ∈ R

n×m

with entries

c

ij

= a

ji

, i = 1, . . . , n, j = 1, . . . , m

Notation : A

T

.

CSci 5304: BG– 1-8a

Transpose Conjugate: for complex matrices, the trans-

pose conjugate matrix denoted by A

H

is more relevant:

A

H

=

¯

A

T

= A

T

.

CSci 5304: BG– 1-9

Review: Matrix-matrix and Matrix-vector producs

® Recall deﬁnition of C = A×B.

® Recall what C represents [in terms of mappings]..

® Can do the product column-wise [Matlab notation used]:

C

:,j

=

n

k=1

b

kj

A

:,k

® Can do it row-wise:

C

i,:

=

n

k=1

a

ik

B

k,:

CSci 5304: BG– 1-10

® Can do it as a sum of ‘outer-product’ matrices:

C =

n

k=1

A

:,k

B

k,:

Verify (prove) all 3 formulas above..

Complexity? [number of multiplications and additions]

What happen to these 3 diﬀerent approaches when B

has one column (p = 1)?

CSci 5304: BG– 1-11

Square matrices, matrix inversion, eigenvalues

® Square matrix: n = m - so A ∈ R

n×n

® Identity matrix: square matrix with

a

ij

=

_

1 if i = j

0 otherwise

® Notation: I.

® Property: AI = IA = A

® Inverse of A (when it exists) is a matrix C such that

AC = CA = I

Notation: A

−1

CSci 5304: BG– 1-12

Eigenvalues and eigenvectors

A complex scalar λ is an eigenvalue of the square matrix

A if a nonzero vector u of C

n

exists such that

Au = λu.

The vector u is an eigenvector of A associated with λ.

The set of all the eigenvalues of A is the spectrum of A.

Notation: λ(A).

® λ is an eigenvalue of A if and only if det(A−λI) = 0

® p

A

(λ) = det(A−λI) is a polynomial of degree n in λ

= characteristic polynomial of A.

® λ ∈ λ(A) if and only if λ is a root of the characteristic

polynomial p

A

(λ).

CSci 5304: BG– 1-13

® Spectral radius = The maximum modulus of the eigen-

values

ρ(A) = max

λ∈λ(A)

|λ|.

® Trace of A = sum of diagonal elements of A.

tr(A) =

n

i=1

a

ii

.

® tr(A) = sum of all the eigenvalues of A counted with

their multiplicities.

CSci 5304: BG– 1-13a

® Recall that det(A) = product of all the eigenvalues of

A counted with their multiplicities.

Example: Trace, spectral radius, and determinant of

A =

_

2 1

3 0

_

.

CSci 5304: BG– 1-14

For two n × n matrices A and B are the eigenvalues

of AB and BA the same?

If Ais nonsingular what are the eigenvalues/eigenvectors

of A

−1

?

What are the eigenvalues/eigenvectors of A

k

for a given

integer power k?

What are the eigenvalues/eigenvectors of p(A) for a

polynomial p?

Review the Jordan canonical form. Deﬁne the eigen-

values, and eigenvectors from the Jordan form.

CSci 5304: BG– 1-15

Range and null space

® Range: Ran(A) = {Ax | x ∈ R

n

} ⊆ R

m

® Null Space: Null(A) = {x ∈ R

n

| Ax = 0 } ⊆ R

n

® Range = linear span of the columns of A

® Rank of a matrix rank(A) = dim(Ran(A)) ≤ n

CSci 5304: BG– 1-15a

® Ran(A) ⊆ R

m

→ rank (A) ≤ m →

rank (A) ≤ min{m, n}

® rank (A) = number of linearly independent columns of

A = number of linearly independent rows of A

® A is of full rank if rank(A) = min{m, n}. Otherwise

it is rank-deﬁcient.

CSci 5304: BG– 1-16

Rank+Nullity theorem for an m×n matrix:

dim(Ran(A)) + dim(Null(A)) = n

Apply to A

T

: dim(Ran(A

T

))+dim(Null(A

T

)) = m →

rank(A) + dim(Null(A

T

)) = m

® Terminology:

• dim(Null(A)) is the Nullity of A

• n −rank(A) is called the co-rank of A

• so: co-rank of A equals nullity of A.

CSci 5304: BG– 1-17

Show that A ∈ R

n×n

is of rank one iﬀ [if and only if]

there exist two nonzero vectors u and v such that

A = uv

T

.

What are the eigenvalues and eigenvectors of A?

Is it true that

rank(A) = rank(

¯

A) = rank(A

T

) = rank(A

H

) ?

Matlab exercise: explore the matlab function rank.

Find the range and null space of the matrix

_

_

_

_

_

−1 1 0

1 2 3

1 −2 −1

2 −1 1

_

_

_

_

_

Verify your result with matlab.

CSci 5304: BG– 1-18

CSci 5304: BG– 1-19

Types of matrices (square)

• Symmetric A

T

= A. • Skew-symmetric A

T

= −A.

• Hermitian A

H

= A. • Skew-Hermitian A

H

= −A.

• Normal A

H

A = AA

H

.

• Nonnegative a

ij

≥ 0, i, j = 1, . . . , n

• Similarly for nonpositive, positive, and negative matrices

• Unitary Q

H

Q = I.

CSci 5304: matrices– 1-20 GvL (2.1);Heath 2.1; TB 1-2

[Note: Common useage restricts this deﬁnintion to com-

plex matrices an orthogonal matrix is a unitary real matrix

– not very natural ]

• Orthogonal Q

T

Q = D [orthogonal columns]

® The term “orthonormal” matrix is never used.

CSci 5304: matrices– 1-21 GvL (2.1);Heath 2.1; TB 1-2

What is the inverse of a unitary matrix?

What can you say about the diagonal entries of a skew-

symmetric (real) matrix?

What can you say about the diagonal entries of a

Hermitian (complex) matrix?

What can you say about the diagonal entries of a skew-

Hermitian (complex) matrix?

The following types of matrices are normal [true-false]:

real symmetric, real skew-symmetric, complex Hermitian,

complex skew-Hermitian.

Find all real 2 ×2 matrices that are normal.

Show that any triangular matrix that is normal is diag-

onal

CSci 5304: matrices– 1-22 GvL (2.1);Heath 2.1; TB 1-2

Matrices with structure

• Diagonal a

ij

= 0 for j = i. Notation :

A = diag (a

11

, a

22

, . . . , a

nn

) .

• Upper triangular a

ij

= 0 for i > j.

• Lower triangular a

ij

= 0 for i < j.

• Upper bidiagonal a

ij

= 0 for j = i or j = i + 1.

• Lower bidiagonal a

ij

= 0 for j = i or j = i −1.

• Tridiagonal a

ij

= 0 when |i −j| > 1.

CSci 5304: matrices– 1-23 GvL (2.1);Heath 2.1; TB 1-2

• Banded a

ij

= 0 only when i − m

l

≤ j ≤ i + m

u

,

‘Bandwidth’ = m

l

+ m

u

+ 1.

• Upper Hessenberg a

ij

= 0 when i > j + 1. Lower

Hessenberg matrices can be deﬁned similarly.

• Outer product A = uv

T

, where both u and v are vectors.

• Block tridiagonal generalizes tridiagonal matrices by

replacing each nonzero entry by a square matrix.

CSci 5304: matrices– 1-24 GvL (2.1);Heath 2.1; TB 1-2

Special matrices

Vandermonde :

® Given a column of entries [x

0

, x

1

, · · · , x

n

]

T

put its (com-

ponent-wise) powers into the columns of a matrix V :

V =

_

_

_

_

_

1 x

0

x

2

0

· · · x

n

0

1 x

1

x

2

1

· · · x

2

1

.

.

.

.

.

.

.

.

.

.

.

.

1 x

n

x

2

n

· · · x

n

n

_

_

_

_

_

Try the matlab function vander

What does the matrix-vector product V a represent?

Interpret the solution of the linear system V a = y

where a is the unknown. Sketch a ‘fast’ solution method

based on this.

CSci 5304: matrices– 1-25 GvL (2.1);Heath 2.1; TB 1-2

CSci 5304: matrices– 1-26 GvL (2.1);Heath 2.1; TB 1-2

Toeplitz :

® Entries are constant along diagonals, i.e., a

ij

= r

j−i

.

® Determined by m + n −1 values r

j−i

.

T =

_

_

_

_

_

_

_

_

r

0

r

1

r

2

r

3

r

4

r

−1

r

0

r

1

r

2

r

3

r

−2

r

−1

r

0

r

1

r

2

r

−3

r

−2

r

−1

r

0

r

1

r

−4

r

−3

r

−2

r

−1

r

0

_

_

_

_

_

_

_

_

. ¸¸ .

Toeplitz

CSci 5304: matrices– 1-26a GvL (2.1);Heath 2.1; TB 1-2

® Toeplitz systems (m = n) can be solver in O(n

2

) ops.

® The whole inverse (!) can be determined in O(n

2

) ops.

Explore toeplitz(c,r) in matlab.

CSci 5304: matrices– 1-27 GvL (2.1);Heath 2.1; TB 1-2

Hankel : Entries are

constant along anti-

diagonals, i.e., a

ij

=

h

j+i−1

.

Determined by m+n−

1 values h

j+i−1

.

H =

_

_

_

_

_

_

_

_

h

1

h

2

h

3

h

4

h

5

h

2

h

3

h

4

h

5

h

6

h

3

h

4

h

5

h

6

h

7

h

4

h

5

h

6

h

7

h

8

h

5

h

6

h

7

h

8

h

9

_

_

_

_

_

_

_

_

. ¸¸ .

Hankel

CSci 5304: matrices– 1-27a GvL (2.1);Heath 2.1; TB 1-2

Circulant : Entries

in a row are cyclically

right-shifted to form

next row. Determined

by n values.

C =

_

_

_

_

_

_

_

_

v

1

v

2

v

3

v

4

v

5

v

5

v

1

v

2

v

3

v

4

v

4

v

5

v

1

v

2

v

3

v

3

v

4

v

5

v

1

v

2

v

2

v

3

v

4

v

5

v

1

_

_

_

_

_

_

_

_

. ¸¸ .

Circulant

CSci 5304: matrices– 1-27a GvL (2.1);Heath 2.1; TB 1-2

Explore hankel(c,r) in matlab.

How can you generate a circulant matrix in matlab?

CSci 5304: matrices– 1-28 GvL (2.1);Heath 2.1; TB 1-2

Sparse matrices

® Matrices with very few nonzero entries – so few that

this can be exploited.

® Many of the large matrices encountered in applications

are sparse.

® Main idea of “sparse matrix techniques” is not to rep-

resent the zeros.

® This will be covered in some detail at the end of the

course.

CSci 5304: matrices– 1-29 GvL (2.1);Heath 2.1; TB 1-2

- Eigenaxes - an aid to understanding eigenvalues and eigenvectorsUploaded byThirumal Venkat
- dsp file 2003Uploaded byANAKJ
- Fuzzy Eigenvalues and Fuzzy Eigen Vectors for Fuzzy MatrixUploaded byarcherselevators
- Acet May2016 SolUploaded bypooja sharma
- Eigenvalues and Eigenvectors Note2Uploaded byWiloeD
- Convex Functions HandoutUploaded byRicardo E.
- Linear Systems ErrataUploaded bymxjoe
- Matrices and System of EquationUploaded bySufi Jutawan Kosmetik
- Home12Uploaded byJacob Higer
- ES 21 First Exam NotesUploaded byGil Kenneth San Miguel
- CE GATE-2018 Paper 11-FEB-2018 Afternoon SessionUploaded byNishit kumar
- Variationl Principal of EigenvaluesUploaded byKumari Ujwala Sinha
- An Introduction to Optimization 4th Edition Solution ManualUploaded byFaraz Hassan Khan
- Unitary and Hermitian MatricesUploaded byJo Popsc
- Further Partial Differential EquationsUploaded byalexandrosmf
- Inner Product SpaceUploaded byDharmendra Kumar
- MOHAMMED_2017_J._Phys.__Conf._Ser._783_012017Uploaded bycyberea
- 2010 Using Orientation Zernike Polynomials to predict the imaging performance of optical systems with birefringent and partly polarizing components.pdfUploaded bymod_abo_hashish
- Design of Sparse Filters for Channel Shortening.pdfUploaded bywuchennuaa
- cs229-notes10.psUploaded byKristina Nikkhah
- Cross-Correlation Dynamics in Financial Time SeriesUploaded byTraderCat Solaris
- QMB09sol_2Uploaded byMuzamil Shah
- 29881254709860Uploaded bysacharya2011
- MA ECONOMICS 1ST SEMESTERUploaded byNaresh Sehdev
- Damage Detection by Finite Element Model Updating Using Modal Flexibility ResidualUploaded by3come
- Application of Stochastic Modeling in GeomagnetismUploaded byInternational Journal of Research in Engineering and Technology
- Sign Restriction BinningUploaded byAllister Hodge
- 1482238985Uploaded bykslag-1
- Uzawa-SOR Method for Fuzzy Linear SystemUploaded bySEP-Publisher
- Octopus Exercise3Uploaded byLucas Berti

- Lec04 PerturbationUploaded byadi.s022
- Lec07-LeastSquaresUploaded byadi.s022
- Lec06-SpecialSystemsUploaded byadi.s022
- Lec05 SystemsUploaded byadi.s022
- Lec08-SVDUploaded byadi.s022
- chap02Uploaded byadi.s022
- 00b7d51f18159dbbb1000000Uploaded byadi.s022
- chap03Uploaded byadi.s022
- Lec03 FloatsUploaded byadi.s022
- Hartshorne Algebraic Geometry SolutionsUploaded byΠΤΟΛΕΜΑΙΟΣ ΗΡΑΚΛΕΙΔΗΣ
- Failure to Thrive With NotesUploaded byadi.s022

- System of EquationsUploaded byWill Matcham
- ch03_Uploaded byMohammad Bilal Mirza
- Unit 4A Student Guide HONORS 16-17Uploaded byAurolver
- Introduction to Descriptive GeometryUploaded byAsim Beciragic
- 01. Rational NumbersUploaded bythinkiit
- 1997PascalContest.pdfUploaded bySahabudeen Salaeh
- calculus0_10Uploaded byCristian Borquez
- If EmUploaded byapi-3700440
- Msc syllabusUploaded bysairajdream
- 2008 comp soUploaded byDwayne Fuller
- irrational numbers study guide keyUploaded byapi-276774049
- PDE_Syllabus_S10Uploaded bylenin_eda
- Unit Step FunctionUploaded byDavid Pavković
- Jarkko Kari- Cellular Automata: TutorialUploaded byldkgfd
- conic sections and circles 2Uploaded byapi-339611548
- Lagrange multiplier.pdfUploaded byjcblackmore
- Shape From MetricUploaded bypinkall
- 5A_Practice_46-49Uploaded bySanja Erceg
- Lie Groups HooftUploaded byJosh Burke
- Vector _ Paul's Blog@WildbunnyUploaded byIon Mincu
- 110Mid-1-2010Uploaded bygeocodes
- Section 6 Appendix Euclidean AlgorithmUploaded byty391988
- Fuzzy Soft B-Open Sets in Fuzzy Soft Topological SpaceUploaded byIJSTE
- MilneSimpsonProof.pdfUploaded byMadhur Mayank
- Precal Semester 1 StandardUploaded byfrankdan
- Eigenmath manualUploaded byLERINES
- Topology Exercises and Sol1Uploaded byChris Walsh
- 2012 PWMC IndividualUploaded byodan81
- Am Week10 s2Uploaded byKannan Sreenivasan
- Mat Lab SheetUploaded byfemme.