4 views

Uploaded by Jr Ang

Numerical Analysis

- Python Week8 Lecture4 Handout
- Linear Algebra 01
- Completely formated DS Programs
- DSP LAB 2
- Matlab Tutorial PartI 2005
- Chapter2b
- 01 Basic R
- 3. MATLAB Variables
- MATH Final Exam Review Sheet
- Special Fuzzy Matrices for Social Scientists, by W.B.Vasantha Kandasamy, F.Smarandache, K.Ilanthenrall
- Vectors and Matrices
- 43605-0132369753_8
- Wap to Do Transpose of a Matrix
- (Graduate Studies in Mathematics 128) J. M. -Tensors_ Geometry and Applications-American Mathematical Society (2012)
- comp1hw
- Mathematics II Jun2003 or 210156
- Typed Lecture for Sec 7_1
- Chap12 Slides
- Smart Base Isolated Benchmark Building-Part I: Problem Definition
- Kinematic Modeling of Wheeled Modile Robots

You are on page 1of 8

vectors

inner products and linear functions

angle between vectors

matrices

matrix-vector product

matrix-matrix product

block matrix notation

cost of matrix operations

11

Vectors

n-vector x:

x =

_

_

x

1

x

2

.

.

.

x

n

_

_

also written as x = (x

1

, x

2

, . . . , x

n

)

set of n-vectors is denoted R

n

x

i

: ith element or component of x

special vectors (size follows from context):

x = 0 (zero vector): x

i

= 0, i = 1, . . . , n

x = e

i

(ith basis vector or ith unit vector ): x

i

= 1, x

k

= 0 for k = i

Vectors and matrices 12

Vector operations

scalar multiplication of a vector x with a scalar

x =

_

_

x

1

x

2

.

.

.

x

n

_

_

addition and subtraction of two n-vectors x, y

x +y =

_

_

x

1

+y

1

x

2

+y

2

.

.

.

x

n

+y

n

_

_

, x y =

_

_

x

1

y

1

x

2

y

2

.

.

.

x

n

y

n

_

_

Vectors and matrices 13

Geometrical interpretation

for n 3: x is a point with coordinates x

i

example: x = (4, 0), y = (2, 2)

x

0.75x

y

1.5y

0.75x + 1.5y

0.75 x =

_

3

0

_

, 1.5 y =

_

3

3

_

, 0.75 x + 1.5 y =

_

6

3

_

Vectors and matrices 14

Examples of vectors

sampled signal (e.g., audio signal)

a function u(t) of time, sampled with period T for n 1 periods

u(t)

t

0 T 2T 3T

(n 1)T

x

1

x

2

x

3

x

4

xn

can be represented as an n-vector with

x

1

= u(0), x

2

= u(T), . . . , x

n

= u((n 1)T)

Vectors and matrices 15

image: N N pixels, numbered 1 to N

2

1 2 3 N

N + 1 N + 2 N + 3 2N

2N + 1 2N + 2 2N + 3 3N

N

2

N

+ 1

N

2

N

+ 2

N

2

N

+ 3

N

2

grayscale image: can be represented by a vector x of size n = N

2

;

x

i

is the grayscale level of pixel i

color image: color of each pixel is given by three values (red, green, blue

intensity); image can be represented by three vectors x, y, z of size N

2

Vectors and matrices 16

Inner products

the inner product of two n-vectors x, y is dened as

x

1

y

1

+x

2

y

2

+ +x

n

y

n

notation: x

T

y

properties

(x)

T

y = (x

T

y) for scalar

(x +y)

T

z = x

T

z +y

T

z

x

T

y = y

T

x

Vectors and matrices 17

Linear functions

denition: a function f : R

n

R is linear if

f(x +y) = f(x) +f(y)

for all scalars , and all n-vectors x, y

property: f is linear if and only if it can be written as

f(x) = a

T

x = a

1

x

1

+a

2

x

2

+ +a

n

x

n

for some a (a has to be independent of x!)

Vectors and matrices 18

f(x) =

1

3

(x

1

+x

2

+x

3

) is linear: choose

a =

_

_

1/3

1/3

1/3

_

_

f(x) = x

1

is linear: choose

a =

_

_

1

0

0

_

_

f(x) = max{x

1

, x

2

, x

3

} is not linear: apply the denition to

x =

_

_

1

0

0

_

_

, y =

_

_

0

0

0

_

_

, = 1, = 1

we have f(x) = 1, f(y) = 0, f(x +y) = 0, f(x) +f(y) = 1

Vectors and matrices 19

mechanics example

unit mass, with zero position/velocity at t = 0, subject to force F(t)

F(t)

t

F(t)

0

0 1 2 3 4 5 6 7 8 9 10

x

1

x

1

x

1

x

1

x

2

x

2

x

2

x

2

x

3

x

3

x

3

x

3

x

4

x

4

x

4

x

4

x

5

x

5

x

5

x

5

x

6

x

6

x

6

x

6

x

7

x

7

x

7

x

7

x

8

x

8

x

8

x

8

x

9

x

9

x

9

x

9

x

10

x

10

x

10

x

10

F(t) = x

j

for j 1 t < j, j = 1, . . . , 10 (x is sequence of forces)

f(x) is position of mass at t = 10

g(x) is velocity of mass at t = 10

are f and g linear functions of x?

Vectors and matrices 110

t

velocity s

(t)

g(x) g(x) g(x)

0

0 1 2 3 4 5 6 7 8 9 10

t

position s(t)

f(x) f(x) f(x)

0

0 1 2 3 4 5 6 7 8 9 10

let s(t) be the position at time t; from Newtons law: s

(t) = F(t)

g(x) = s

(10) =

_

10

0

F(t) dt =

10

i=1

x

i

f(x) = s(10) =

_

10

0

s

(t) dt =

10

i=1

(

1

2

+ 10 i) x

i

i.e., both functions are linear

Vectors and matrices 111

(Euclidean) norm

the Euclidean norm of an n-vector x is

x =

_

x

2

1

+x

2

2

+ +x

2

n

=

x

T

x

properties

x = ||x for scalar

x +y x +y (triangle inequality)

x 0 and x = 0 only if x = 0

interpretation:

x measures the magnitude or length of x

x y measures the distance between x and y

Vectors and matrices 112

Angle between vectors

Cauchy-Schwarz inequality

xy x

T

y xy

angle between nonzero vectors x, y

cos =

x

T

y

xy

(with 0 )

x

y

= arccos(

x

T

y

xy

)

= arccos(

6

3

8

)

= /4 radians

Vectors and matrices 113

angle and inner product

orthogonal vectors

= /2, x

T

y = 0

aligned vectors

= 0, x

T

y = xy

anti-aligned (opposed) vectors

= , x

T

y = xy

acute angle

< /2, x

T

y > 0

obtuse angle

> /2, x

T

y < 0

Vectors and matrices 114

exercise: use the Cauchy-Schwarz inequality to prove that

nx

n

i=1

x

i

nx

for all n-vectors x; when does equality hold?

Vectors and matrices 115

Matrices

mn-matrix A:

A =

_

_

A

11

A

12

A

1n

A

21

A

22

A

2n

.

.

.

.

.

.

.

.

.

.

.

.

A

m1

A

m2

A

mn

_

_

A

ij

are the elements or coecients

set of mn-matrices is denoted R

mn

m, n are the dimensions

special matrices (dimensions follow from context):

A = 0 (zero matrix): A

ij

= 0 for i = 1, . . . , m, j = 1, . . . , n

A = I (identity matrix): m = n, A

ii

= 1, A

ij

= 0 for i = j

Vectors and matrices 116

Matrix transpose

A

T

=

_

_

A

11

A

21

A

m1

A

12

A

22

A

m2

.

.

.

.

.

.

.

.

.

.

.

.

A

1n

A

2n

A

mn

_

_

A

T

is n m if A is mn

(A

T

)

T

= A

a square matrix A is symmetric if A = A

T

, i.e., A

ij

= A

ji

Vectors and matrices 117

Scalar multiplication and addition

scalar multiplication of an mn-matrix A with a scalar

A =

_

_

A

11

A

12

A

1n

A

21

A

22

A

2n

.

.

.

.

.

.

.

.

.

.

.

.

A

m1

A

m2

A

mn

_

_

addition of two mn-matrices A and B

A+B =

_

_

A

11

+B

11

A

12

+B

12

A

1n

+B

1n

A

21

+B

21

A

22

+B

22

A

2n

+B

2n

.

.

.

.

.

.

.

.

.

.

.

.

A

m1

+B

m1

A

m2

+B

m2

A

mn

+B

mn

_

_

Vectors and matrices 118

Matrix-vector product

product of mn-matrix A with n-vector x

Ax =

_

_

A

11

x

1

+A

12

x

2

+ +A

1n

x

n

A

21

x

1

+A

22

x

2

+ +A

2n

x

n

.

.

.

A

m1

x

1

+A

m2

x

2

+ +A

mn

x

n

_

_

dimensions must be compatible:

# columns in A = # elements in x

Vectors and matrices 119

Linear functions

f : R

n

R

m

is linear if

f(x +y) = f(x) +f(y)

for all n-vectors x, y and all scalars ,

property: f is linear if and only if it can be expressed as

f(x) = Ax

where A is mn

Vectors and matrices 120

Operator interpretation

think of a function f : R

n

R

m

in terms of its eect on x

f x y = f(x)

systems interpretation: a system with n inputs x

i

, m outputs y

i

programming interpretation: a subroutine with n input arguments x

i

and m output arguments y

i

f is linear if we can represent its action on x as a a product f(x) = Ax

Vectors and matrices 121

some examples in R

3

reverse the order of the components of x: y

1

= x

3

, y

2

= x

2

, y

3

= x

1

a linear function:

A =

_

_

0 0 1

0 1 0

1 0 0

_

_

sort the components of x in decreasing order: not linear

scale x

1

by a given number d

1

, x

2

by d

2

, x

3

by d

3

a linear function:

A =

_

_

d

1

0 0

0 d

2

0

0 0 d

3

_

_

replace each x

i

by its absolute value |x

i

|: not linear

Vectors and matrices 122

geometrical examples of linear functions f : R

2

R

2

rotate x counterclockwise over an angle

x

f(x)

f(x) = Ax with

A =

_

cos sin

sin cos

_

project x on the vector (1, 1)

x

f(x)

e x

f(e x)

f(x) = Ax with

A =

1

2

_

1 1

1 1

_

Vectors and matrices 123

sampled signal (see p.15): dene (n 1)-vector y as

y

k

=

1

T

(u(kT) u((k 1)T)) =

1

T

(x

k+1

x

k

), k = 1, . . . , n 1

(k 1)T kT (k + 1)T

u(t) x

k

x

k+1 x

k+2

slope y

k

slope y

k+1

a linear function of x: y = Ax with

A =

1

T

_

_

1 1 0 0 0

0 1 1 0 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0 0 0 1 0

0 0 0 1 1

_

_

Vectors and matrices 124

Matrix-matrix product

product of mn-matrix A with n p-matrix B:

(AB)

ij

= A

i1

B

1j

+A

i2

B

2j

+ +A

in

B

nj

dimensions must be compatible: # of columns in A = # of rows in B

AB = BA in general! (even if the dimensions make sense)

_

1 0

0 1

_ _

0 1

1 0

_

=

_

0 1

1 0

_ _

1 0

0 1

_

there are exceptions, e.g., AI = I A for all square A

(AB)

T

= B

T

A

T

Vectors and matrices 125

operator interpretation: y = (AB)x = A(Bx)

PSfrag

x

B A

z = Bx

y = Az = ABx

explains why in general AB = BA :

f(x) =

_

1 0

0 1

_ _

0 1

1 0

_ _

x

1

x

2

_

swaps the two elements of x; then changes the sign of the rst element

f(x) =

_

0 1

1 0

_ _

1 0

0 1

_ _

x

1

x

2

_

changes the sign of rst element of x; then swaps the two elements

Vectors and matrices 126

Block matrix notation

B =

_

2 2

1 3

_

, C =

_

0 2 3

5 4 7

_

, D =

_

1 0

, E =

_

1 6 0

A =

_

B C

D E

_

=

_

_

2 2 0 2 3

1 3 5 4 7

1 0 1 6 0

_

_

A is a block matrix; B, C, D, E are the blocks of A

dimensions of the blocks must be compatible!

(B and D have the same number of columns; B and C have the same

number of rows, etc.)

Vectors and matrices 127

adding block matrices

_

A B

C D

_

+

_

A

B

C

D

_

=

_

A+

A B +

B

C +

C D +

D

_

dimensions must conform (i.e., matrix additions make sense)

multiplying block matrices

_

A B

C D

_ _

W X

Y Z

_

=

_

AW +BY AX +BZ

CW +DY CX +DZ

_

dimensions must conform (i.e., multiplications and additions make sense)

Vectors and matrices 128

The cost of matrix/vector operations

usually evaluated by counting the number of oating-point operations

one oating-point operation (op): +, , /, or

other common denition: one multiplication (or division) followed by

one addition (or subtraction)

example: inner product of two n-vectors x, y

prod = x(1)*y(1);

for i = 2:n

prod = prod + x(i)*y(i);

end

cost (with our denition): 2n 1 ops ( 2n for large n)

other vector-vector operations (on n-vectors):

addition, subtraction: n ops; scalar multiplication: n ops

Vectors and matrices 129

matrix-vector product: y = Ax for A mn

for i = 1:m

y(i) = A(i,1)*x(1);

for j = 2:n

y(i) = y(i) + A(i,j)*x(j);

end

end

cost: (2n 1)m ops ( 2mn for large n)

special cases:

m = n, A diagonal: n ops

m = n, A lower triangular: n(n + 1) ops

A very sparse (lots of zero coecients): #ops 2mn

Vectors and matrices 130

matrix-matrix product: C = AB if A is mn, B is n p

for i = 1:m

for j = 1:p

C(i,j) = A(i,1)*B(1,j);

for k = 2:n

C(i,j) = C(i,j) + A(i,k)*B(k,j);

end

end

end

cost: mp(2n 1) ops ( 2mnp for large n)

Vectors and matrices 131

exercise: evaluate y = ABx two ways (A and B are n n, x is a vector)

1. y = (AB)x (e.g., coded in Matlab as: C = A*B; y = C*x;)

2. y = A(Bx) (e.g., coded in Matlab as: z = B*x; y = A*z;)

both methods give the same answer, but which method is faster?

exercise: evaluate y = (I +uv

T

)x where u, v, x are n-vectors

1. A = I +uv

T

followed by y = Ax (in Matlab: y = (eye(n)+u*v)*x)

2. w = (v

T

x)u followed by y = x +w (in Matlab: y = x + (v*x)*u)

Vectors and matrices 132

- Python Week8 Lecture4 HandoutUploaded byreliableplacement
- Linear Algebra 01Uploaded byfurqan
- Completely formated DS ProgramsUploaded byVinay Ahlawat
- DSP LAB 2Uploaded bySabit Hussain
- Matlab Tutorial PartI 2005Uploaded bygdenunzio
- Chapter2bUploaded bymcruz_888218
- 01 Basic RUploaded byArthit Thetkham
- 3. MATLAB VariablesUploaded byVince Rabin Candidato
- MATH Final Exam Review SheetUploaded byEbrahim Shahid
- Special Fuzzy Matrices for Social Scientists, by W.B.Vasantha Kandasamy, F.Smarandache, K.IlanthenrallUploaded bymarinescu
- Vectors and MatricesUploaded bydorathi
- 43605-0132369753_8Uploaded byputrace
- Wap to Do Transpose of a MatrixUploaded byAnkit Prasad
- (Graduate Studies in Mathematics 128) J. M. -Tensors_ Geometry and Applications-American Mathematical Society (2012)Uploaded byElíGomaraGil
- comp1hwUploaded byTienanh Nguyen
- Mathematics II Jun2003 or 210156Uploaded byNizam Institute of Engineering and Technology Library
- Typed Lecture for Sec 7_1Uploaded byjtownball
- Chap12 SlidesUploaded byසම්පත් චන්ද්රරත්න
- Smart Base Isolated Benchmark Building-Part I: Problem DefinitionUploaded bypezhmankhan
- Kinematic Modeling of Wheeled Modile RobotsUploaded byAhmed El-Kabbany
- Rr410101 Computer Aided Analysis DesignUploaded bySrinivasa Rao G
- 2.1 Parallel AlgorithmsUploaded byChetan Solanki
- Numerical analysisUploaded byTeeni Abeysekara
- Thermomechanical Behavior of Rotor With RubbingUploaded byDearRed Frank
- MATLAB_session_1.pptUploaded byndsram
- Identification of Dynamic Systems From Data Composed by Combination of Their Response Compoments 2002 Engineering StructuresUploaded byldd1216
- AHPUploaded byankitsurolia
- Matrix CrimesUploaded byjames-lawson
- 04305428Uploaded byGopal Jee Mishra
- Syllabus and Model Question Paper for MSc. MathmaticsUploaded byParveen Dagar

- FermatUploaded byandykapur
- MathType Training HandoutUploaded byvineetya
- Proc9_RR3Uploaded bydzikra12
- Digital Signal Processing (Qb)Uploaded byMATHANKUMAR.S
- 1-Motion.docUploaded byAnonymous BnZrzImmC9
- Lecture11 IMCUploaded bygarcogiaz
- Ahmad, Linnhoff, Smith - Design of Multipass Heat Exchangers - An Alternative Approach (ASME)Uploaded byjdgh1986
- Assignment 2 ( 2-12-2016).pptxUploaded byrinsonlee80
- Ian Shanahan - A World of Becoming (PhD Thesis) OCRUploaded byIan Shanahan
- continumn mechanics solidsUploaded byKarthi Keyan
- Set5.k2.pdfUploaded byHarry Hing
- Gate Study Material for Electrical EnginneringUploaded byumamaheshwarrao
- MATH.pdfUploaded byAbegail Bernal
- Book Chapter Lag Rang IanUploaded byryancrankin
- Homework 3Uploaded bydorei
- class newsletter 2017 1Uploaded byapi-346911762
- Mathematics-Derivatives.pdfUploaded byKartik Purwar
- Difference Between Stress and StrainUploaded byRamkrishna Choudhury
- Mathematics 710 SyllabusUploaded byazhari7ahmad
- Introduction to Probability Theory and Stochastic ProcessesUploaded byjoshtromp
- m8_publication19Uploaded byDiana Alvarez-Marin
- Makesens Manual 2002Uploaded byRoberto Carlos Villacrés
- Rgpv Course File Cs-801 Soft ComputingUploaded byJoin2Fly
- Chap 5 1 NN ClassificationUploaded byayman
- ps1_soln_fall09Uploaded bytanmay2495
- Math 8 Chapter 1Uploaded byRubal Chib
- Assignment 1Uploaded byFarrina Aziz
- Field TheoryUploaded byashgame1473
- Revisionguide FinalUploaded byEracleous62
- Program Min ChallengesUploaded bylz1233