Mathematical treatise on linear Algebra

© All Rights Reserved

10 views

Mathematical treatise on linear Algebra

© All Rights Reserved

- 3 Dimension&Basis
- IIT kanpur Machine Learning end sem paper
- Exercises Algebra
- 3.2 Vector Space Properties
- 224 Notes
- 00rad-multicriteria optimisation
- Hw1 Sol Amcs202
- Misha Poisson Reconstruction
- Holonomic Control of a Robot With an Omni-Directional Drive
- Lecture 3
- 110fall2012.midterm1
- (1.1.0)Marriage
- PS3-AnsKey (1)
- Burrough McDonnell Two
- Fuzzy Predictive Control of Fractional-Order
- ECL4
- Constraint Based Physics
- 90NH
- Kernel Via Semi Definite.pdf
- (International Series of Numerical Mathematics _ Internationale Schriftenreihe zur Numerischen Mathematik _ Série Internationale D’Analyse Numérique 14) John Todd (auth.)-Basic Numerical Mathematics_ .pdf

You are on page 1of 7

(1) Vocabulary

(a) Linear combination

A linear combination of two vectors is the sum of two vectors multiplied by

corresponding scalar values. Each term is multiplied by a constant that applies

to the entire vector.

(b) Linear dependence

A set of vectors is said to be linearly dependent if one of the vectors can be

dened as a linear combination of the others.

(c) Linear independence

A set of vectors is said to be linearly independent if none of the vectors are

linear combinations of each other. Additionally, in solving the system Ax = 0,

a linearly independent set will only have x=0 as a possible solution, meaning

there is no solution but the trivial solution.

(d) Subspace (of a `)

A subspace is dened as a euclidean space which is contained wholly in another

space. The characteristics of a subspace include: Having the zero-element,

being closed under addition, and being closed under scalar multiplication. We

say a space U is closed under addition if the vector a + b is U given that

aU and b U . We say that a space U is closed under scalar multiplication

if, given that a is a vector in U , a is also a vector in U for any R

(e) Span (of a set of vectors)

The "span" of a set of vectors is dened as the subspace occupied by all of the

linear combinations of vectors within the specied set.

(f ) Basis (of a subspace)

A "basis" of a subspace is a set of linearly independent vectors which have the

same span of the original subspace.

(g) Inner Product, Dot Product We dene the inner product of vectors x,y as the

sum of the products of corresponding elements. Thus < x, y >= x1 y1 + x2 y2 +

...xn yn . This denition is synonymous with our calculation of the dot product,

notated as x y.

(h) Norms

Norms are functions which take vectors as input and return non-negative real

numbers as output. Norms can be considered a means to measure the length

or magnitude of a vector. Norms satisfy a few conditions:

If x is R then kxk 0.

If x = 0 then kxk = 0.

If kxk = 0 then x = 0.

If R then kxk = |kxk.

1

2

Lastly, if x and y are both R, then kx + yk kxk + kyk, which satises the

triangle inequality.

(i) Linear functions

For a function f :U V

If u1 and u2 are U , then f (u1 + u2 ) = f (u1 ) + f (u2 ).

If u U and R, then f (u) = f (u).

Additionally, note that f (0) = 0, by taking = 0.

(j) dimension

The number of basis elements needed to dene a space is called the dimension

of the space.

(k) orthogonal (vectors)

Two vectors are orthogonal if and only if the lines drawn from the origin to the

points with coordinates dened by x and y are perpendicular to each other.

Additionally, the dot product of orthogonal vectors is 0.

(l) orthonormal (vectors)

Orthonormal vectors are both orthogonal and normalized (usually to 1 in the

case of QR decomposition).

(m) rank (of a matrix)

The dimension of the range of A is called the rank of A. This can be written

as rank(A ) = dim(range(A)).

(n) range (of a matrix)

The range of A is the span of the columns of A. This is also called the column

space of A.

(o) nullspace (of a matrix)

The nullspace of a matrix is dened as the set of vectors x which, when multi-

plied by the matrix A equal 0. A

Additionally, we can dene the nullspace of

as the set of vectors x AT .

that are orthogonal to the range of

T

We make the claim that Range(A is orthogonal to Null(A). This means that

for:

y Range(AT ) means y = AT x.

z Null(A) means Az = 0.

T T T T

Thus y z = (A x) z = x Az = 0.

T T

This suggests that Range(A A) = Range(A )

(p) inverse (of a matrix)

We dene the inverse of a matrix A as A1 , which when multiplied by A

becomes the identity matrix I.

Note that this means that both AA1 = I and A1 A = I.

(q) elementary vectors e1 , . . . , e n in Rn

The standard basis consists of the "unit vectors", e1 , e2 , ... en , which form a

basis for the space Rn . Applying a function to the standard basis vectors will

produce a matrix which models the function.

(r) Echelon form and reduced Echelon form

We dene a matrix to be in row echelon form when it satises the following

3

conditions:

The rst non-zero element in each row, called a "pivot", is a 1.

Each pivot is in a column right of the previous row.

satises the following condition:

Every leading coecient is 1 and is the only non-zero entry in its column.

(a) Solving linear systems of equations

We solve linear systems of equations by creating an augmented matrix by which

the right most column represents the sum of the remaining left-hand columns.

Left-hand columns represent variables. Upon setting the augmented matrix

to reduced row echelon form, we can set the solution of the system of linear

equations equal relative to the coecients modeled by the pivots.

(b) Equivalence of linear functions from Rn to Rm and matrices

A function from f: Rn Rm can be modeled by a matrix of dimension Rmn .

Recall that we consider linear functions to be functions which satisfy two con-

ditions:

f (x + y) = f (x) + f (y). Compatibility with addition.

f (x) = f (x). Compatibility with scalar multiplication.

(c) Matrix multiplication

If A Rmr andB Rrn , then we can dene the product C = AB, which

will be a matrix in R

mn . Note that this product is only dened if the number

by multiplying rows by columns i.e.

a11 a12

a21 a22

A= , B = b11 b12 b13

a31 a32 b21 b22 b23

a41 a42

thus

c11 c12 c13

c21 c22 c23

C=

c31

c32 c33

c41 c42 c43

Where

c12 = a11 b12 + a12 b22

c43 = a41 b13 + a42 b23

4

We can nd a basis for a subspace by identifying the set of linearly independent

elements which span the span of the subspace. One method of doing so is

modeling the subspace with a matrix and reducing the matrix to reduced row

echelon form. The pivot columns of the matrix in reduced row echelon form

correspond to the basis vectors in the original matrix i.e. If a reduced matrix

has pivots in columns 1, 4, and 7, then the span of the original matrix can be

modeled by its rst, fourth, and seventh columns.

(e) Finding a basis for the range/nullspace of a matrix

To nd the nullspace of a matrix, solve the system for the zero-element.

Recall that the dimension of the nullspace and the dimension of the rank of

a matrix must add up to the dimension of the euclidean space in which the

matrix resides i.e. For a matrix A Rn , rank(A)+ dim(A) = n.

The range of a matrix can be found by identifying the span of its columns.

(f ) Row operations as (invertible) matrix multiplication

For any legal row operation applied to matrices, there exists a matrix which,

when multiplied by the host matrix, models this row operation. Additionally

row operations are reversible, and thus for each matrix multiplication which

models a row operation there exists a matrix which is precisely the inverse, and

reverses the operation.

(g) Finding the inverse of a matrix

To nd the inverse of a matrix, form an augmented matrix by which the left-

hand side is the original matrix and the right-hand side is the Identity matrix.

As Gauss-Jordan elimination is applied to the original matrix, the inverted

row operations which occur on the Identity matrix. Thus, when the left-hand

matrix is fully reduced to the Identity matrix, the right-hand side is precisely

the inverse of the original matrix.

(h) Linear regression

Practice problems

(3) Take any norm k k. Are the following norms?

(a) f (x) = 2kxk Yes. This is the same as taking the norm of 2 x.

(b) f (x) = k xk Yes, the norm of x is equal to the norm of x by the denition

of a norm. Therefore, a function which takes the norm of x is the same as

nding the norm of x and thus is clearly a norm.

(c) f (x) = kxk2 No, Clearly this does not work with scalar multiplication, as

f (2x) 6= 2f (x).

(d) f (x) = kx 2k No, The zero-norm is a non-zero element.

(a) Lines through the origin (in Rn ) Yes. The sum of two vectors which pass

through the origin is another vector, which still passes through the origin.

5

origin will produce another vector which still runs through the origin.

R3 .

(b) All vectors perpendicular to 1 2 3 in

Yes. To be perpendicular or orthogonal, the cross product of two vectors

R3

must be zero. Clearly the set of vectors perpendicular to 1 2 3 in is a

subspace, as the sum of any two vectors which satisfy these conditions, u and

R3

v will still have a dot product of 0 with 1 2 3 in due to the properties

of matrix addition. Additionally, scalar multiplication of dot product which is

equal to zero results in a dot product which is still equal to zero.

(c) All vectors y so that hy, e1 i = 2. No. Scaling some vector y by a real number

6= 1 will change the inner product, meaning that this set does not satisfy the

condition of compatibility with scalar multiplication.

(d) Span of e1 , e2 in Rn .

Spans are dened as subspaces.

(e) Set of solutions to x2 = 0 in R. The only solution to x2 = 0 is x = 0. Adding

2

this solution to itself i.e. 2x = 0 still holds true for when x = 0. Scaling x = 0

by any real value will produce the equation x = 0

f : R3 R2 with

(5) Suppose I give you a linear function

1 1 0

1 3 1

f 0 = , f 1 = , f 0 = .

1 2 1

1 0 2

Find an explicit matrix A so that f (x) = Ax. Add The rst column to the second

1

0 0

4

column. v2 = 1 = Divide the third column by 2 0 = 2 Subtract

3 1

1 1

2

3 7

1 0

the third column from the rst and second 0 = 2. 1 = 2

1 3

0 0

2 2

1 0 0

0 1 0 = 1.5 3.5 0.5

0.5 1.5 0.5

0 0 1

(a) Polynomials of degree exactly 2. No, this does not contain the zero-element

(b) Polynomials of degree less than or equal to 2. Yes.

(c) Even functions, f (x) = f (x). Yes

(d) Odd functions, f (x) = f (x). Yes

6

Linear functions of 0 always

equal 0, because f (x) = f (x) holds true for

= 0 R Therefore the sum of any two functions which set x to 0 will produce

a function which still sets x to 0, and scaling a zero-function will still produce

0 as output.

(f )

4 00

Polynomials in p P , with p (1) = 1.

4 3

We can think of all polynomials of degree 4 to be in the form ax + bx + cx +

2

6bx + 2c. Based on the condition p(1) = 1, this means that 12a + 6b + 2c = 1.

Clearly, if we were to add two polynomials which would meet this condition

1 4 47 3 13 4 47 3

i.e

12 x and x4 6 x , the resulting polynomial 12 x 6 x no longer meets

the condition, as it's second derivative will no longer equal 1. Therefore this is

NOT a subspace.

(7) Let

rs

S = {x R3 | x = 2s , r, s R}.

r

Verify that S is a linear space; then nd a basis for S. What is the dimension of S?

2 2 2

A = 0 1 2

1 2 3

(a) What is the rank of A? Give a basis for the range R(A).

(b) What is the dimension of Null(A)? Give a basis for the null space N (A).

(c) What is the dimension of Null(AT )? Give a basis for the range R(AT ).

S = {p P 4 , p(0) = 0, p0 (1) = 0}

(10) Find four linearly independent vectors that are all orthogonal to 1 3 3 2 1

in R5 .

that solves a linear system Ax = b.

(a) Show that + v also solves Ax = b, for any v Null(A).

x

(b) Show that if x

solves Ax = b, then you can write x = x

+w for some w

Null(A).

7

(c) Now use the previous two problems to write down all solutions to the system

x1

1 0 1 0 x2

= 3

0 1 0 0 x3 2

x4

.

(12) Let

1

V1 = 2 , V2 = 1 1 1 .

3

Be careful to think of these as matrices. Compute the following.

(a) V2 V1

(b) V1 V2

(c) V2 V2T

(d) V2T V2

(a) Explain how you would use Newton's method to do this.

(b) Starting with the base point x = 1, compute the rst iterate of Newton's

method for the problem.

(c) Can you also use the secant method? What would be dierent? Which would

you use?

1

2

1

1

as 1 v1 + 2 v2 , where v1 = 1 and v2 is orthogonal to v1 .

0

1. The range of P is U and

2. Pu = u for any u in U.

1

(a) Show that for any v, the matrix

kvk2

vv T is a projector onto the span of v.

(b) Show that P2 = P.

1 1

(c) Compute the projection of 2 onto the span of 1.

1 0

- 3 Dimension&BasisUploaded byalienxx
- IIT kanpur Machine Learning end sem paperUploaded byJivnesh Sandhan
- Exercises AlgebraUploaded byTien Kim
- 3.2 Vector Space PropertiesUploaded bymuffinabc
- 224 NotesUploaded byRGM36
- 00rad-multicriteria optimisationUploaded bydraganug
- Hw1 Sol Amcs202Uploaded byFadi Awni Eleiwi
- Misha Poisson ReconstructionUploaded bymiroseh
- Holonomic Control of a Robot With an Omni-Directional DriveUploaded byGuilherme Pohl
- Lecture 3Uploaded bylvrevathi
- 110fall2012.midterm1Uploaded byHilmi Nur Ardian
- (1.1.0)MarriageUploaded byarfnw
- PS3-AnsKey (1)Uploaded byNaci John Trance
- Burrough McDonnell TwoUploaded byVioleta Zamora
- Fuzzy Predictive Control of Fractional-OrderUploaded byRuchit Pathak
- ECL4Uploaded byJOel Suárz Rz
- Constraint Based PhysicsUploaded byManuel Zanin
- 90NHUploaded bySoumyadeep Roy
- Kernel Via Semi Definite.pdfUploaded bysoumensahil
- (International Series of Numerical Mathematics _ Internationale Schriftenreihe zur Numerischen Mathematik _ Série Internationale D’Analyse Numérique 14) John Todd (auth.)-Basic Numerical Mathematics_ .pdfUploaded byaufal Riswan
- ChE SyllabusUploaded byM.A. Monim
- BSAD282 Portfolio OptimizationUploaded bybing miranda
- sec4_2Uploaded byFilozófus Önjelölt
- Midterm1 Spaced Out Questions-solutionsUploaded bymarkydee_20
- Math 306 Spring 17 Exam 3 ReviewUploaded bytvbanks
- Ya Vas Liubliu-wordsUploaded byIra Li
- IJMET_10_01_040Uploaded byIAEME PUBLICATION
- casc02-25.psUploaded bySucram Suintop Sunisoc
- 2012 Art Bssa BemUploaded byPanji Hoetomo
- Fuzzy Information and Engineering 2010, Vo - Bing-Yuan Cao.pdfUploaded byLushanio

- The Database Life CycleUploaded byPetros Gavai
- 020__BCA__upto_2014_Batch_Final_Result_Dec.2015.pdfUploaded byDhruv Rajpal
- IVCO 2008 FORUM Gender MainstreamingUploaded byforum-ids
- D.J. Littler (Eds.)-Turbines, Generators and Associated Plant. Incorporating Modern Power System Practice-Pergamon (1991)Uploaded byAnonymous s0NxfrYK
- lec 1 intro to modeling and simulationUploaded byapi-247714257
- A Comparative Study of Learning Styles oUploaded byAlert Mendoza
- MechatronicsUploaded byinhenyerongpogi
- CFSFinalReport Final Dec7Uploaded byenriquesanjuan67
- Particle Size Distribution Analysis for Ceramic Pot Water Filter ProductionUploaded bySharath Chandra
- En - Hydraulic CoursesUploaded byAhmed Moustafa
- Project ChecklistUploaded bynadimzpk
- CTO's and Product Managers BriefingUploaded byAlan Wilensky
- cvv_cvv_sUploaded bypatootom5
- Letter for Invitation for Maglaunch for College(1)Uploaded byNima Thingg
- Air Start SystemUploaded byPaolo Ferjančić
- Value Partners Q4 2014 CommentaryUploaded byCanadianValue
- Lab Task 1Uploaded byblahblahmamamama
- 4 0 kt keytermsUploaded byapi-291530288
- Load & Price Forecasting Webinar SlidesUploaded byRushikeshKatkar
- Certification of BIM Competency for Individuals and Companies in singapore.pdfUploaded byAmar Nath
- EN 15242Uploaded byDragan Veselica
- An Insight on the Working of JIT in Indian Automobile IndustriesUploaded byHimanshu Sanga
- BitFury's Questionable Iceland ConnectionUploaded byrigadd
- Uribe Notes Civil Review 2Uploaded byGrace Riguer
- Microlab.M500.pdfUploaded byblueword66
- Progression Report - Collective TimelineUploaded bybien_9
- M93 Hornet Wide Area Munition - Archived 12/2003Uploaded byVigilanteDesign
- app176.pdfUploaded byAnisur Rahman
- Cebu Marine Beach Resort vs NLRC,YAMBAOUploaded byMaCai Yambao
- Significance & Potential of Packaging in Chocolate IndustryUploaded byVishal Singh