You are on page 1of 27
Inverse and Determinant October- 10-12 1:09 PM Given L and shit. V, can we solve
Inverse and
Determinant
October-
10-12
1:09 PM
Given L and
shit.
V,
can we solve the inverse problem?
Is there
always a solution to this problem?
Yes, no
Given
L
and V,
find
u where v = L(u)
Let
M
be the matrix associated
with L, so Mu =
v
Can we
find
N
such
that
NM=I?
So we are looking for N (inverse of M) that satisfies NM
= I
Mu= v
NMu = Nv
Iu
= Nv
U
=
Nv
ae + fc
= 1
eb + fd = 0
Example:
ag + hc =
gb + hd =
0
1
2x2
matrix
Therefore, after manipulations:
M =
(given)
N =
(unknown)
Multiply NM=
N
=
Therefore,
Therefore,
we do have a
solution for N if ad-bc
As
such,
some matrices do
not have an inverse.
example:
For projection:
not unique
review projections
BUT the
determinant
of
this matrix = 0
projection
on
x-axis
Therefore,
there is
no solution to this.
if
given a projection, can we identify
a vector uniquely that would
projection?
No.
have that
Determinant

Anamjit Sivia Page 1

For the matrix M = the quantity ad-bc is called the determinant of Matrix M.
For the matrix M =
the quantity
ad-bc is
called the
determinant
of
Matrix M.
Denoted as:
(not absolute
value)

Anamjit Sivia Page 2

Unit 11: System of Linear Equations

October-10-12

1:45 PM

Arise frequently when modelling a wide range of engineering systems e.g. electric circuits

Modelling: the art of writing a mathematical description of a system being analyzed

Model consisting one equation and 3 unknowns:

Continued in next section.

Anamjit Sivia Page 3

Unit 12: Reduced Normal Form (RNF)

October-17-12

1:06 PM

RECALL:

We can write the solution to AX=B by starting at a single solution (particular solution) and then moving along the homogeneous solution (plural).

The key is that we have been able to express each leading variable in terms of the free variables.

AX = B

If A =

M =

M =

A matrix is in RNF if:

a.

b.

c.

d.

The first non-zero entry in each row is A1.

The other entries in the columns containing these leading 1's are zero.

The leading 1's move to the right as we move down the rows.

Any zero rows are collected at the bottom.

Example:

(augmented matrix)

infinite number of solutions

So, if M is already in RNF, it is easy to simply write down the solution to AX = B.

If not, then we are going to perform operations on M to put it in reduced normal form (M'), where M' corresponds to an equivalent linear system with the same solution as M.

Elementary Operations- Gaussian Elimination

1.

2.

3.

Interchange two equations (i.e. two rows of the augmented matrix)

Multiply one equation (row) by a non-zero constant.

Add a multiple of one equation to another equation (row).

Add a multiple of one equation to another equation (row). An electrical circuit can be described

An electrical circuit can be described in terms of a mathematical model.

to another equation (row). An electrical circuit can be described in terms of a mathematical model.

Anamjit Sivia Page 4

Housekeeping: 1. Multiply row 1 by -1 2. Subtract 1 from 2. 3. Subtract 1

Housekeeping:

1.

Multiply row 1 by -1

2.

Subtract 1 from 2.

3.

Subtract 1 from 6

4.

Multiply 2 by -1

5.

Row 3 - row 2

6.

Row 5 - row 2

7.

And so on…

sivia at 18/10/2012 12:10 PM

- row 2 7. And so on… sivia at 18/10/2012 12:10 PM (augmented matrix) One unique

(augmented matrix)

One unique solution:

Summary:

There are three possible outcomes for a linear system:

1. Consistent Systems:

a.

Unique solution (occurs when every variable in RNF is a leading variable i.e. free variable)

b.

Infinite solutions (occurs when there is at least one free variable)

2. Inconsistent Systems: No solution (occurs when a contradiction is seen in the matrix)

Matrix Ranks

Anamjit Sivia Page 5

It turns out that the RNF of a matrix is a unique.

The number of leading 1's in the RNF of the matrix A is called the RANK of A.

r = rank(A)

In terms of solutions to AX = B with m equations and n variables, if the rank is r, the number of free variables is (n -r)

r = number of leading variables

(n-r) = number of free variables

n = total number of variables

Anamjit Sivia Page 6

System of Linear Equations

October-11-12

12:18 PM

Can you solve:

Example:

 

2x1 + 3x2 - 4x3 = 1

x

+ y + 3w

= 2

z

- 2w = 1

We are looking for all values

 

two equations with four unknowns Are there values of x, y, z and w that satisfy both equations?

Declare x1 to be a basic variable and x2 and x3 as free variables

x and z are leading (basic) w and y are free variables. (can be freely assigned)

2x1 = 1-3x2 + 4x3

 

x

= 2 - y - 3w

 

z

= 1 + 2w

 

x

2

-1

-3

y

0

1

0

z

=

1

y[ 0 ]

z

2

+ x2

+x3

w

0

0

1

 

Not a unique solution.

 

Anamjit Sivia Page 7

Unit 13: Homogeneous Systems

October-18-12

12:50 PM

Balance the following equation:

Not correct

Converting to RNF using Gaussian elimination:

There exists one free variable (

There are an infinite number of solution to this system.

In general, homogeneous systems (AX=0):

Unique solution

Infinite solutions (including trivial solution)

- x=0 is always a solution to homogeneous systems and is called the trivial solution.

- Therefore, only two possibilities exist for homogeneous systems:

Anamjit Sivia Page 8

Unit 14: Matrix Inverses October-24-12 1:32 PM - Limit to square systems - AX =
Unit 14:
Matrix Inverses
October-24-12
1:32 PM
-
Limit to square systems
-
AX = B
as a starting
point.
The inverse of A will be denoted as
. It will be shown
that this is
a
unique matrix.
-
Assume that
AX = B, where A is a square matrix and has a
unique solution.
-
This means the RNF of [A|B] must be of the form [I|C]
-
So we are looking for a matrix (D) such that
Formal Definition of a Matrix Inverse:
-
D
is called an
inverse of A
if we can find D such that AD = I and DA = I.
-
Can A have more than one inverse?
Let's say D and E are both
inverses of A
Therefore, DA = I = AE (= AD = EA)
D = DI = D(AE) = (DA)E = IE
= E
Hence,
there
is only one inverse
to a matrix. It is denoted
as
-
Going back to use of elementary operations to obtain a matrix in RNF
Elementary matrices: We
can associate matrices with the three
elementary operations.
 Interchange two rows:
Referred to as a permutation matrix
 Multiply one row by a non
zero scalar
Referred to as a diagonal matrix.
 Add a multiple of one row
to another row
Referred to as a shear matrix.
Therefore, we can conclude that there
such that:
is a sequence of elementary matrices

Anamjit Sivia Page 9

Algorithm to Find A: Note: this is an augmented matrix where A is dimension. a
Algorithm to
Find A:
Note: this is an augmented matrix where A is
dimension.
a matrix and I is the identity matrix in
the same
For example,
if A is
4x4, I is also
4x4
-
Perform Gaussian Elimination on M to
bring it to RNF.
Changes to:
Therefore:
Every elementary matrix
is itself invertible
and
is also an elementary matrix
corresponding to the inverse of
the row operation
that produced
.
REVIEW:
Every elementary matrix E is invertible and
is also an
elementary matrix
corresponding to
the inverse of the row operation that produced E.
sivia at 31/10/2012
1:24 PM
Two Important properties:
-
Property 1: If A and B are both invertible matrices
the inverse of
and of the same size, so is AB invertible and
We will make use of the formal definition of an inverse to
prove
this property.
Proof:
Take
as a
candidate inverse for
Therefore,
is the inverse of
-
Property 2: If A is invertible, so is
and
Proof: Take A as a candidate inverse of
(from the inversion of A, then A is the inverse of
-
Using property 2:
Using property 1:

Anamjit Sivia Page 10

Therefore, every invertible matrix can be expressed as a product of elementary matrices.

This is a significant result,

don't know why.

Anamjit Sivia Page 11

Unit 16: Curve Fitting

November-07-12

1:51 PM

Curve Fitting: Take a set of points and fit a function to those points.

Take a set of points and fit a function to those points. Interpolation: Takes a curve

Interpolation: Takes a curve fit and constructs new, related data points on the curve

Extrapolation: Take a curve, extend the graph to generate new data points i.e. new data point outside the original data interval

Anamjit Sivia Page 12

Unit 17: The Least Squares Problem

November-08-12

12:34 PM

We are dealing with the class of AX = B problems where there is no solution. We begin by defining an error vector.

Sizes: A m x n, X n x 1, B m x 1 There is no solution to this problem. Let's extend the notion of magnitude of an

(where

or

vector to

If we choose X to minimize ||E||, this is the same X that minimizes the

Another way to write matrix multiplication

This gives us a geometric way of thinking about the consistency of AX = B, namely AX=B is consistent if and only if B can be expressed as a linear combination of the column vectors of A. (or B is in the column space of A)

Let's define

as the column space of A. Therefore, as

varies over

will vary over the entire

column space

The least squares problem arisen when vector B does not lie in the column space

N.

Solving this problem amounts to finding

of this as solving for:

such that

is the closest vector in W to B. We can think

It then follows that vector of A.

is orthogonal to

Therefore,

is orthogonal to every column

These equations are referred to as the normal equations.

Define

This is called the Least Squares Solution (Assuming that

exists)

Numerical Example:

We have (independent variables) and Y 1 0 1 1 0 1 0 2
We have
(independent variables) and
Y
1
0 1
1
0
1 0
2

is a dependent variable

Anamjit Sivia Page 13

1 -1 -1 0 2 0 0 1 Find a solution to AX = B
1
-1
-1
0
2
0
0
1
Find a solution to AX = B where
Augmented Matrix:

Therefore, no solution. But, we will not give up. We are EngScis. Why don't we try a least squares solution?

Normal System:

Now:

Therefore,

i.e. Least Squares Solution

Find AX that is closest to B.

Find the distance from AX to B.

AX closest to B:

B =

Distance from AX to B

AX - B =

Anamjit Sivia Page 14

Unit 18: Numerical Integration

October-31-12

1:37 PM

This is the application section of ESC103. fucking awesome.

The Fundamental Theorem of Calculus:

This theorem sates that:

Where f(x) is a continuous function and F is a function such that

This cannot be used when f(x) is not defined or cannot be found. We will use numerical integration in such cases. We will approximate:

By partitioning the interval [a,b] into n evenly spaced subintervals. 1 1 1 … 1
By partitioning the interval [a,b] into n evenly spaced subintervals.
1
1 1
… 1
1
Step size

3 RECTANGULAR APPROXIMATIONS:

1 1 1 … 1 1 Step size 3 RECTANGULAR APPROXIMATIONS: Left hand limit, Right hand

Left hand limit, Right hand limit, average

TRAPEZOIDAL APPROXIMATION:

Anamjit Sivia Page 16

Unit 19: Simpson's Rule

November-01-12

12:51 PM

Another approach rather than using simple rectangles to approximate area:

Consider fitting a polynomial P(x) through the discrete values such that P(x) is approximately f(x)

Where

We can use the 2-point Newton Cotes rule to fit a polynomial of degree 1 (line) but this is just a trapezoid.

Now, lets use the 3- point Newton Cotes rule to fit a polynomial of degree 2 to each triplet of points:

Find [a,b,c] in P(x) =

Such that:

Because 'area' is unaffected by horizontal position, we will use instead solve the simpler problem

This is a linear system of equations. AX = B

Therefore, and so on…

B =

Anamjit Sivia Page 17

Therefore, the estimate of the total area:

Where n is an even number:

As such:

This is known as Simpson's Rule

Unit 20: Errors in Numerical Integration

November-07-12

1:07 PM

Errors pervade all areas of Science. Therefore, it is important to understand, quantify and control them.

Derivation of Error Bounds

Recall, that if we use Simpson's rule to get a quadratic for three collinear points parallel to the x-axis, b and c would be zero. Similarly, if it is any straight line, we would have c=0.

We use the Taylor Series expansion to derive these.

Trapezoid Rule:

The slope of the function does not affect the function much; rather, it is the second derivative that gives us an estimate of the error we can expect. (e.g. slope may change faster and yield a greater error in a given function)

Example:

Given:

Evaluate:

Since 1

then

Therefore, K = 2.

Take n =5. Therefore,

This is an estimate on the upper bound of the error. In order to adjust this, we can increase n.

Therefore,

will get smaller.

Actual Error: Calculate

Anamjit Sivia Page 19

Simpson's Rule: The order of the integration method is the first integer m such that

Simpson's Rule:

The order of the integration method is the first integer m such that the method does not compute

exactly. For example, in the Trapezoid Rule: Second Order Simpson's Rule: Fourth Order

Curve Fitting

Sometimes we need to fit data where the data points do not follow a linear/ quadratic/ perfect solution

Use gaussian elimination to obtain a =2, b = -3, c = 1

What if we had tried to fit a straight line rather than a quadratic?

We obtain no solution. Through gaussian elimination.

Transpose If A is an m x n matrix, the transpose ( columns of A written in the same order.

is the n x m matrix, where the rows of a transpose are the

Example:

and

Properties:

1.

2.

3.

4.

5.

If A is invertible, so is its transpose and

Proof of property 5:

exists by assumption.

Similarly, for the other case for non-commutative multiplication.

Anamjit Sivia Page 20

Unit 21: Numerically Solving Differential Equations

November-15-12

12:24 PM

We haven't yet done any fucking differential equations but we still do this shit.

Circuit Example:

(Ohm's Law)

A change in resistance represents an instant change in voltage. This is a static system.

Spring Example:

(Newton's Law)

A mass is attached to a spring, on a frictionless surface. The spring does not instantaneously change

its position when it is stretched or compressed; it oscillates

In terms of derivatives:

This is a dynamic system.

We are trying to solve for

Initial Value Problems

Example:

A system is described by

Propose that

, a function.

Find where A is a constant.

.

(t) only becomes unique with an initial value.

Say,

Therefore,

What if

?

In this case, numerical methods may have to be used as it is not intuitive to guess a function that

fulfills the conditions.

intuitive to guess a function that fulfills the conditions. Euler's Method We are going to make

Euler's Method

We are going to make use of the fact the derivative is always known.

Start at Use of slope at this point

. Lets take a step,

Algorithm (Euler's Method)

Where

gives appropriate values to

S is our slope estimate(L- left and R- right). Consider

.

Anamjit Sivia Page 21

Unit 22: Euler's Method

November-21-12

1:07 PM

The Euler's Method is a numerical approach to solving first order IVP's that makes use of derivative information.

Given:

Start at

Algorithm:

and move forward in time by

----> slope

using the 'slope' to estimate the change in y.

Where

are approximate values of

Sources of Error:

1. The formula for going from

2. The information going into the formula is not exact.

to ,

How may we improve on the formula?

is not exact.

Consider

: slope estimate at

: slope estimate at

Q: What do we use for

A: Replace

by the estimate obtained using Euler's Method

Summary of Algorithm:

Anamjit Sivia Page 22

Unit 23: Higher Order Systems

November-21-12

1:28 PM

Introducing vector notation:

Therefore, F transforms

to

Example:

Given that

Solve for

.

(a second order system)

- is a solution to this problem since

second order system) - is a solution to this problem since We will solve for both

We will solve for both y and y'.

to this problem since We will solve for both y and y'. The system equation enters

The system equation enters here

Euler's Method:

OR

and

These types of equations are called difference equations. These equations allow us to solve for y at different times.

For example, if we choose

Improved Euler's Method:

Choose

Anamjit Sivia Page 23

Recap:

Numerical solutions to differential equations (Initial Value Problems)

- Euler's Method, improved Euler's method

- In a class of predictor-corrector methods (Runge-Kutta)

A general approach to representing higher order systems. For example, consider two springs attached together.

For example, consider two springs attached together. Let Equations of motion describing this system: be the

Let

Equations of motion describing this system:

be the compression of the spring with

and similarly

Let's introduce two new variables:

This representation is referred to as the STATE-SPACE DESCRIPTION.

Unit 24: Boundary Value Problems (BVP)

November-22-12

12:28 PM

Suppose we want to solve an equation of the following form:

on the interval [0,1], with the boundary conditions

This is distinct from the previous chapter as we are given an endpoint, and two points on y but nothing about the derivative of y.

Looking at the beam example: x = 0 X = L The function Represents the
Looking at the beam example:
x = 0
X = L
The function
Represents the vertical displacement of the beam

Young's Modulus and I = the second moment of area = linear load density

If the beam is hinged at 0 and free at x = L, the Boundary Conditions are:

As with numerical integration, we are going to partition the interval into 1 1 1
As with numerical integration, we are going to partition the interval into
1
1
1
1
1
A
=

[a,b]

We will simultaneously estimate the solution at the grid points where

evenly spaced sub-intervals.

… are approximate values of

We will approximate the derivatives in the differential equation using finite differences keeping in mind that:

Different differences may be used:

Forward difference

denotes the 'forward difference'

Backward difference

Central Difference:

Second derivatives can be difined:

Anamjit Sivia Page 25

for

and

for

This is now an algebraic equation in terms of the unknown values of y. This can be further simplified:

This equation can be easily solved. LOL

Next, approximate the derivatives using finite differences. For this example:

Next, substitute finite differences into the differential equation:

Let's choose n = 5

We want to now simultaneously estimate

Substitute

at the grid points using the given boundary conditions.

Now, let's write this equation at each interior grid point.

Keep in mind boundary conditions.

Anamjit Sivia Page 26

Use Gaussian elimination to solve this system.

For this problem, there is an exact solution.

Actual Values:

We can change the grid point (n) to increase the accuracy which increases the accuracy of finite differences. These results are approximate to two decimal places. However, doing so would involve solving a larger matrix.

Anamjit Sivia Page 27