02 Signal Processing Fundamental Mathematics

Measurement
&
Signal Processing
Chapter 2. Fundamental Mathematics
7‐Mar‐23
1 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

Summation Algebra
 In the some situations, we have one or two (or perhaps three)
lists, and we wish to refer to particular numbers in those lists.
 In this case, we represent numbers in a list with a notation of
the form
 where the symbol i is called “subscript”.
 For example
x i , i = 1, 2,, 5  x 1 , x 2 , x 3 , x 4 , x 5

Summation Algebra
 To combat the difficulties that arise when more than one list is
being discussed, it is often more convenient to use double
subscript notation.
 The first subscript refers to the row that the particular value is
in, the second subscript refers to the column.
 It is also a matrix form(three row and three column)

Summation Algebra
 Based on the previous notation, consider a single summation
notation
stop index
start index
 In the above expression, the i is the summation index, 1 is the
start value, N is the stop value.
 The start index is not necessarily to be 1.
5 5
å xi = x1 + x 2 + x 3 + x 4 + x 5 å xi = x 3 + x 4 + x 5
i =1 i =3

Summation Algebra
 First Constant Rule – Simplified
 The Second Constant Rule of Summation Algebra
 The Distributive Rule

Summation Algebra (Proof)
 First Constant Rule – Simplified
N
åa = a
+ a +
a +  +a = Na
i =1 N
 The Second Constant Rule of Summation Algebra
N æ ö÷ N
ç ÷
åax i = ax ax 3 +  + ax N = a ççx
1 + ax 2 +
 x 3 +  + x N ÷ = a å x i
1 + x 2 + ÷÷
i =1 N
çè N ø i =1
 The Distributive Rule
N
å (x i + yi ) = (x1 + y1 ) + (x 2 + y2 ) +  + (x N + yN )
i =1
= (x1 + x 2 +  + x N ) + (y1 + y2 +  + yN )
 
N N
N N
= å x i + å yi
i =1 i =1

Summation Algebra(ex.)
 The sample mean, or arithmetic average of a list of N numbers
is defined as
close all; %% close all the figure
clear; %% clear all the variable in workspace
clc; %% clear command screen
A = 5 + randn(20,1); %% add white noise
4.9209 A_sum = sum(A);
A_avg = sum(A)/length(A);
plot(A,'o')
axis([0 20 0 10])
xlabel('Sample Data')
ylabel('Values')
 For the element deviation, it is defined by

 The sum of deviations is zero, that is
N
4.9209
å[dx ]i = 0
i =1
 Proof:
 Recall that by definition
1 N N
x· = å xi  å xi = N ⋅ x ·
N i =1 i =1
 Therefore
N N N N
å[dx ]i =å[x i - x · ] = å x i - å x ·
i =1 i =1 i =1 i =1
N
= å xi - N ⋅ x ·
i =1
= N ⋅ x· - N ⋅ x· = 0

DOUBLE AND MULTIPLE SUMMATION
 For double and multiple summation, lets refer to the double
subscript notation
 How to take a summation of these elements by using a single
notation? 3 3 3
å å x ij = å (x i 1 + x i 2 + x i 3 )
i =1 j =1 i =1
= (x 11 + x 12 + x 13 ) for i = 1
+ (x 21 + x 22 + x 23 ) for i = 2
+ (x 31 + x 32 + x 33 ) for i = 3
 Take care about the location of power
n
N æ N ö
å i ¹ çççå x i ÷÷÷
x n
i =1 èi =1 ø
 For example, xi = i
5
å x i2 = 12 + 22 + 32 + 42 + 52 = 55
i =1
æ 5 ö÷2
ççå x i ÷ = (1 + 2 + 3 + 4 + 5)2 = 152 = 225
çèi =1 ÷ø
 Apparently 5 æ 5 ö2
å x i2 ¹ çççå x ÷÷÷
i =1 èi =1 ø

DC and AC signals
 Direct Current (DC) always flows in the same direction (always
positive or always negative), but it may increase and decrease.
 Batteries and regulated power supplies provide steady
DC which is ideal for electronic circuits.
 Electronic circuits normally require a steady DC supply which
is constant at one value or a smooth DC supply which has only
a small variation called ripple.
https://electronicsclub.info/acdc.htm
AC and DC signals
 Alternating Current (AC) flows one way, then the other way,
continually reversing direction.
 An AC voltage is continually changing between positive (+) and
negative (‐).
 The rate of changing direction is called the frequency of the
AC and it is measured in hertz (Hz), which is the number of
forwards‐backwards cycles per second.
https://electronicsclub.info/acdc.htm
DC and AC signals (Ex.)
DC Sinusoidal Wave DC with Ripple
Saw-tooth Saw-tooth Saw-tooth

In mathematics, RMS is defined as the square
root of the mean square (the arithmetic mean
of the squares of a set of numbers).
Root Mean Square
 The root mean square value of a quantity is the square root of
the mean value of the squared values of the quantity taken
over an interval.
 Therefore, the RMS for a continuous function over time T is
1 T 2
fRMS = lim ò ( f (t )) dt
T ¥ T 0
 The RMS value of a function over the range [T1,T2] can be
defined as 1 T2 2
fRMS = ò ( f (t )) dt
T2 -T1 T1
 For discrete case, for a set of number x = {x1 , x 2 ,, x n }, the RMS is

1 n 2 1 2
x RMS = å k
n k =1
x =
n
(x 1 + x 22 +  + x n2 )

Root Mean Square (ex.)
 Consider a perfect DC sampled signal set x
Signals without any
x = {3, 3, 3, 3, 3} non‐zero frequency component
 The corresponding RMS can be calculated as follows
 Consider x1 = x 2 =  = x n Ex.
1 2 1 2
x RMS =
n
(x 1 + x 22 +  + x n2 ) x RMS =
n
(x1 + x 22 +  + x n2 )
1 1 2
=
n
(n ⋅x 2) =
5
( 3 + 32 + 32 + 32 + 32 )
= x2 = x 1
=
5
(5 ⋅ 32 )
RMS for a constant value serial number
= 32 = 3
is equal to its signal element

 Next, consider a set of a series number
Note that RMS of real numbers is always
x = {2, -3,1, -4, 5}
equal to or larger than zero(non‐negative)
 The corresponding RMS can be calculated as follows
1 2
x RMS =
n
(x1 + x 22 +  + x n2 )
1 2
=
5
( 2 2
2 + (-3) + 12 + (-4) + 52 )
= 3.32
RMS  Energy Averaging
 RMS of a sine function : f (t ) = A sin (wt )
T Î [ 0, 2p ]
1 2p 2 2
fRMS = ò A sin (wt )dt
2p 0
A2 2 p 1
= ò (cos (wt - wt ) - cos (wt + wt ))dt
2p 0 2
product-to-sum
2
A 2p 1
= ò (cos (0) - cos (2wt ))dt
2p 0 2
A2 2 p 1
= ò (1 - cos (2wt ))dt
2p 0 2
sum-to-product
2 2p
A æç 1 1 ö÷ A
= t - sin (2 wt ) ÷÷ = = 0.707A
2p çè 2 4w ø0 2
fRMS = fPeak x 0.707

 RMS of a sine function : f t   A sin t 
1 T 2 2 T Î [ 0, ¥ ]
fRMS = ò A sin (wt )dt
T 0
T
A2 æç 1 1 ö÷
= T - sin (2wT )÷÷
T çè 2 4w ø0
A2 æç 1 ö÷ fRMS = fPeak x 0.707

= 1 - sin (2wT ) ÷÷
2 çè 4wT ø
A
 It is clear that when time approaches to infinity, fRMS 
2
 For a low frequency together with a short T, it gives the appro.
estimate uncertainty .
1/2
 As a result, at lower frequencies, longer averaging times are
required to reduce the measurement uncertainty.

 Consider the following harmonic vibration signals with noise.
 Please estimate the magnitude of the vibration, that is ‘Aest=?’
 Recall that
1 2p 2
fRMS = ò est
A sin 2
(wt )dt A
2p 0
A
= est
2
 Therefore
Aest = fRMS ⋅ 2
 fRMS
PS: the correct way is to filter out the noise before computing .
Crest Factor
 Crest factor is the peak amplitude of the waveform divided by
the RMS value of the waveform:
x peak
CF =
x RMS
 Crest factor is a measure of a waveform, such as alternating
current or sound, showing the ratio of peak values to the
effective value. In other words, crest factor indicates how
extreme the peaks are in a waveform.
 CF =1 indicates no peaks, such as direct current(that is DC
component).
 High crest factors indicate obvious peaks included in the signal.
 For example, shock waves tend to have high crest factors.
Crest Factor (ex.)
 Consider the following two signals
Acc_RMS = 0.0158
CF_Acc = 6.4091 load A20160218_145435.txt
A = A20160218_145435(:,1);
t=1:length(A);
time = t/8192;
%%... Impulse Force
Acc_RMS = sqrt(sum(A.^2)/length(A))
CF_Acc = max(abs(A))/Acc_RMS
%%... Harmonic Vibration
y = 0.2*sin(2*pi*5*time);
y_RMS = sqrt(sum(y.^2)/length(y))
CF_y = max(abs(y))/y_RMS
figure
subplot(2,1,1)
plot(time, A)
ylabel('Impact Force(g)')
subplot(2,1,2)
plot(time,y)
y_RMS = 0.1414 xlabel('Time(s)')
CF_y = 1.4142 ylabel('Harmonic Vibration(g)')

Crest Factor (ex.)
 Consider the following two signals again
Acc_RMS = 0.0158
CF_Acc = 6.4091
For a smooth harmonic vibration
The CF will be 1.414
Why??
y_RMS = 0.1414
CF_y = 1.4142
Crest Factor (ex.)
 For the following two hamming data, what’s the difference?
Acc_RMS = 0.0165
CF_Acc = 6.9660
x peak
CF =
x RMS
Acc_RMS = 0.1687
CF_Acc = 8.3782

Vector
 Summation
交換律
結合律
分配律

X = [x 1 x 2  x n ] Î  n
Vector Norm X = [x 1 x 2  x n ] = x 12 + x 22 +  + x n2
 A norm is a function that assigns a length, or size, to a vector
 For example, let X = [1 2] and Y = [3 4], we have
é1ù é 3ù
X = ê ú = 12 + 22 = 5 > 0, Y = ê ú = 32 + 42 = 25 = 5 > 0
êë2úû êë 4úû
 The norm of a scalar is equivalent to the corresponding
absolute value. vector norm is
[ ]
X = 10 = 102 = 10
non-negative
X = [x 1 x 2  x n ] Î  n
 For example, let X = [1 2] and a negative scaling factor ‘‐5’.
 Considering the scalability, we have
é1ù é -5 ù
X = -5 ⋅ = ê
ê ú ú = (-5)2 + (-10)2 = 125 = 5 5 > 0
êë2úû êë-10úû
é1ù é 1ù
X = -5 ⋅ ê ú = -5 ⋅ ê ú = 5 ⋅ 12 + 22 = 5 5 > 0  a⋅X = a ⋅ X
êë2ûú ëê2ûú
X = [x 1 x 2  x n ] Î  n
 For example, let X = [1 2] and Y = [‐3 ‐4].
 Considering the triangle inequality, we have
é1ù é-3ù é-2ù
X+Y = ê ú + ê ú = ê ú = (-2)2 + (-2)2 = 8 = 2 2 > 0
êë2úû êë-4úû êë-2úû
é 1ù é-3ù
X + Y = ê ú + ê ú = 5 +5  X+Y £ X + Y
êë2ûú êë-4ûú

Vector Inner Product
 For a given two vectors x and y, the inner product of these two
vectors is defined as
n éx ù é y1 ù
x, y = å x i yi ê 1ú ê ú
i =1 êx2 ú ê y2 ú
where x = ê ú Î n´1 , y = ê ú Î n´1
= x 1y1 + x 2y2 +  + x nyn êú êú
ê ú ê ú
= xT y Î 1´1 êx ú ê ú
a scalar! ë nû ëyn û

Vector Inner Product
 For example

Vector Normalization
 Vector normalization
original vector
x
xN =
x
magnitude of the vector
 The size of the normalized vector is always unit.
 Therefore, it is also referred to as unit vector.
 The normalized/unit vector only shows the direction.

Vector Normalization(ex.)
 Consider a vector
é 3ù é 3ù
ê ú ê ú
é 3ù êë4úû êë4úû é 3 / 5ù æ 3 ö÷2 æ 4 ö÷2
x = ú Î  2´1
ê xN = = =ê ú xN = çç ÷÷ + çç ÷÷ = 1
êë 4úû é 3ù 5 ë4 / 5úû
ê è5ø è5ø
ê ú
êë4úû
 Consider another vector
é 1ù é 1ù
ê ú ê ú
ê2 ú ê2 ú
é1ù ê ú ê ú é1 / 14 ù
ê ú ê 3ú ê 3ú ê ú æ 1 ö÷2 æ 2 ö÷2 æ 3 ö÷2
xN = ë û ë û
= = 2 / 14 ú
ê xN = çç ÷÷ + çç ÷÷ + çç ÷÷ = 1
x = ê2ú Î  3´1 é1ù 14 ê ú è14 ø è14 ø è14 ø
ê ú ê ú ê 3 / 14ú
ê 3ú ê2 ú ë û
ë û
ê ú
ê 3ú
ë û

Trigonometric
Identities
Vector Projection
 Geometric interpretation of vector operations
 What does inner product means?
T T
x = [x 0 , x 1 ] , y = [y 0 , y1 ]
Law of cosines

Vector Projection
 The vector projection of a vector 𝑨 onto a nonzero vector B is
the (orthogonal) projection of A onto a straight line parallel
to B.
 Law of cosines
A
^    
Aproj 
   A, B  A⋅ B  B
Aproj = A cos q = A   = A   = A ⋅ 
q  A B A B B
 B
Aproj scalar
         
  B  B  A, B B  A⋅ B B A⋅ B 
Aproj = Aproj ⋅  = A cos q ⋅  = A   ⋅  = A   ⋅  = 2B
B  B A B B A B B B
 scalar 
unit vector unit vector
 
  ^ ^    A⋅ B  vector
 A = Aproj + Aproj \ Aproj = A - Aproj = A -  2 B
B

Vector Projection
 The vector projection of a vector 𝑨 onto a nonzero vector B is
the (orthogonal) projection of A onto a straight line parallel
to B.
 Projection Rule
A   
^ ìï   B  
Aproj ïï Aproj = A ⋅  , Aproj = A ⋅ B B
ïï 2
B B
q ï
 í  
 B ïï  ^    A⋅ B 
Aproj ïïAproj = A - Aproj = A -  2 B
ïï B
î
 Briefly, the projection of the vector A onto B can be formally
expressed as
 
  A,B  ^    
Aproj  ProjB (A) =   B
 Aproj  A - Aproj = A - ProjB (A)

B,B

A ^
Aproj
q 
Vector Projection 
Aproj
B
 Following is the proof that ⇀
𝐴 ,⇀
𝐴 0
 Proof:  
  A, B  ^  
 Since Aproj = ProjB (A) =   B Aproj = A - ProjB (A)
B,B
 One has
 ^  ^   
B ( )( B ( ))
Aproj , Aproj = ATproj Aproj = ProjT
A A - Proj  A
         
æ
A, B  ç  A, B  ÷ A, B  ö A, B A, B T 
=   BT ççA -   B ÷÷ =   BT A -     B B
B,B çè B , B ÷÷ø B , B B,B B,B
       
A, B A, B A, B A, B  
=   -     B,B
B,B B,B B,B
       
A, B A, B A, B A, B
=   -   =0
B,B B,B
Vector Norms Visualization
 So far, we already use the “2‐norm” to measure the size of a
vector. Actually, there are some different measurement metrics.
 Following we consider a comparison of different norms.
T
 By definition, for a given vector x = [x1 x 2  xm ] Î m
 1‐norm m
x 1 = å x i = x1 + x 2 +  + x m
i =1
 2‐norm 1/ 2
æ m 1/ 2
ö 1/ 2
 x 2 = x, x
x 2 = ççå x i2 ÷÷÷ = (x12 + x 22 +  + x m2 )
çèi =1 ø 2
 x 2 = x, x = xT x
 ‐norm
x ¥ = max { x 1 , x 2 , x 3 ,, x m }

T
 Consider a visualization of the norms for x = [x 1 x 2 ] Î  2
x2
 Let the x1 & x2 range between (-1,1) (1,1)
x1 Î [-1 +1] x1
x 2 Î [-1 +1] (-1,-1) (1,-1)
 To visualization the pattern of the norms of the vector x, we
are going to find “the changing of x1 & x2”, such that the
following conditions (or constraints) are satisfied.
m
x 1 = å xi = 1 x 1 = x1 + x 2 = 1
i =1
1/ 2
m =2 æ 2 2 ö÷1/2
æ m 2 ö÷ x 2 = ççå x i ÷÷ = x12 + x 22 = 1
x 2 = ççå x i ÷÷ =1 çèi =1 ø
èçi =1 ø 2D case
x ¥ = max { x 1 , x 2 ,, x m } = 1 x ¥ = max { x 1 , x 2 } = 1

x1 Î [-1 +1]
x 2 Î [-1 +1]
 For the “1‐norm”, there are four cases satisfy the constraint.
x 1 = x1 + x 2 = 1
ìï +0.0 + +1.0 = 1 ì
ï -0.0 + +1.0 = 1 ì
ï -0.0 + -1.0 = 1 ìï +0.0 + -1.0 = 1
ïï ï ï ïï
ïï +0.1 + +0.9 = 1 ï
ï ï
ï ïï +0.1 + -0.9 = 1
ï -0.1 + +0.9 = 1 ï -0.1 + -0.9 = 1 ïï
ïï ï
ï ï
ï
ïï +0.2 + +0.8 = 1 ï -0.2 + +0.8 = 1 ï -0.2 + -0.8 = 1 ïï +0.2 + -0.8 = 1
L1 í ï
L2 í ï
L3 í L4 í
ïï  ï  ï  ïï 
ïï ï
ï ï
ï ï
ï ï ïï +0.9 + -0.1 = 1
ïï +0.9 + +0.1 = 1 ï
ï - 0.9 + + 0.1 = 1 ï
ï - 0.9 + - 0.1 = 1 ïï
ïï ï
ï ï
ï ïï +1.0 + -0.0 = 1
ïî +1.0 + +0.0 = 1 ï
î - 1.0 + + 0.0 = 1 ï
î - 1.0 + - 0.0 = 1 î
x2 x2 x2 x2
+1 +1
L1 L2
-1 +1
x1 x1 x1 x1
+1 -1
L3 L4
-1 -1
x1 Î [-1 +1]
x 2 Î [-1 +1]
 For the “‐norm”, there are four cases we have to consider
x2
 Case‐1 : L1
L1 : max { x 1 , x 2 } = max {+1, -1 £ x 2 £ +1} = 1 x1

x2
 Case‐2 : L2
L1
L2 : max { x1 , x 2 } = max {-1, -1 £ x 2 £ +1} = 1 x1
x2
 Case‐3 : L3
L3 : max { x 1 , x 2 } = max {-1 £ x 1 £ +1, +1} = 1 x1
x2
 Case‐4 : L1
L4 : max { x 1 , x 2 } = max {-1 £ x1 £ +1, -1} = 1 x1

L4

x1 Î [-1 +1]
x 2 Î [-1 +1]
 Moreover, it is trivial to show that the “2‐norm” is an unit
circle since æ 2 ö1/2
x 2 = ççå x i2 ÷÷÷ = x12 + x 22 = 1
çèi =1 ø
 Following we put all the charts in a single figure given the
x2
constraints
x 1 = x1 + x 2 = 1 1-norm  diamond
æ 2 2 ö÷1/2 x1
x 2 = ççå x i ÷÷ = x12 + x 22 = 1
çèi =1 ø 2-norm  circle
x ¥ = max { x 1 , x 2 } = 1 -norm  square
 More, it is easy to show a property that x¥£ x2£ x1

 For example é 3ù
x= ê ú  x ¥ =4£ x 2 =5£ x 1 =7
ëê-4úû
 Inequality explanation:
x¥£ x2£ x1
éx 1 ù
 Consider the following 2D case, where x = ê ú Î  2´1
êëx 2 úû
2 2
1-norm : x 1 = x1 + x 2 x1 + x2
æ 2 2 ö÷1/2
2-norm : x 2 = ççå x i ÷÷ = x12 + x 22
çèi =1 ø x2
-norm : x ¥ = max { x 1 , x 2 }
x1
Vector Norms
 In Linear Algebra, a “norm” refers to the length of all the
vectors in a space.
 So far, we already introduced that there are different ways to
measure the magnitude of vectors, including:
 1‐norm
m
x 1 = å x i = x1 + x 2 +  + x m
i =1
 2‐norm 1/ 2
æ m ö 1/ 2
1/ 2
 x 2 = x, x
x 2 = ççå x i2 ÷÷÷ = (x12 + x 22 +  + x m2 )
çèi =1 ø 2
 x 2 = x, x = xT x
 ‐norm
x ¥ = max { x 1 , x 2 , x 3 ,, x m }
What are the engineering applications for these norms? Bonus!

Gram‐Schmidt Process
 In mathematics, particularly linear algebra and numerical
analysis, the Gram–Schmidt Process (GSP) is a method
for orthonormalizing a set of vectors in an inner product space,
most commonly the Euclidean space Rn equipped with
the standard inner product.
 The Gram–Schmidt Process (GSP) takes a finite, linearly
independent set S = {v1, ..., vk} for k ≤ n and generates
an orthogonal set S′ = {u1, ..., uk} that spans the same k‐
dimensional subspace of Rn as S.
S = {v1 , v2 ,, vk }  S ' = {u1 , u2 ,, uk }

 ui , u j = uTi u j = 0 when i ¹ j

 Recall the projection operator defined by
v, u
Proju (v) = u
u, u
 <v,u> denotes the inner product of the vectors v and u.

 This operator projects the vector v orthogonally onto the
line spanned by vector u.
 The first two steps of the Gram–Schmidt process is
v2
u1 = v1
u2 = v2 - Proju1 (v2 ) u2
v1
 Where u1 & u2 are orthogonal. e2 u1
e1 & e2 are unit vectors. Proju1 (v2 )
e1

v2
 Given u1 = v1 u2
u2 = v2 - Proju1 (v2 ) v1
where v ,u e2 u1
Proju1 (v2 ) = 2 1 u1
u1 , u1 Proju1 (v2 )
e1
 It is easy to show that u1 and u2 are mutually perpendicular.
 Proof: u1 , u 2 = u1 , v2 - Proju (v2 ) = Proju (v2 ), v2 - Proju (v2 )
1 1 1
v2 , u1 v ,u v ,u v , u v2 , u1 T
= u1 , v2 - 2 1 u1 = 2 1 uT1 v2 - 2 1 u1 u 1
u1 , u1 u1 , u1 u1 , u1 u1 , u1 u1 , u1
v 2 , u1 v , u v 2 , u1
= v2 , u 1 - 2 1 u1 , u1
u1 , u1 u1 , u1 u1 , u1
v 2 , u1 v ,u
= v2 , u 1 - 2 1 v2 , u1 = 0
u1 , u1 u1 , u1

v, u
Proju (v) = u
u, u
 The overall Gram–Schmidt process works as follows:
u1 = v1
Note that when this process
u 2 = v2 - Proju1 (v2 )
is implemented on a computer,
u 3 = v 3 - Proju1 (v 3 ) - Proju2 (v 3 ) the vectors uk are often NOT
quite orthogonal, due to
u 4 = v 4 - Proju1 (v4 ) - Proju2 (v 4 ) - Proju3 (v 4 )
rounding errors. The (classical)
 Gram–Schmidt Process is
k -1
uk = vk - å j =1 Proju j (vk ) numerically unstable.
 The sequence u1, ..., uk is the required system of orthogonal

vectors.
 The normalized vectors e1, ..., ek form an orthonormal set,
where
u1 u u u
e1 = , e2 = 2 , e3 = 3 , , ek = k
u1 u2 u3 uk

u1 = v1
u2 = v2 - Proju1 (v2 )
Gram‐Schmidt Process u 3 = v 3 - Proju1 (v 3 ) - Proju2 (v 3 )
 Following illustrates the geometric procedure of the GSP:
Z Z
u3
v3
u2 = v2 - Proju1 (v2 ) Y Proju2 (v 3 ) Y
v2
Proju1 (v2 ) Proju1 (v 3 )
u1 = v1 X u1 X
u1 = v1
u2 = v2 - Proju1 (v2 ) u 3 = v 3 - (Proju1 (v 3 ) + Proju2 (v 3 ))

Orthogonal Vector

Orthogonal Vector(ex.)
 Consider a set S, which is composed of two vectors
S = {x1 , x2 }
x2
 For the following vectors
ìï x , x = xT x = 0
ï 1 2 1 2
é 1ù é0ù ïï T x1
x1 = ê ú , x2 = ê ú  íx1 x1 = 1
êë0úû êë1úû ïï orthonormal
ïïxT2 x2 = 1
ïî
 For the following vectors
x1
ì
ïx1 , x2 = xT1 x2 =0
ï
é1ù é1ù ï ï
x1 = ê ú , x2 = ê ú  íxT1 x1 = 2 ¹ 1
êë1úû êë-1úû ï
ï
ï T orthogonal
ïx x =2 ¹1
ï
î 2 2
 Geometric interpretation : perpendicular x2

v, u
Proju (v) = u
u, u
Orthogonal Vector(ex.)
 Given the following three vectors in a set S, please find the
associated orthogonal set S’: v
v2 1 v3 u1
é 1ù é 1ù é7 ù v1 u1
v2 v3
ê ú ê ú ê ú
v1 = 2 , v2 = 5 , v3 = ê 8ú
ê ú ê ú
ê ú ê ú ê ú
ê 3ú ê 6ú ê 9ú
ë û ë û ë û
u1 = v1 u2
u2
u 2 = v2 - Proju1 (v2 ) u3
u 3 = v 3 - Proju1 (v3 ) - Proju2 (v 3 ) u3
 Please note that the computation may yield poor results when
some of the vectors {vi} are almost linearly dependent.

Matrix
 Matrices are commonly written in box brackets or parentheses:
é a11 a12  a1n ù 1st row

ê ú
ê a21 a22  a2n ú
A=ê ú Î Âmń
ê     ú
ê ú
ê  amn ûú
ëam 1 am 2
1st column
 The entry in the i‐th row and j‐th column of a matrix A is
sometimes referred to as the i,j, (i,j), or (i,j)th entry of the
matrix, and most commonly denoted as ai,j, or aij.
A=[11 22 33 44;55 66 77 88; 99 00 -11 -22];

%%... detect the size of a given matrix
[MM,NN] = size(A)

Matrix
 We can use MATLAB to extract every element from a given
matrix. é11 22 33ù
A=[11 22 33;44 55 66;77 88 99];
%%... extract a11
ê ú a11=A(1,1)
A = 44 55 66ú Î Â 3´3
ê %%... extract a12
ê ú a22=A(2,2)
ê ú
ë77 88 99û %%... extract a13
a33=A(3,3)
 Moreover, one can specify a row
é 11 22 33ù %%... 1st row
é 11 22 33ù é11 22 33ù
ê ú ê ú ê ú A1 = A(1,:)
A = 44 55 66ú
ê A = ê 44 55 66ú A = 44 55 66ú
ê %%... 2nd row
ê ú ê ú ê ú A2 = A(2,:)
ê77 88 99ú ê77 88 99ú ê77 88 99ú %%... 3rd row
ë û ë û ë û A3 = A(3,:)
 Or a column
%%... 1st column
é 11 22 33ù é 11 22 33ù é 11 22 33ù A1 = A(:,1)
ê ú ê ú ê ú %%... 2nd column
A = ê 44 55 66ú A = ê 44 55 66ú A = ê 44 55 66ú A2 = A(:,2)
ê ú ê ú ê ú %%... 3rd column
ê77 88 99ú ê77 88 99ú ê77 88 99ú A3 = A(:,3)
ë û ë û ë û
Matrix
 Specially, one can also extract a sub matrix from the given
matrix.
é 11 22 33 44 ù é 11 22 33 44 ù
ê ú ê ú
ê 55 66 77 88 ú ê 55 66 77 88 ú
A=ê ú Î Â 4´4 A= ê ú Î Â 4´4
ê 99 0 -11 -22ú ê 99 0 -11 -22ú
ê ú ê ú
ê-33 -44 -55 -66ú ê-33 -44 -55 -66ú
ë û ë û
A_sub = A(1:2,1:2) A_sub = A(2:4,2:4)
é 11 22 33 44 ù é 11 22 33 44 ù
ê ú ê ú
ê 55 66 77 88 ú ê 55 66 77 88 ú
A= ê ú Î Â 4´4 A= ê ú Î Â 4´4
ê 99 0 -11 -22ú ê 99 0 -11 -22ú
ê ú ê ú
ê-33 -44 -55 -66ú ê-33 -44 -55 -66ú
ë û ë û
A_sub = A(:,2:3) A_sub = A(2:4,:)

Matrix
 Basic operations (Wikipedia)
Operation Definition
Addition The sum A+B of two m-by-n matrices A and B is calculated
entrywise:(A + B)i,j = Ai,j + Bi,j, where 1 ≤ i ≤ m and 1 ≤ j ≤ n.
Scalar multiplication The product cA of a number c (also called a scalar in the
parlance of abstract algebra) and a matrix A is computed by
multiplying every entry of A by c:(cA)i,j = c · Ai,j.
This operation is called scalar multiplication, but its result is
not named "scalar product" to avoid confusion, since "scalar
product" is sometimes used as a synonym for "inner product".
Transposition The transpose of an m-by-n matrix A is the n-by-

m matrix AT formed by turning rows into columns and vice
versa:(AT)i,j = Aj,i.

Matrix
 Basic operations (Wikipedia)
Operation Definition
Addition The sum A+B of two m-by-n matrices A and B is calculated
entrywise:(A + B)i,j = Ai,j + Bi,j, where 1 ≤ i ≤ m and 1 ≤ j ≤ n.
Scalar multiplication The product cA of a number c (also called a scalar in the
parlance of abstract algebra) and a matrix A is computed by
multiplying every entry of A by c:(cA)i,j = c · Ai,j.
This operation is called scalar multiplication, but its result is
not named "scalar product" to avoid confusion, since "scalar
product" is sometimes used as a synonym for "inner product".
Transposition The transpose of an m-by-n matrix A is the n-by-

m matrix AT formed by turning rows into columns and vice
versa:(AT)i,j = Aj,i.

Matrix
 Definition: if A is an mxn matrix and B is an nxr matrix, then the
product AB = C is an mxr matrix, whose entries are defined by
é a11 a12  a1n ù é b11 b12  a1r ù
ê úê ú
ê a21 a22 ú ê b21 b22 ú
C = AB = ê ú ê ú  cij = ånk =1aikbkj
ê   úê   ú
ê úê ú
êam 1  ú ê
amn û ëan 1  bnr úû
ë
 Note that when both A and B are mxm matrices, AB BA in
usual!! é1 2 ù é 5 6ù é 5 6ù é1 2 ù
ê úê ú¹ê úê ú matrix multiplication is not,
êë 3 4úû êë7 8úû êë7 8úû êë3 4úû in general, commutative.
 
A B B A
 Definition: An nxn matrix “A” is said to be invertible if the inverse

“A‐1” exists.
 Definition: An nxn matrix “A” is said to be singular if it does not have
an inverse; that is “A‐1” doesn’t exist.
Matrix (Ex.)
 Matrices can be used to compactly write and work with
multiple linear equations, that is, systems of linear equations.
a11x1 + a12x 2 +  + a1n x n = b1 é a11 a12  a1n ù é x1 ù é b1 ù

ê ú ê ú ê ú
a21x 1 + a22x 2 +  + a2n x n = b2 ê a21 a22  a2n ú êx2 ú êb ú
ê ú ê ú = ê 2ú
 ê     ú êú êú
ê ú ê ú ê ú
êa ú êx ú êb ú
am 1x 1 + am 2x 2 +  + amn x n = bm m 1 am 2  amn û
ë ë nû ë mû

AÎÂmń XÎÂn´1 bÎÂm´1
1
A must exist
m=n
AX  b X  A 1b
 where A is an m‐by‐n matrix, x designates a column vector
(that is, n×1‐matrix) of n variables x1, x2, ..., xn, and b is
an m×1‐column vector.

Matrix
 For a system of linear equations, there are three conditions:
 Case. A : uniquely determined : if the number of equations
equals to unknowns unique solution exists
 Case. B : underdetermined : if there are fewer equations
than unknowns  infinity solutions
 usually minimum distance (norm) solution is prefer
 Case. C : overdetermined : if there are more equations than
unknowns  no exact solution
 but there still exists an optimal one

Matrix (Ex.)
 Case. A : uniquely determined case:
1⋅ x1 + 1⋅ x 2 + 1⋅ x 3 = 8 é1 1 1 ù éx1 ù é 8 ù
ê úê ú ê ú
1⋅ x1 - 2 ⋅ x 2 + 1⋅ x 3 = 4 ê1 -2 1 ú êx 2 ú = ê 4 ú
ê úê ú ê ú
1 ⋅ x 1 + 1 ⋅ x 2 - 1 ⋅ x 3 = -4 ê1 1 -1ú êx 3 ú ê-4ú
ëû ëû  ë û
A X b
Square
 The matrix A is of full rank and therefore the inverse of A exists.
é1 1 1ù é8ù
ê ú ê ú
A = ê1 -2 1 ú b=ê 4 ú X = A-1b
ê ú ê ú
ê1 1 -1ú ê-4ú
ë û ë û
A=[1 1 1;1 ‐2 1;1 1 ‐1];
b=[8;4;‐4];
x_ans = inv(A)*b

X = A-1b X = A+ b
Matrix (Ex.) Pseudo-inverse
 Case. B : underdetermined (under‐constrained):
11⋅ x 1 + 22 ⋅ x 2 + 33 ⋅ x 3 = 10 éx ù
44 ⋅ x1 + 55 ⋅ x 2 + 66 ⋅ x 3 = 11 é11 22 33ù ê 1 ú é10ù
ê ú êx 2 ú = ê ú
êë 44 55 66úû ê ú êë11úû
 êx 3 ú 
A=[11 22 33;44 55 66]; A ëû b
X
b=[10;11]; Non-square
x_ans = pinv(A)*b
 Case. C : overdetermined case (over‐constrained):
1⋅ x1 + 1⋅ x 2 = 1
1⋅ x1 - 1⋅ x 2 = 3 é 1 1ù é 1ù
ê ú éx1 ù ê ú
-1 ⋅ x 1 + 2 ⋅ x 2 = - 2 ê 1 -1ú ê ú = ê 3ú
ê ú êëx 2 úû ê ú
ê-1 2úû  êë-2úû
ë X 
A=[1 1;1 ‐1; ‐1 2];
A b
b = [1;3;‐2]
x_ans = pinv(A)*b Non-square

Matrix (Ex.)
 To find out whether the linear equation has unique solution,
infinite solutions or no solution, use the so‐called “Reduced
Row‐echelon Form”
é1 1 1 ù éx1 ù é 8 ù éx ù é 1 1ù é 1ù
ê úê ú ê ú é11 22 33ù ê 1 ú é10ù ê ú éx1 ù ê ú
ê1 -2 1 ú êx 2 ú = ê 4 ú ê ú êx 2 ú = ê ú ê 1 -1ú ê ú = ê 3ú
ê úê ú ê ú êë 44 55 66úû ê ú êë11úû ê ú êëx 2 úû ê ú
ê1 1 -1ú êx 3 ú ê-4ú  êx 3 ú  ê-1 2úû  êë-2úû
ëû ëû  ë û A ëû b ë X 
A X b X A b
A=[1 1 1;1 ‐2 1;1 1 ‐1]; A=[11 22 33;44 55 66]; A=[1 1;1 ‐1; ‐1 2];

b=[8;4;‐4]; b=[10;11]; b = [1;3;‐2]
rref([A b]) rref([A b]) rref([A b])
unique solution infinite solution no solution

Underdetermined Matrix Solution
 As for now, we already know that there are
 infinity solutions for underdetermined case
 no solution for overdetermined case.
 Actually, for the “underdetermined” case, we can find a so‐
called minimum 2‐norm solution.
 On the other hand, for the “overdetermined” case, we can find
a so‐called minimum squared error solution.
éx1 ù é 1 1ù é 1ù
é11 22 33ù ú é10ù
ê ê ú éx1 ù ê ú
ê ú êx ú = ê ú ê 1 -1ú ê ú = ê 3ú
êë 44 55 66úû ê 2 ú êë11úû ê ú êëx 2 úû ê ú
 êx 3 ú  ê-1
A ëû b 2úû  êë-2úû
ë
X
 X 
A b
under-constrained over-constrained
no unique solution no solution
 Derivation of the minimum 2‐norm solution via a Lagrange
multiplier.
 For the underdetermined case, we tend to
éx ù
minimize x x T é11 22 33ù ê 1 ú é10ù
ex. ê ú êx ú = ê ú
subject to Ax = b êë 44 55 66úû ê 2 ú êë11úû
 êx 3 ú 
A ëû b
x
 By introducing a Lagrange multiplier, we have
L (x , l ) = xT x + l (Ax - b)
 The optimal solution can be obtained by taking
¶ ¶
L (x, l ) = 0  L (x, l ) = 2x + lAT = 0
¶x ¶x
¶ ¶
L (x, l ) = 0  L (x, l ) = Ax - b = 0
¶l ¶l

 Based on the two equations
T AT
2x + l A = 0  x = - l
2
 We have
æ AT ö÷ -1
Ax - b = A çç- l ÷÷ - b = 0  l = -2(AAT ) b
çè 2 ø÷
 Finally, it is easy to obtain the optimal solution by substituting
 into
2 ()
AT AT -1
xopt := - l =- -2(AA ) b T
2
l
-1
= AT (AAT ) b = A+b
T -1
 A := A (AA )
† T
Note that the term is called pseudo inverse.

 Proof of minimum norm solution:
 Based on the previous derivations, we already know that
-1
xopt = AT (AAT ) b = A+b
is one of the solution satisfying Ax = b
 xs
Suppose there exists another arbitrary solution , which also
Ax = b
satisfies the equation .
 Then, we have
Axopt = büïï
ý  A (xs - xopt ) = 0
Axs = b ïþï
 It gives
T -1
(AA )
T T
 (xs - xopt ) xopt = (xs - xopt ) A T
b
( )
T -T
= bT (AAT )
T
 (xs - xopt ) xopt A (xs - xopt ) = 0

0

 Proof of minimum norm solution (continue):
 The previous result shows that
T
(xs - xopt ) xopt = 0  (xs - xopt ) ^ xopt  Dx ^ xopt
 Form the geometric point of view, it follows
 xs = xopt + Dx  xs = xopt + Dx xs
T
Dx := xs - xopt
 xTs xs = (xopt + Dx) (xopt + Dx)
= xTopt xopt + 2 xTopt Dx + DxT
Dx
  
³0 xopt
³0 0
2 2 2 2 2 2
 xs = xopt + Dx  xopt = xs - Dx
 Therefore, it is evident that for any .
Dx ¹ 0  Dx ³ 0
 xs ³ xopt
This concludes that  minimum norm solution.
 Consider the following examples
éx1 ù
éx1 ù é11 22 33ù ú é10ù
ê
Ex.1 1 1] ê ú = 1
[ Ex.2 ê ú êx ú = ê ú
êëx 2 úû b êë 44 55 66úû ê 2 ú êë11úû
A1  1  êx 3 ú 
x1 A2 ëû b2
x2
 Taking the formula of the minimum norm solution
-1
xopt = AT (AAT ) b = A†b
gives the associated solutions æ ö÷-1
çç ÷
é1ù çç é1ù ÷÷ é 0. 5 ù
÷
xopt 1 = ê ú çç[1 1] ê ú ÷ 1 = ê ú
x1 + x 2 = 1 êë1úû ç  êë1úû ÷÷ b êë 0.5úû
 çç A1  ÷÷
AT 1 è
ç AT 1 ø
÷
min x 2 (0.5, 0.5) the shortest distance
solution é-0.4141ù
- 1 ê ú
xopt 2 = A2 (A2A2 ) b2 = 0.0101 ú
T T ê
ê ú
ê 0.4343 ú
ë û
 An engineering application: a multi‐rotor force generation
problem
 Here we have a force distribution matrix
é F1 ù
ê ú
Thrust éu ù é 1 1 1 1 1 1 êF2 ú
ù
ê ú ê úê ú
Roll torque ê tj ú ê-lc cos (qH ) lc cos (qH ) 1 lc cos (qH ) -lc cos (qH ) -1ú êF3 ú
ê ú=ê úê ú
ê tq ú ê -l sin (q ) -l sin (q ) 0 l sin (q ) l sin (q ) 0 ú êêF4 úú
Pitch torque  ê ú ê c H c H c H c H ú
êty ú ê m -m m -m m -mú êF ú
Yaw torque  ëû ë û ê 5 ú
b A êF ú
ë 6û
X
 An engineering application: a multi‐rotor force generation
problem
There are 6 variables but only 4 constraints!
é F1 ù
ê ú
éu ù é 1 1 1 1 1 1 ù êF2 ú
ê ú ê úê ú
ê tj ú ê-lc cos (qH ) lc cos (qH ) 1 lc cos (qH ) -lc cos (qH ) -1ú êF3 ú
ê ú=ê úê ú
ê ú ê c H c H c H c H ú
êt ú ê m -m m -m m -múû êêF5 úú
ë yû ë 
b A êF ú
ë 6û
X
generated by flight controller
 Problem statement : given “b” and “A”, please find “X”.
 The minimum norm solution is
-1
xopt = AT (AAT ) b = A†b

Orthogonal Matrix
 For an nxn square matrix Q, whose column vectors form an
orthonormal set in Rn. Such matrices are called orthogonal
matrices.
Q = [ x1 , x2 , xn ] Î nń , xi Î n´1
 Based on the property of the orthonormal set, one has
xTi x j = 0 i¹j
if
xTi x j = 1 i=j
 Therefore, we can obtain that
é xT ù éxT x xT x  xT1 xn ùú é1 0  0ù
ê ú1 ê 1 1 1 2
ê Tú ê T ú êê0 1  0ú
ú
ê x2 ú ê x2 x1 xT2 x2 T
 x 2 xn ú
Q Q = ê ú [ x1 , x2 , xn ] = ê ú = êê  ú = Inń
T
ê  ú ê     ú ê   ú
ê Tú ê T ú ê ú
êë xn úû êëxn x1 xTn x2  xn xn úû ë0
T 0  1úû

Orthogonal Matrix
 For an nxn square matrix Q, whose column vectors form an
orthonormal set in Rn. Such matrices are called orthogonal
matrices.
Q = [ x1 , x2 , xn ] Î nń , xi Î n´1
 Based on the property of the orthonormal set, one has
xTi x j = 0 i¹j
if
xTi x j = 1 i=j
 Theorem:
 If Q is an orthogonal matrix, then Q is invertible and QT=Q‐1
QT Q = Inń  QT = Q-1

Orthogonal Matrix
 Consider an orthogonal matrix Q, it has the following
properties:
 Preservation of inner products
T
Qx, Qy = (Qx) (Qy) = xT Q

T
Q y = xT
y
I
 Preservation of norms
( )
1/ 2 1/ 2 1/ 2
= (x Q Qx) = (x x)
T
Qx = (Qx) (Qx) T T T
= x
 Preservation of distances
Qx - Qy = x - y
 Preservation of angles
æ(Qx)T (Qx)ö÷ æ xT y ö÷
-1 ç ÷÷ = cos çç
-
(Qx, Qy) = cos çç 1
÷ =  ( x, y )
çè Qx ⋅ Qy ÷ø÷ èç x ⋅ y ÷ø÷

Orthogonal Matrix (Ex.)
 Show that the rotation matrix RF is an orthogonal matrix
R F := R y (y) R q (q ) Rf (f)
écos y -sin y 0ù é cos q 0 sin q ù é1 0 0 ù
ê ú ê ú ê ú
RTF = R-F 1 ê
= sin y cos y 0 ⋅ 0 ú ê 1 0 ⋅ 0 cos f -sin f ú
ú ê
ê ú ê ú ê ú
 ê 0 0 1û ë-sin q 0 cos q û ë 0 sin f cos f úû
ú ê ú ê
ë
RTF R F = I écos y cos q - sin y cos f + cos y sin q sin f sin y sin f + cos y sin q cos f ù
ê ú
= sin y cos q cos y cos f + sin y sin q sin f - cos y sin f + sin y sin q cos f ú
ê
ê ú
ê -sin q cos q sin f cos q cos f ú
ë û
yaw pitch roll

ìï V ,V = 0 　if 　i ¹ j
ïï i j
í ' '
ïï V ,V = 0 　if 　i ¹ j
Orthogonal Matrix (Ex.) ïî i j
 Consider R F = R y (y) R q (q) Rf (f)

= R y (45 ) R q (45 ) Rf (45 ) V3
V1' = R FV1 V3' V2'
V3 V2' = R FV2
V2
V3' = R FV3 V1
V2
V1 V1'
after
rotation
Orthogonal Matrix Properties:

1. Preservation of inner products
2. Preservation of norms
3. Preservation of distances
4. Preservation of angles
V1' = R FV1
V2' = R FV2
V3' = R FV3

 Rotation matrix sequences: Rrot = R y (y) y : 0  45
 Rotation 321 convention: Rrot = R y (y) R q (q ) q : 0  45

Rrot = R y (y) R q (q ) Rf (f) f : 0  45
Rotates w.r.t the

current body frame!

 Rotation matrix sequences: Rrot = R y (y) y : 0  45
 Rotation 123 convention: Rrot = R q (q ) R y (y) q : 0  45

Rrot = Rf (f) R q (q ) R y (y) f : 0  45
Rotates w.r.t the

global inertial frame!

Rotation 321 convention Rotation 123 convention
Rrot = R y (y) y : 0  45 Rrot = R y (y) y : 0  45

Rrot = R y (y) R q (q ) q : 0  45 Rrot = R q (q ) R y (y) q : 0  45
Rrot = R y (y) R q (q ) Rf (f) f : 0  45 Rrot = Rf (f) R q (q ) R y (y) f : 0  45
Rotates w.r.t the Rotates w.r.t the

current body frame! global inertial frame!

Rotation w.r.t the Body Frame Rotation w.r.t the Inertial Frame
Rrot = R y (y) y : 0  45 Rrot = Rf (f) f : 0  45

Rrot = R y (y) R q (q ) q : 0  45 Rrot = R q (q ) R f (f) q : 0  45
Rrot = R y (y) R q (q ) Rf (f) f : 0  45 Rrot = R y (y) R q (q ) Rf (f) y : 0  45
The transition behavior is different.
But, the steady state is the same
Rotation w.r.t the Body Frame Rotation w.r.t the Body Frame
Rrot = R y (y1 ) y1 : 0  45 Rrot = Rf (f1 ) f1 : 0  45

Rrot = R y (y1 ) R q (q ) q : 0  45 Rrot = Rf (f1 ) R y (y) y : 0  45
Rrot = R y (y1 ) R q (q ) R y (y2 ) y2 : 0  45 Rrot = Rf (f1 ) R y (y) R f (f2 ) f2 : 0  45

Orthogonal Function
 The given two functions x(t) and y(t) are said to be orthogonal
over the interval [T1, T2] if
T
x (t ), y (t ) = òT 2 x (t )y (t )dt = 0
1
 Sometimes, the given two functions(or signals) are also
referred to as independent (or uncorrelated) if they are
mutually orthogonal.
 In addition, the individual function is said to be orthonormal
over the interval [T1, T2] if
2 æ T2 ö÷2
x (t ) , x (t ) = x (t ) = çç òT1 x (t )x (t )dt ÷÷ = 1
è ø
2 æ T2 ö÷2
y (t ), y (t ) = y (t ) = çç òT1 y (t )y (t )dt ÷÷ = 1
è ø
Orthogonal Function
 If two functions are orthogonal, they can also be referred as
mutually independent.

product-to-sum
Orthogonal Function (ex.)
 Consider the cosine functions
km
k=m

 Consider the following signals that is a linear combination of
three harmonic functions with known different frequencies:
y (t ) = A1 sin(w1t ) + A2 sin(w2t ) + A3 sin(w3t )
 The question is: how to steal the coefficient (i.e., magnitude)

of with respect to each harmonic sinewave, respectively?
 Note that for practical consideration, A1~A3 are unknown. The
only available signal is y(t) and the frequencies 1, 2 and 3.
 Recall that the orthogonal property
ìï 2p sin nx sin mx = 0 ìï 2p sin nx sin nx = p
ïïò0 ( ) ( ) ïïò0 ( ) ( )
í 2p if n ¹m í if n =m
ïï ïï 2p cos mx cos mx = p
ïîò0 cos (nx ) cos (mx ) = 0 ïîò0 ( ) ( )

 For any n & m, the following identity holds
2p 1 2p
ò0 sin (nx ) cos (mx ) = ò (sin ((n + m )x ) + sin ((n - m )x ))dx
2 0
é 2p ù
1 1 1
= - êê cos ((n + m )x ) + cos ((n - m )x ) úú
2 ê(n + m ) (n - m ) 0 úû
ë
éæ 1 1 ö÷ù
êçç ÷ú
êçè(n + m ) cos ((n + m )⋅ 2p) + (n - m ) cos ((n - m ) ⋅ 2p)÷÷øú
1
= - êê ú
ú
2ê æ 1 1 ö ú
ê-ççç cos ((n + m )⋅ 0) + cos ((n - m )⋅ 0)÷÷÷ ú
êë è(n + m ) (n - m ) ÷ø ú
û
=0

 y (t ) = A1 sin(w1t ) + A2 sin(w2t ) + A3 sin(w3t )
Since , by multiplying each
harmonic function to y(t), it is obviously that
2p 2p
0 0
ò0 (y (t )sin(w1t ))dt = ò0 (A1 sin(w1t )sin(w1t ) +A2 sin(w2t )sin(w1t ) +A3 sin(w3t )sin(w1t ))dt = pA1
0 0
2p 2p
ò0 (y (t )sin(w2t ))dt = ò0 (A1 sin(w1t )sin(0w2t ) +A2 sin(w2t )sin(
0
w2t ) + A3 sin(w3t )sin(w2t ))dt = pA2
2p 2p
ò0 (y (t )sin(w3t ))dt = ò0 (A1 sin(w1t )sin(w3t ) +A2 sin(w2t )sin(w3t ) +A3 sin(w3t )sin(w3t ))dt = pA3
 Thus we have
2p 2p 2p
ò0 (y (t )sin(w1t ))dt ò0 (y (t )sin(w2t ))dt ò0 (y (t )sin(w3t ))dt
A1 = A2 = A3 =
p p p
 Further application will be extended for Fourier Transform
(spectrum analysis)

Orthogonal Function
 For N number of 2π period, where N is positive integer
N ⋅2 p
ò0 sin (nx )sin (nx ) = N ⋅ p
N ⋅2 p
ò0 cos (nx ) cos (mx ) = N ⋅ p
 Proof: N ⋅2 p N ⋅2 p
ò0 sin (nx ) sin (nx )dx = ò0 sin 2 (nx )dx
N ⋅2 p
1 N ⋅2p 1æ 1 ö
= ò0 (1 - cos 2nx )dx = ççx - sin 2nx ÷÷÷
2 2è 2n ø0
1æ 1 ö
= ççN ⋅ 2p - sin 2N ⋅ 2p÷÷÷
2è 2n ø
= N ⋅p

 Based on the preceding example, it is now modified to read
N ⋅2p
ò0 (y (t )sin(w1t ))dt = N ⋅ pA1
N ⋅2p
N ⋅2p
 Thus it follows
N⋅2p N ⋅2p N ⋅2p

ò0 (y (t )sin(w1t ))dt ò0 (y (t )sin(w2t ))dt ò0 (y (t )sin(w3t ))dt
A1 = A2 = A3 =
N ⋅p N ⋅p N ⋅p

 Consider the mixed signal
y (t ) = 3 sin(6t ) + 4 sin(7t ) + 5 sin(8t )
close all; clear; clc
A1 = 3; A2 = 4; A3 = 5;
w1 = 6; w2 = 7; w3 = 8;
dT = 0.001; N = 1;
t = 0:dT:N*2*pi;
y1 = sin(w1*t);
y2 = sin(w2*t);
2p
y3 = sin(w3*t);
ò0 (y (t )sin(w1t ))dt
y = A1*y1 + A2*y2 + A3*y3; A1 =
subplot(4,1,1); plot(t,y); axis([0 2*pi ‐15 15]); ylabel('y'); N ⋅p
subplot(4,1,2); plot(t,A1*y1,'r'); axis([0 2*pi ‐5 5]); ylabel('A1*y1'); 2p
subplot(4,1,3); plot(t,A2*y2,'g'); axis([0 2*pi ‐5 5]); ylabel('A2*y2'); ò0
(y (t )sin(w2t ))dt
subplot(4,1,4); plot(t,A3*y3,'m'); axis([0 2*pi ‐5 5]); ylabel('A3*y3'); A2 =
N ⋅p
xlabel('Time');
2p
A1_est = sum(y.*y1*dT)/(N*pi) A1_est = 3.0000 ò0
(y (t )sin(w3t ))dt
A2_est = sum(y.*y2*dT)/(N*pi)
A2_est = 4.0000 A3 =
A3_est = sum(y.*y3*dT)/(N*pi) N ⋅p
A3_est = 5.0000

Homework
 1. Can we take the inner product for the following pair?(5%) Why?
éx ù é y1 ù
ê 1ú ê ú
êx2 ú ê y2 ú
x = ê ú Î Rn , y = ê ú Î Rm , n ¹ m
êú ê  ú
ê ú ê ú
êx n ú êym ú
ë û ë û
 x = [ 3 1 -2 ]
2. For the given vector , find all the orthogonal vectors based
on the following candidates. (Please include detail calculations!) (10%)
x1 = [ 0 4 2 ] x2 = [-1 -1 -2 ] x 3 = [-3 -1 2 ] x 4 = [ 0.5 -1 0.25 ]
 3. Please calculate the corresponding unit vectors. (5%)
x = [1 2 3 4 ]
y = [-2 3 1 -5 7 10 ]

Homework
 4. Please use matrix operation to solve the following linear
equations. If there is no exact solution, please provide your
reason. Also attach your MATLAB code. (20%)
ìïx 1 - 3x 2 + x 3 = 1
ìï3x1 + 2x 2 - x 3 = 4 ì
ïx 1 - x 2 + 2x 3 = 4 ïï
ì
ïx1 - 2x 2 = 3 ï
ï ï
ï ïï2x1 + x 2 - x 3 = 2
ï
A.í B.íx 1 - 2x 2 + 2x 3 = 1 C .í2x 1 + 3x 2 - x 3 = 1 D.í
ï
ï2x
î 1 - x 2 = 9 ï
ïï11x + 2x + x = 14 ï
ï ïïx 1 + 4x 2 - 2x 3 = 1
ïî 1 2 3 ï
ï7
î 1x + 3 x 2 + 4x 3 = 7 ïï
ïî5x 1 - 8x 2 + x 3 = 5
 5. Determine if the following matrices satisfy the orthogonality.
(Please provide the computation detail.) (20%)
é 3 / 5 0 -4 / 5ù é cos q -sin q 0ù
é 0 1ù é1 -1ù ê ú ê ú
A. Q = ê ú B. Q = ê ú ê
C. Q = 0 1 0 ú D. Q = sin q cos q 0ú
ê
êë-1 0úû êë1 1 úû ê ú ê ú
ê4 / 5 0 3 / 5 ú ê 0 0 1úû
ë û ë

Homework
 6. Consider the following rotation matrix RF.
écos y cos q -sin y cos j + cos y sin q sin j sin y sin j + cos y sin q cos j ù
ê ú
R F = sin y cos q cos y cos j + sin y sin q sin j - cos y sin j + sin y sin q cos j ú
ê
ê ú
ê -sin q cos q sin j cos q cos j ú
ë û
 Please give a proof that the matrix is orthogonal. (5%)
RTF = R-F 1
 Hint :

RTF R F = I

Homework
 7. Here we have an experiment : a gyroscope was set static on
a desk. There are Nx6 dimension signals sampled at 100Hz and
recorded in the Matlab file ‘IMU_10min.mat’. The last three
columns stands for the gyroscope data (rad/sec). close all
load IMU_10min
 That is gyro = data_list;
dT = 0.01;
time = dT*[0:length(gyro)‐1];
th
 The 4 column is the x‐axis gyroscope data. omega_x = gyro(:,4);
figure
th
 The 5 column is the y‐axis gyroscope data. plot(time, omega_x)
hold on; grid on;
 The 6th column is the z‐axis gyroscope data.
xlabel('Time')
ylabel('\omega_x(rad/s)')
 Please use crest factor ‘CF’ to evaluate which of the axis was
contaminated (by abrupt perturbations) the most; and which
of them has the most minor contamination. (5%)
 Please plot all the time responses for the gyroscope. (5%)
Homework
 8. Consider the given file “Signal.mat”, please plot the time response signal
and further recover the amplitudes A1~A4 of the harmonic component.
Note that the period of the signal N = 3. (10%)
 Please provide your MATLAB Code!
y (t ) = A1 sin(10t ) + A2 sin(13t ) + A3 sin(20t ) + A4 sin(30t )
Hint: A1 = ?
Load the “Signal.mat” in
your MATLAB workspace. A2 = ?
There are two variables in
A3 = ?
the file “t” & “y” A4 = ?

Homework
 9. Based on the Gram–Schmidt process, given
u1 = v1
u2 = v2 - Proju1 (v2 )
u 3 = v 3 - Proju1 (v 3 ) - Proju2 (v 3 )
 Please proof that (10%)
ui , u j = 0, when i ¹ j
 That is
u1 , u 2 = 0
u2 , u 3 = 0
u1 , u 3 = 0
 Note: detail derivations MUST be given!
Homework
 10. For the multi‐rotor UAV, suppose that
 Given é u ù é10ù
ê ú ê ú ìïlc = 0.3
ê tj ú ê 3 ú ïï
Y = ê ú = ê ú, íqH = p / 6
ê tq ú ê 4 ú ïï
ê ú ê ú ïïîm = 0.05
ê ú ê ú
ëty û ë 5 û
 Please find (5%)
T
X = [F1 F2 F3 F4 F5 F6 ] = ?
 where é F1 ù
ê ú
éu ù é 1 1 1 1 1 1 êF2 ú
ù
ê ú ê úê ú
ê tj ú ê-lc cos (qH ) lc cos (qH ) 1 lc cos (qH ) -lc cos (qH ) -1ú êF3 ú
ê ú=ê úê ú
ê ú ê c H c H c H c H ú
êty ú ê m -m m -m m -mú êF ú
ëû ë û ê 5 ú
b A êF ú
ë 6û
X
Homework
 11. Please gives the proof of the following properties: (5%)
Orthogonal Matrix Properties:

1. Preservation of inner products
2. Preservation of norms
3. Preservation of distances
4. Preservation of angles

02 Signal Processing Fundamental Mathematics

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

02 Signal Processing Fundamental Mathematics

Uploaded by

Copyright:

Available Formats

Measurement

1 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

2 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

3 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

4 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

5 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

6 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

7 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

8 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

10 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

DC Sinusoidal Wave DC with Ripple

Saw-tooth Saw-tooth Saw-tooth

13 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

 For discrete case, for a set of number x = {x1 , x 2 ,, x n }, the RMS is

14 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

15 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

fRMS = fPeak x 0.707

A2 æç 1 ö÷ fRMS = fPeak x 0.707

18 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

21 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

23 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

24 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

27 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

28 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

29 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

30 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

31 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

32 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

33 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

36 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

37 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

L1 : max { x 1 , x 2 } = max {+1, -1 £ x 2 £ +1} = 1 x1

L4 : max { x 1 , x 2 } = max {-1 £ x1 £ +1, -1} = 1 x1

39 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

 More, it is easy to show a property that x¥£ x2£ x1

What are the engineering applications for these norms? Bonus!

S = {v1 , v2 ,, vk }  S ' = {u1 , u2 ,, uk }

43 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

 <v,u> denotes the inner product of the vectors v and u.

44 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

45 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

 The sequence u1, ..., uk is the required system of orthogonal

46 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

u2 = v2 - Proju1 (v2 ) Y Proju2 (v 3 ) Y

47 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

48 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

49 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

50 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

é a11 a12  a1n ù 1st row

A=[11 22 33 44;55 66 77 88; 99 00 -11 -22];

51 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

53 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

Transposition The transpose of an m-by-n matrix A is the n-by-

54 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

Transposition The transpose of an m-by-n matrix A is the n-by-

55 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

 Definition: An nxn matrix “A” is said to be invertible if the inverse

a11x1 + a12x 2 +  + a1n x n = b1 é a11 a12  a1n ù é x1 ù é b1 ù

57 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

58 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

59 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

Matrix (Ex.) Pseudo-inverse

60 Dr. C. C. Peng (ccpeng@mail.ncku.edu.tw)

A=[1 1 1;1 ‐2 1;1 1 ‐1]; A=[11 22 33;44 55 66]; A=[1 1;1 ‐1; ‐1 2];