Signal Spaces: AGC DSP

AGC
DSP Signal Spaces

 The purpose of this part of the course is to introduce
the basic concepts behind generalised Fourier
Analysis
 The approach is taken via vector spaces and least
squares approximation
 Modern Signal Processing is based in a substantial
way on the theory of vector spaces. In this course we
shall be concerned with the discrete–time case only
Professor A G Constantinides© 1
AGC
DSP Signal Spaces
 In order to compare two signals we need to define a
measure of “distance” known as a metric.
 A metric is a function d ( x, y ) that produces a
scalar value from two signals such that
 1)
d ( x, y )  d ( y , x )
 2) d ( y, x)  0
 3) d ( x, x )  0
 4) d ( x, z )  d ( x, y )  d ( y , z )
AGC
DSP Signal Spaces
 There are many metrics that can be used.
 Examples:
 1) If we have a set of finite numbers representing
signals then a metric may be
n
d1 ( x, y )   xi  yi
i 1
 This is known as the l1 metric or the Manhattan

distance.
AGC
DSP Signal Spaces
 2) Another metric for the same set is
n 1/ 2

d 2 ( x, y )   xi  yi 
2
i 1 
This is called the l2 metric or the Euclidean
distance
 3) Yet another form is the lp metric
n 1/ p

d p ( x, y )   xi  yi 
p
i 1 
AGC
DSP Signal Spaces
 4) As the integer p  the last form becomes
d  ( x, y )  max xi  yi
i 1, 2 ,..., n
This is called the l metric or distance.

 In channel coding we use the Hamming Distance as a
metric
n
d H ( x, y )   xi  yi 
i 1
where  is the modulo-2 addition of two binary
vectors
AGC
DSP Signal Spaces
 When the set of vectors which we use is defined
along with an appropriate metric then we say we
have a metric space.
 There are many metric spaces as seen from the

earlier discussion on metrics.
 (In the case of continuous time signals we say we

have function spaces)
AGC
DSP Vector Spaces
 We think of a vector x as an assembly of elements
xarranged
i i  1as
,2,3,4,...
 x1 
x 
x   2
.
x 
 n

The length may be in certain cases infinite
AGC
DSP Vector Spaces
 A linear vector space S over a set of scalars R is
formed when a collection of vectors is taken together
with an addition operation and a scalar multiplication,
and satisfies the following:
 1) S is a group under addition ie the following are
satisfied
 a) for any x and y in S then x  y is also in S

b) there is a zero identity element ie
 c) for every element there is another such that their sum is
zero x0x
 d) addition is associative ie
(x  y )  z  x  (y  z )
AGC
DSP Vector Spaces
 2) For any pair of scalars a, b and
R x, y  S
ax  S
a (bx)  (ab)x
(a  b)x  ax  bx
a (x  y )  ax  ay
 3) There is a unit element in the set of scalars R such
that
1x  x
(The set of scalars is usually taken as the set of real
numbers)
AGC
DSP Linear Combination
 Often we think of a signal as being composed of
other simpler (or more convenient to deal with)
signals. The simplest composition is a linear
combination of the form
m
x[n]   ci pi [n]
i 1
 Where { pi [ n]} i  1,2,3,4,... are the simpler

signals, and the coefficients are in the scalar set.
AGC
DSP Vector space …
AGC
DSP Vector space …
AGC
DSP Linear Independence
 If there is no nonzero set of coefficients
{ci } i  1,2,3,4,...
m
such that  ci pi [ n]  0
i 1
then we say that the set of vectors

{ pi [n]} i  1,2,3,4,...
is linearly dependent
AGC
 Examples:
 2  3 4
T
 1) p1
p 2   1 6  2
T
p 3  1 6 2
T
Observe that
4p1  5p 2  3p 3  0
ie the set is linearly dependent
AGC
 Examples
 2) p1 t
p2 1  t
the set is linearly independent
AGC
AGC
DSP The Span of Vectors
 The Span of a vector set is the set of all possible
vectors that can be reached (ie made through linear
combinations) from the given set.
 That is there exists a set {ci } i  1,2,3,4,...

such that
m
x[n]   ci pi [n]
i 1
AGC
DSP The Span of Vectors
 Example: The vectors below are in 3-D real vector
space.
p1  1 1 0
T
p 2  0 1 0
T
 Their linear combination forms
x  c1 c1  c2 0
T
which is essentially a vector in the plane of the given

two vectors.
AGC
DSP Basis and Dimension
 If { pi [ n]} i  1,2,3,4,... is a selection of
linearly independent vectors from a vector space
such that they span the entire space then we say the
selected vectors form a (complete) basis.
 The number of elements in the basis is the cardinality
of the basis
 The least number of independent vectors to span a
given vector space S is called the dimension of the
vector space, usually designated as
dim(S )
AGC
DSP IMPORTANT!
 Every vector space has a basis.
 Thus for many purposes whatever

operations we want to do in the vector
space can be done with the basis.
AGC
DSP
Basis
AGC
DSP Vector Spaces
 Let us start with the intuitive notion that we can
represent a signal as

x[n]   ci pi [n]
i 0
 This representation is called a projection of x[n],the
signal, into the linear vector space
{ pi [n]} i  1,2,3,4,...
 The vectors above are linearly independent and can
span any signal in the space
AGC
DSP Vector Spaces
 Examples are seen in Matrix Theory and typically in
Fourier Analysis.
 The length of a vector is known as the norm.

 We can select any convenient norm, but for
mathematical tractability and convenience we select
the second order norm.
 A real valued function x is the norm of x
AGC
DSP Norm
 A real valued function x is the norm of x when
it satisfies
 Positivity x 0
 Zero length x 0 if x0
 Scaling x   x

Triangle inequality xy  x  y
AGC
Induced norm/Cauchy
DSP Schwartz inequality
 Induced norm of the space follows from the inner

product as
2
x, x  x
 The l2 norm is represented as x 2
 The following condition (Cauchy-Schwartz) is
satisfied by an induced norm (eg l2 )
x, y  x 2 y 2
AGC
DSP Inner Product
 The inner product of two vectors in a scalar, and has
the following properties
*
 x, y  x, y
 if the vectors are real then x, y  y , x
  x, y   y , x
AGC
DSP Inner Product

x  y , z  x, z  y , z

x, x  0 x  0
 x, x  0 x  0
 In finite-dimensional real space
x, y  xT y   xi yi
AGC
DSP Direction of Vectors

From the two and three dimensional cases we define
the angle  between two vectors to be given from
x, y
cos 
x2y2
 If y  ax the vectors are colinear
 If x, y  0 the vectors are orthogonal x y

(the zero vector is orthogonal to all vectors)
AGC
DSP Orthonormal
 A set of vectors [p1 , p 2 , p 3 ,...p m ] is
orthonormal when
p i , p j   ij
 (Pythagoras)
If x  y then the indiced norms satisfy
2 2 2
xy  x  y
AGC
DSP Weighted inner product
 Very often we want a weighted inner product which
we define as
H
x, y W
 y Wx
where W is a Hermitian matrix

For the induced norm to be positive for yx we
must have for all non-zero vectors
H
x, x W  x Wx  0
This means that W must be positive definite
AGC
DSP Example
 Let
1  2
x1    x2   
1 1 
 Clearly  2
xT1 x 2  1 1   3  0
 while
1 
 2  2  2  2  2
 2 2  x 2  1 1 2 2  1   0
T
x1
    
AGC
DSP Example
 Note that the inner product in the previous example
cannot serve as a norm as for any
a 
x 
a 
we have T
x Wx  0
 This violates the conditions for a norm
AGC
DSP Complete spaces
 If every signal in a signal space is reachable (ie can
be spanned) by a set of vectors then the space is
complete and the reaching a complete set
 This means that there will be no left over part of a
given signal expressed as an appropriate linear
combination of basis vectors
 For example a Fourier series reaches all periodic
signals in the least square sense, and hence the set
of all complex exponentials is a complete set
AGC
DSP Hilbert spaces
 Complete linear vector spaces with induced l2
norms are known as Hilbert Spaces
 In signal processing we are mainly interested in finite
energy signals ie in Hilbert spaces
 If the norm above is orther than the second then we
have Banach Spaces. (Such norms are useful in
approximating signals and system functions as we
shall see later)
AGC
DSP Orthogonal subspaces
 Let S be a inner product signal (vector) space and V
and W be subspaces of S.
 Then V and W are orthogonal if for every v  V
and w  W we have v, w  0
 In above the set of all vectors orthogonal to a
subspace V is called the orthogonal complement
of the of the subspace denoted by V 
AGC
DSP Inner-sum spaces
and W be subspaces of S with v  V and w  W
 Then V +W is the inner sum space that consists of
all vectors x  w  v
 Example: Let S be the set of all 3-tuples in GF(2)

and
v  1 0 1 S w  0 0 1 S
Then
x  w  v  1 0 0 S
AGC
DSP Example …
 Let A  span0 1 0
B  span1 0 0
 Then
A  0 0 0, 0 1 0
B  0 0 0, 1 0 0
 Note that since vectors in one of these are pairwise
orthogonal to the vectors in the other the two
subspaces are are orthogonal
AGC
DSP Example …
 The orthogonal complement of A is found by
observing the the second member of the set is
orthogonal to all vectors having as their first entry a
zero. Thus
A  0 0 0, 0 1 0, 0 0 1, 0 1 1


 Thus observe that B  A
 And the pair-wise sum
A  B  0 0 0, 0 1 0, 1 0 0, 1 1 0
AGC
DSP Disjoint spaces
 If two linear vector spaces of the same dimensionality
have only the zero vector in common they are called
disjoint.
 Two disjoint spaces are such that one is the algebraic
complement of the other
 Their sum is the entire vector space
AGC
DSP Disjoint spaces …
and W be subspaces of S
 Then for every x  W  V there exist unique

vectors
v V w W
xwv
such that
if and only if the two sbspaces are disjoint.
(ie if they are not disjoint a vector may be generated
form a pair-wise sum of more than one pair)
AGC
DSP Projections
 From above a pictorial representation can be
produced as
V
v x
W
w
S
AGC
DSP Projections
and W be subspaces of S
 We can think of v  V and w  W as being the
projections of x  w  v in the component sets.
P : S V
We introduce the projection operator
x  S we have

such that for any

Px  v
x
 That is the operation returns that component of
that lies in V
AGC
DSP Projections
 Thus if x is already in V the operation does not
change the value of x
 Thus
P(Px)  Px
 This gives us the definition
A linear tranformation is a projection if
P
P2  P
(Idempotent operator)

Signal Spaces: AGC DSP

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Signal Spaces: AGC DSP

Uploaded by

Copyright:

Available Formats

AGC

DSP Signal Spaces

 This is known as the l1 metric or the Manhattan

This is called the l metric or distance.

 There are many metric spaces as seen from the

 (In the case of continuous time signals we say we

 Where { pi [ n]} i  1,2,3,4,... are the simpler

then we say that the set of vectors

ie the set is linearly dependent

the set is linearly independent

 That is there exists a set {ci } i  1,2,3,4,...

 Their linear combination forms

which is essentially a vector in the plane of the given

 Every vector space has a basis.

 Thus for many purposes whatever

 The length of a vector is known as the norm.

 Zero length x 0 if x0

 Induced norm of the space follows from the inner

 if the vectors are real then x, y  y , x

 In finite-dimensional real space

 If x, y  0 the vectors are orthogonal x y

where W is a Hermitian matrix

 Example: Let S be the set of all 3-tuples in GF(2)

 Then for every x  W  V there exist unique

such that for any

You might also like