Topic 6-Linear Systems and Power Spectra

(Chapters 9 and 10 in Papoulis)
1.5 weeks

•  Linear Filtering of a Random Signals
•  Power Spectrum Analysis

1

(Deterministic) Systems with Stochastic Inputs
A deterministic system1 transforms each input waveform X (t,ξ i ) into
an output waveform Y (t, ξ i ) = L[X(t, ξ i )] by operating only on the
time variable t. Thus a set of realizations at the input corresponding
to a process X(t) generates a new set of realizations {Y (t,ξ )} at the
output associated with a new process Y(t).
Y (t, ξ i )

X (t, ξ i )
X (t )
!!
!→

L[⋅]

(t )
!Y!→
!
t

t

Our goal is to study the output process statistics in terms of the input
process statistics and the system function.
1A stochastic

system on the other hand operates on both the variables t and ξ
PILLAI

2

Deterministic System (not necessarily linear)
Deterministic Systems
Memoryless Systems
Y (t ) = g[ X (t )]
Time-varying
systems

Systems with Memory
Time-Invariant
systems

Linear systems
Y (t ) = L[ X (t )]

Linear-Time Invariant
(LTI) systems

X (t )

h (t )
LTI system

+∞

Y (t ) = ∫ h(t − τ ) X (τ )dτ
−∞

(6-1)

+∞

= ∫ h(τ ) X (t − τ )dτ . = X (t ) ∗ h(t )
−∞

3

Memoryless Systems (not necessarily linear)
The output Y(t) in this case depends only on the
present value of the input X(t). i.e., Y (t ) = g{ X .(t )}
Strict-sense
stationary input

Memoryless
system

Strict-sense
stationary output.

Wide-sense
stationary input

Memoryless
system

Need not be
stationary in
any sense.

X(t) stationary
Gaussian

Memoryless
system
Note: the filters are not random

Y(t) stationary, but
not necessarily
Gaussian
4

LTI Systems: WSS input good enough
X (t )
wide-sense
stationary process

LTI system
h(t)

Y (t )
wide-sense
stationary process.

(a)

X (t )
strict-sense
stationary process

LTI system
h(t)
(b)

X (t )
Gaussian
process (also
stationary)

Linear system

Y (t )
strict-sense
stationary process
Y (t )
Gaussian process
(also stationary)

(c)
5

Linear Time-Invariant (LTI) Continuous Systems
Time-Invariant System
Shift in the input results in the same shift in the output.

Y (t ) = L{X (t )} ⇒ L{X (t − t0 )} = Y (t − t0 )

(6-2)

Linear Time-Invariant System: linear system with time-invariant property.
•  Linear system has two important properties: superposition and
homogeneity

L{a1 X (t1 ) + a2 X (t2 )} = a1 L{ X (t1 )} + a2 L{ X (t2 )}.

−∞

δ (t )dt = 1

δ (t )

h(t ) = L{δ (t )}
LTI

h(t )

h (t )

(6-3)

Impulse
response of
the system
t

Impulse

Impulse
response

6

Linear (LTI) Filtering of a Random Signal
Y (t )
X (t )
t

X (t )

t

Y (t )

LTI

+∞

Y (t ) = ∫ − ∞ h(t − τ ) X (τ )dτ

arbitrary
input

+∞

+∞

X (t ) = ∫ − ∞ X (τ )δ (t − τ )dτ

(6-4a)

(6-1)

= ∫ − ∞ h(τ ) X (t − τ )dτ

+∞

Y (t ) = L{ X (t )} = L{∫ − ∞ X (τ )δ (t − τ )dτ }

Note: the filter
is not random.

+∞

= ∫ − ∞ L{ X (τ )δ (t − τ )dτ }

By Linearity

+∞

= ∫ − ∞ X (τ ) L{δ (t − τ )}dτ
+∞

By Time-invariance
+∞

= ∫ − ∞ X (τ )h (t − τ )dτ = ∫ − ∞ h(τ ) X (t − τ )dτ .

(6-4b)
7

Mean of the Output
•  Expectation and linear filtering are both linear operations

that are interchangeable.
•  Not all linear operations are interchangeable (e.g., matrix
multiplication).
•  The mean of the output of a LTI filter is the convolution of
the mean of the input with the the filter impulse response:

$
E[Y (t)] = E & ∫ h(τ )X(t − τ )d τ ') =
% −∞
(
= E[X(t)]∗ h(t)


−∞

h(τ )E[X(t − τ )]d τ
(6-5)

•  Question: suppose the input is a deterministic function s(t)

plus a zero-mean random input X(t), what is the mean of
the filter output.
8

Filter Output With a Wide Sense Stationary (WSS) Input
•  If the input to an LTI filter with impulse response h(t) is a WSS

process X(t), the output Y(t) has the following properties:
•  Y(t) is a WSS process with expected value and autocorrelation

function
(6-6)
µY = E[Y (t )] = µ X ∫ h(τ )dτ
−∞

∫ h(u) ∫ h(v)R

RY (τ ) =

X

−∞

• 

(τ + v − u)dudv

(6-7)

−∞

X(t) and Y(t) are jointly WSS and have I/O cross-correlation

RXY (τ ) =


−∞

h(u)RX (τ − u)du = RX (τ ) ⊗ h(t)

(6-8)

•  The output autocorrelation is related to the I/O cross-correlation

by

RY (τ ) =


−∞

h(−w)RXY (τ − w)dw = RXY (τ ) ⊗ h(−τ ) (6-9)
9

Proof:

To show that
we recall that

RY (τ ) =

Y (t) =

∫ h(u) ∫ h(v)R

X

−∞

(τ + v − u)dudv

−∞

(6-10)

∫ X(t − u)h(u)du = ∫ h(t − u)X(u)du

So,

RY (τ ) = E[Y (t + τ )Y (t)] = E #$ ∫

∫ X(t + τ − u)X(t − v)(u)h(u)h)v)du dv%&

∫ ∫ E [ X(t + τ − u)X(t − v)]h(u)h)v)dudv
= ∫ ∫ R (τ + v − u)h(u)h)v)dudv
=

(6-11)

X

See if you can prove the other relationships on the previous chart
on your own !. If not, see Papoulis.
10

Linear Filtering of a Discrete Time Random Signal
•  If the input to a discrete-time LTI filter with impulse response hn is
a WSS random sequence, Xn, the output Yn has the following
properties.
•  Yn is a WSS random sequence with expected value and autocorrelation
function

µY = E[Yn ] = µ X
RY [n] =

∑ hn .

(6-12)

n = −∞

∑ ∑h h R
i

j

X

[n + i − j ].

(6-13)

i = −∞ j = −∞

•  Yn and Xn are jointly WSS with I/O cross-correlation

RXY [n] =

∑h R
i

X

[n − i ].

(6-14)

i = −∞

•  The output autocorrelation is related to the I/O cross-correlation by

RY [n] =

∑h

−i

i = −∞

RXY [n − i ].

(6-15)
11

Power Spectrum Analysis
Definition: Fourier Transform

Functions g (t ) and G ( f ) are a Fourier transform pair if

G ( f ) = ∫ g (t )e
−∞

− j 2πft

dt ,

g (t ) = ∫ G ( f )e j 2πft df

(6-16)

−∞

Definition: Power Spectral Density

The power spectral density function of the WSS stochastic
process X(t) is defined as
1 $
2&
SX ( f ) ≡ lim
E % XT ( f ) '
T→∞ 2T
(6-17)
2
&
1 $ T
− j 2 π ft
= lim
E * ∫ X(t)e
dt +.
−T
T→∞ 2T
%
'
12

Power Spectrum Density (PSD) and the Autocorrelation
Function are a Fourier Transform Pair

If X(t) is a WSS stochastic process, RX (τ ) and SX ( f ) are the
Fourier transform pair
SX ( f ) =


−∞

RX (τ )e

− j 2π f τ

dτ ,

RX (τ ) =


−∞

SX ( f )e j 2 π f τ df

(6-18)

We will use the symbol ℑ to denote the Fourier Transform
This is a major result, called the Wiener-Khinchin Theorem.
On the next charts we will prove this important result.

13

Proof that the PSD and Autocorrelation are Fourier Pairs
For a deterministic signal x(t), the spectrum is well defined: If X (ω )
represents its Fourier transform, i.e., if
+∞

(6-19)

X (ω ) = ∫ −∞ x(t )e− jωt dt ,

then | X (ω ) |2 represents its energy spectrum. This follows from
Parseval s theorem since the signal energy is given by
+∞

∫ −∞

x (t )dt = 21π
2

+∞

2
|
X
(
ω
)
|
dω = E.
∫ −∞

(6-20)

Thus | X (ω ) |2 Δω represents the signal energy in the band (ω , ω + Δω )
(see Figure below).
2
| X (ω )|

X (t )

0

t

0

Energy in (ω ,ω +Δω )

ω ω + Δω

ω

PILLAI

14

However for stochastic processes, the Fourier relationship
generates a sequence of random variables for every ω. Moreover,
for a stochastic process, E{| X(t) |2} represents the ensemble average
power (instantaneous energy) at the instant t.
To obtain the spectral distribution of power versus frequency for
stochastic processes, it is best to avoid infinite intervals to begin with,
and start with a finite interval (– T, T ) so that the Fourier Transform
of a process X(t) based over the interval (– T, T ) is given by
T

(6-21)
X T (ω ) = ∫ −T X (t )e− jωt dt
so that the energy per unit time, or the power spectral density (PSD)
2
(6-22)
| X T (ω ) |2 1 T
− jω t
=
X (t )e dt

−T
2T
2T
represents the power distribution associated with that realization based
on (– T, T ). Notice that (6-22) represents a random variable for every
ω and its ensemble average gives, the average power distribution
based on (– T, T ). Thus the power spectral density is given by
15

# | X T (ω ) |2 $ 1 T T
− jω ( t1 − t2 )
*
PT (ω ) = E %
=
E
{
X
(
t
)
X
(
t
)}
e
dt1dt2
&
1
2

− T ∫− T
' 2T
( 2T
1 T T
− jω ( t1 − t2 )
=
R
(
t
,
t
)
e
dt1dt2
(6-23)

− T ∫− T XX 1 2
2T
represents the power distribution of X(t) based on (– T, T ). For wide
sense stationary (WSS) processes, it is possible to further simplify
(6-23). Thus if X(t) is assumed to be WSS, then RXX (t1 , t2 ) = RXX (t1 − t2 )
and (6-23) simplifies to

1
PT (ω ) =
2T

T

T

− jω ( t1 − t2 )
R
(
t

t
)
e
dt1dt2 .
∫ −T ∫ −T XX 1 2

(6-24)

Let τ = t1 − t2 and using this change of variables we have
1 2T
− jωτ
PT (ω ) =
R
(
τ
)
e
(2T − | τ |)dτ

− 2 T XX
2T
2T

= ∫ − 2T RXX (τ )e − jωτ (1 − 2|τT| )dτ ≥ 0
to be the power distribution of the WSS. process X(t) over
(– T, T ). Finally letting T → ∞ in (6-25), we obtain

(6-25)
PILLAI

16

+∞

S XX (ω ) = lim PT (ω ) = ∫−∞ RXX (τ )e− jωτ dτ ≥ 0
T →∞

(6-26)

to be the power spectral density of the WSS process X(t). Notice that

​"↓$$ (')!!↔!!!!!!!!!!*.+!!!!!!!!┴​!!-↓$$ (.)≥0 (6-27)
i.e., the autocorrelation function and the power spectrum of a WSS
Process form a Fourier transform pair, a relation known as the
Wiener-Khinchin Theorem. From (6-27), the inverse formula gives

RXX (τ ) = 21π

+∞

jωτ
S
(
ω
)
e

∫−∞ XX

(6-28)

and in particular for τ = 0, we an expression for the total power
1

+∞

∫ −∞ S

2
(
ω
)
d
ω
=
R
(0)
=
E{|
X
(t)
|
}= P
XX
XX

(6-29)

From (6-29), the area under S XX (ω ) represents the total power of the
process X(t), and hence S XX (ω ) truly represents the power
spectrum. (See the figure on the next chart).
PILLAI

17

S XX (ω )

S XX (ω ) Δω represents the power
in the band (ω , ω + Δω )

ω

ω ω + Δω

0

The nonnegative-definiteness property of the autocorrelation function
translates into the nonnegative property for its Fourier
transform (power spectrum), since from (6-28)
n

n

n

n

∑∑ ai a RXX (ti − t j ) = ∑∑ ai a 21π
i =1 j =1

*
j

And it follows that

i =1 j =1

= 21π

*
j

+∞

∫−∞ S

+∞

∫−∞ S XX (ω )e
n

XX

(ω ) ∑i =1 ai e

jω ti

jω ( ti − t j )

2

d ω ≥ 0.

RXX (τ ) nonnegative - definite ⇔ S XX (ω ) ≥ 0.

(6-30)
(6-31)
PILLAI

18

Since X(t) is a real WSS process, then RXX (τ ) = RXX (−τ ) so that
+∞
S XX (ω ) = ∫ −∞ RXX (τ )e − jωτ dτ
+∞

= ∫ −∞ RXX (τ ) cos ωτ dτ

= 2 ∫ 0 RXX (τ ) cos ωτ dτ = S XX ( −ω ) ≥ 0

(6-32)

so that the power spectrum is an even function, (in addition to being
real and nonnegative).

PILLAI

19

Power Spectral Density Summary
•  For a WSS stochastic process, X(t), the Power Spectral Density

(PSD), Sx(f), and the Autocorrelation function, RX(τ) are a Fourier
Transform pair.
•  This is celebrated formula and is known as the Wiener-Khinchin
Theorem
•  The PSD, Sx(f), is a real-valued function with the following
properties:

(a) SX ( f ) ≥ 0 for all f

2
"
S
(
f
)df
=
E
X
∫ −∞ X
# (t)$% = RX (0)
(c) SX (− f ) = SX ( f )

(b)

(6-33)

20

Noise in Communication Systems
• 
• 

Thermal noise is described by a zero-mean Gaussian random
process, n(t).
If the PSD is constant, then it is called white noise..

[w/Hz]

Power spectral
density

Autocorrelation
function
Probability density function
21

White Noise Process and LTI Systems
•  W(t) is said to be wide-sense stationary (w.s.s) white noise if
E[W(t)] = constant, and

RWW (t1 , t2 ) = qδ (t1 − t2 ) = qδ (τ ).
•  The Power Spectral Density of white noise is “flat” as a function of
frequency (where W"∞)
[w/Hz]

•  If W(t) is also a Gaussian process, then all of its samples are
independent RVs
Colored noise
LTI
White noise
N (t ) = h (t ) ∗W (t )
h(t)
W(t)
22

Power Spectral Density of a Discrete Time Random Sequence
•  The discrete-time Fourier Transform (DTFT) is defined as

X(ϕ ) =

∑ xn e

− j 2 πϕ n

,

xn =

n=−∞

1/2
−1/2

X(ϕ )e j 2 πϕ n dϕ (6-34)

•  The power spectral density function of the WSS random sequence Xn is
2(
% L
1
SX (ϕ ) = lim
E ' ∑ X n e− j 2 πϕ n * .
L→∞ 2L +1 '
*)
& n=−L

(6-35)

•  The discrete-time Wiener Khinchin formula is

If X n is a WSS stochastic process, RX [k ] and S X (φ ) are
a discrete - time Fourier transform pair :
S X (φ ) =

∑ RX [k ]e

k = −∞

− j 2πφk

,

1/ 2

RX [k ] = ∫

−1 / 2

S X (φ )e j 2πφk dφ

(6-36)
23

Properties of the PSD for a Discrete Time WSS RP
For a WSS random sequence X n , the power spectral density S X (φ ) has
the following properties :
(a) S X (φ ) ≥ 0 for all φ ,
1/ 2

(b) ∫

-1 / 2

[ ]
2

S X (φ )dφ = E X n = RX [0] .

(6-37)

(c) S X (−φ ) = S X (φ ),
(d) for any integer n, S X (φ + n) = S X (φ ).

24

Power Spectra and Linear Systems
If a WSS process X(t) with autocorrelation
h(t)
function RXX (τ ) ↔ S XX ( f ) ≥ 0 is
X(t)
Y(t)
applied to a linear system with impulse
response h(t), then the cross correlation
function RXY (τ ) and the output autocorrelation function RYY (τ ) are
given below:

RXY (τ ) = RXX (τ ) ∗ h* (−τ ), RYY (τ ) = RXX (τ ) ∗ h* ( −τ ) ∗ h(τ ). (6-38)
But if

f (t ) ↔ F (ω ),

g (t ) ↔ G(ω )

(6-39)

Then
since

f (t ) ∗ g (t ) ↔ F (ω )G(ω )
ℑ{ f (t) ∗ g(t)} =

+∞

∫ −∞

f (t) ∗ g(t)e − jωt dt

(6-40)
(6-41)
PILLAI

25

+∞

ℑ{ f (t) ∗ g(t)}= ∫ −∞

{∫

+∞

= ∫ −∞

+∞
−∞

}

f (τ )g(t − τ )d τ e − jωt dt

(6-42)
f (τ )e − jωτ d τ ∫ −∞ g(t − τ )e − jω (t−τ ) d (t − τ )=F (ω )G(ω ).
+∞

Using the above in (6-38), we get

S XY (ω ) = ℑ{RXX (ω ) ∗ h* (−τ )} = S XX (ω )H * (ω )

(6-43)

and since
+∞

*

∫ −∞ h (−τ )e
where

− jωτ

dτ =
+∞

(

+∞

∫ −∞ h(t )e

− jω t

dt

*

)

= H * (ω ),

H (ω ) = ∫ −∞ h(t )e− jωt dt

(6-44)
(6-45)

represents the transfer function of the system, and we find

SYY (ω ) = ℑ{RYY (τ )} = S XY (ω )H (ω ) = S XX (ω ) | H (ω ) |2 .

(6-46)

The above (6-46) is very useful: the PSD of the output signal
is the product of the PSD of the input and the magnitude
squared of the transfer function.
PILLAI

26

The output power spectrum is real and nonnegative and is
related to the input spectrum and the system transfer function as in
(6-46), which can be used for system identification as well.
WSS White Noise Process: If W(t) is a WSS white noise process,
then
RWW (τ ) = qδ (τ ) ⇒ SWW (ω ) = q.
(6-47)
Thus the spectrum of a white noise process is flat, thus justifying its
name. Notice that a white noise process is unrealizable since its total
power is arbitrarily large (" ∞).
From (6-46), if the input to an unknown system is
a white noise process, then the output spectrum is given by

SYY (ω ) = q | H (ω ) |2

(6-48)

Notice that the output spectrum captures the system transfer function
characteristics entirely, and for rational systems (6-48) may be
used to determine the pole/zero locations of the underlying system.

PILLAI

27

The Power Spectral Density of the Output Signal

SX(f)

H(f)

SY(f) =|H(f)|2Sx(f)

Frequency Domain
The above relationship is very useful in system design and analysis.

28

Example: A WSS white noise process W(t) is passed
through a low pass filter (LPF) with bandwidth B/2. Find the
autocorrelation function of the output process.
Solution: Let X(t) represent the output of the LPF. Then from (6-48)
#q, | ω |≤ B / 2
2
S XX (ω ) = q | H (ω ) | = $
.
(6-49)
%0, | ω |> B / 2
The inverse transform gives the output autocorrelation function
B/2

B/2

RXX (τ ) = ∫ − B / 2 S XX (ω )e jωτ d ω = q ∫− B / 2 e jωτ d ω
sin( Bτ / 2)
= qB
= qB sinc( Bτ / 2)
( Bτ / 2)
R (τ )

(6-50)

XX

| H (ω )|2

qB

1

−B / 2

(a) LPF

B/2 ω

τ

(b) Output autocorrelation

PILLAI

29

Example: Let

Y (t ) =

1

t +T


2T t −T

X (τ )dτ

(6-51)

represent a smoothing operation using a moving window on the input
process X(t). Find the spectrum of the output Y(t) in term of that of X(t).
Solution: Consider a LTI system with impulse
response h(t) as in the figure to the right ,
then using h(t), (6-51) reduces to
+∞

Y (t ) = ∫ −∞ h(t − τ ) X (τ )dτ = h(t ) ∗ X (t )
so that
where

SYY (ω ) = S XX (ω ) | H (ω ) |2 .
+T

H (ω ) = ∫ −T

1
2T

e− jωt dt = sinc(ωT )

h (t )
1 / 2T
−T

T

t

(6-52)
(6-53)
(6-54)
PILLAI

30

so that

SYY (ω ) = S XX (ω ) sinc 2 (ω T ).

(6-55)

sinc2 (ω T )

S XX ( ω )

ω

π

SYY (ω )

ω

ω

T

Notice that the effect of the smoothing operation in (6-51) is to
suppress the high frequency components in the input (beyond π / T ),
and the equivalent linear system acts as a low-pass filter (continuoustime moving average) with bandwidth 2π /T in this case.
By the way, you could also determine the power spectral density by
finding the correlation function of Y(t) directly.
PILLAI

31

Discrete – Time Processes
For discrete-time w.s.s stochastic processes X(nT) with
autocorrelation sequence {rk }+∞
(proceeding as above) or formally
−∞ ,
defining a continuous time process X (t ) = ∑ n X (nT )δ (t − nT ), we get
the corresponding autocorrelation function to be

RXX (τ ) =

+∞

∑ rkδ (τ − kT ).

k =−∞

Its Fourier transform is given by

S XX (ω ) =

+∞

k =−∞

rk e − jωT ≥ 0,

(6-56)

and it defines the power spectrum of the discrete-time process X(nT).
From (6-56),

S XX (ω ) = S XX (ω + 2π / T )
so that S XX (ω ) is a periodic function with period

2B =
.
T

(6-57)

(6-58)

PILLAI

32

This gives the inverse relation
1 B
jkωT
rk =
S
(
ω
)
e

(6-59)

− B XX
2B
and
1 B
2
(6-60)
r0 = E{| X (nT ) | } =
S (ω )d ω

− B XX
2B
represents the total power of the discrete-time process X(nT). The
input-output relations for discrete-time system h(nT) translates into

and
where

S XY (ω ) = S XX (ω ) H * (e jω )

(6-61)

SYY (ω ) = S XX (ω ) | H (e jω ) |2

(6-62)

H (e ) =

+∞

n =−∞

h( nT ) e − jω nT

represents the discrete-time system transfer function.

(6-63)
PILLAI

33

Example: Amplitude Modulated (AM) Signal Noise Analysis
Consider the noisy AM signal

X (t ) = m(t ) cos(ω0t + θ ) + n(t ),

(6-64)

Question: What is the Power Spectral Density of the AM signal?
Solution: In this case from (6-64), if we assume θ ~ U (0, 2π ),
then it can be shown that
1
RXX (τ ) = Rmm (τ ) cos ω 0τ + Rnn (τ )
2

so,

S XX (ω ) =

S XX (ω − ω 0 ) + S XX (ω + ω 0 )
+ S nn (ω ).
2

(a)

(6-66)

S XX (ω )

Smm (ω )

0

(6-65)

ω

−ω 0

(b)

S mm (ω − ω 0 )

ω0

ω
34

Summary: Signal Transmission with Linear Systems

Input

Output
Linear system

• 
• 

Deterministic signals:
Random signals:

(6-66)

35