Professional Documents
Culture Documents
2004
1 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1. STATIONARY RANDOM PROCESS
(2) The random process is called as the K-variate random process (multi-variate
random process)
Time shift
xk(t)
x1(t) t
t
t t+
(3) For discrete sample function, discrete values of any sample random function are
measured at certain time points t1, t2, t3, … tN (N: number of sampling values of
sample function)
2 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
xk (t ) xk (t1 ), xk (t 2 ), xk (t3 )}...xk (t N )} : Discrete sample function
Non-Gaussian process
Non-ergodic signals
Non-stationary
processes
y k (t )
(1) Mean value (Expectation): First-order statistical moment
1
x (t ) E[ x k (t )] Lim N x k (t ) dt
N
3 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1
y (t ) E[ y k (t )] Lim N y k (t )dt
N
2 x (t ) E[( xk (t ) x (t )) 2 ] E[ x 2 k (t )] k2 (t ) : Variance
2 y (t ) E[( y k (t ) y (t )) 2 ] E[ y 2 k (t )] y2 (t ) : Variance
2 x (t ) E[( xk (t )) 2 ] : Variance
2 y (t ) E[( y k (t )) 2 ] : Variance
C xx (0) 2 x (t )
C yy (0) 2 y (t )
C xx (t ) E[ x 2 k (t )] : Mean square
C yy (t ) E[ y 2 k (t )] : Mean square
4 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Variance 2 x (t ) = Mean Square Cxx(t)
Rxx ( ) E[ xk (t ) * xk (t )] : Autocorrelation
R yy ( ) E[ y k (t ) * y k (t )] : Autocorrelation
Rxy ( ) E[ xk (t ) * y k (t )] : Cross-correlation
Note: C xx ( ) Rxx ( ) 2 x
C yy ( ) R yy ( ) 2 y
C xy ( ) Rxy ( ) x y
Zero mean random process: x (t ) 0 , y (t ) 0
C xx ( ) Rxx ( )
C yy ( ) R yy ( )
C xy ( ) Rxy ( )
C xx ( ) Rxx ( ) x2
xx ( ) : Auto-correlation coefficient
Rxx ( ) Rxx ( )
C yy ( ) R yy ( ) y2
yy ( ) : Auto-correlation coefficient
R yy ( ) R yy ( )
C xy ( ) Rxy ( ) x y
xy ( ) : Cross-correlation coefficient
Rxy ( ) Rxy ( )
5 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
C xx ( ) C xx ( )
xx ( )
Rxx ( ) 2x
C yy ( ) C yy ( )
yy ( )
R yy ( ) 2y
C xy ( ) C xy ( )
xy ( )
Rxy ( ) x y
Note 1:
i) C xx ( ) R xx ( )
ii) C xx ( ) C xx (0)
v) | C xy ( ) |2 C xx (0) * C yy (0)
vi) | C xy ( ) |2 x2 y2
Rxy ( ) R yx ( )
Note 2: 0
i) C xx (0) x2 : Variance
6 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
ii) C yy (0) y2 : Variance
S yy ( f ) R yy ( ) * e j 2f d : Auto-spectral density function
S xy ( f ) Rxy ( ) * e j 2f d : Cross-spectral density
function
Inverse Fourier Transform:
Rxx ( ) S xx ( f ) * e j 2f df : Auto-correlation
R yy ( ) S yy ( f ) * e j 2f df : Auto-correlation
Rxy ( ) S xy ( f ) * e j 2f df : Cross-correlation
7 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Sxx(f), Syy(f), Sxy(f): Two-sided spectra, f[-,]
Rxx ( ) 2 S xx ( f ) * cos 2f d
0
Changing the two-sided spectral density Sxx(f) with f[-,] to the one-sided spectral
density Gxx(f) with f[0,]
Gxx ( f ) 2 S xx ( f )
G yy ( f ) 2 S yy ( f )
Gxy ( f ) 2 S xy ( f )
Gxx(f)=2Sxx(f): One-sided
Spectra
Sxx(f): Two-sided
[-,0] [0,]
f(Hz)
Thus,
Gxx ( f ) 4 Rxx ( ) * cos 2f d ; f[0,]
0
Rxx ( ) Gxx ( f ) * cos 2f df ; f[0,]
0
8 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Gxy ( f ) 2 Rxy ( ) * e j 2f d C xy ( f ) iQxy ( f )
C xy ( f ) : Co-spectrum
Qxy ( f )
xy ( f ) tan 1
C xy ( f )
C xy ( f ) | Gxy ( f ) | cos xy ( f )
Rxy ( ) [C xy ( f ) cos 2f iQxy ( f ) sin 2f ]df
0
Rxy (0) C xy ( f ) df
0
xk ( f ) X k ( f )e j 2ft df
0
9 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Spectral density function:
1
Gxx ( f ) 2 LimT E[| X k ( f ) |2 ]
T
1
G yy ( f ) 2 LimT E[| Yk ( f ) |2 ]
T
1
Gxy ( f ) 2 LimT E[ X k ( f ) * Yk ( f )]
T
1.5. Coherence
Coherence plays the same role as the correlation coefficient. The correlation coefficient
is expressed in time domain, whereas the coherence in frequency domain.
| E[ x(t ) * y (t )] |2 E[ x 2 (t )] * E[ y 2 (t )]
| C xy ( ) |2 C xx (0) * C yy (0) 2 x * 2 y
Thus,
C xy ( )
xy ( ) : Correlation coefficient
x * y
0 xy ( ) 1
10 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
xy ( ) 1 : x(t), y(t) Correlated;
(2) Coherence:
| Gxy ( f ) |2 Gxx ( f ) * G yy ( f )
2 | Gxy ( f ) |2
xy ( f )
Gxx ( f ) * G yy ( f )
2
0 xy ( f ) 1
2
(3) Role of coherence function xy ( f ) (constant-parameter linear systems) can be
interpreted as fractional portion of the mean square value at the output y(t) that is
contributed by the input x(t) at frequency value f. In contrast, the quantity
2
[1 xy ( f )] is the portion of mean square value of output y(t) not be contributed by
input x(t) at frequency f.
2
Note: The role of coherence function xy ( f ) is similar to the correlation
11 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
c. SIMO or MISO
2
C xy ( f ) 2 iQxy ( f )
xy (f)
Gxx ( f ) * G yy ( f )
2 C xy ( f ) 2
xy ( f ) : Coherence
Gxx ( f ) * G yy ( f )
Gxx ( f ), G yy ( f ) : Auto-spectra
12 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
2. DIGITAL SIGNALS AND CLASSIFICATION
2.2. Classifications
(1) Signals can be commonly classified in the engineering application by some follows
categories: i) Continuous (analogue) and discrete signals (digital), ii) Deterministic
and random signals
Discrete Signals
(Digital)
Transient signals
13 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Random signals Stationary signals Ergodic signals
Non-ergodic signals
2
x(t ) X 0 cos( t 0 ) (Original signals)
T
a0 1 1
x (t ) [an cos 2n t bn sin 2n t ] (Fourier series)
2 n 1 T T
iii) Almost-periodic signals: can be expressed by the sum of sine functions that
their frequencies are not periodic.
iv) Transient signals: can be defined as totally non-periodic signals (In other
word, the transient signals can be considered as the deterministic signals but
out of any kinds of periodic and almost-periodic signals). Apart from
periodic and almost-periodic signals, however, the spectrum of transient
data only exists under form of continuous spectrum but the discrete spectrum
does not exist.
14 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
(2) The characteristics of the random signals can be clarified by some following
quantities:
i) Amplitude distribution: Mean value (expectation) or root mean square
(standard deviation) [first moment]
ii) Correlation functions (auto- and cross-correlations) [joint or second moment]
iii) Power spectral density (PSD) [power contribution of each frequency
components]
(3) For discrete signals, the above-mentioned quantities can be expressed by formulas:
N N
1 1 2
x
N
i 1
xi ; rms x
N
x
i 1
i
N
1
R xx ( )
N
x (t
i 1
i 0 ) x i (t 0 )
N
1 j 2nk / N
Sx ( f )
N
x e
k 1
i ; n=[1,N]
x (t ) x
R xx (t , ) R xx ( )
iii) Weakly stationary signals (or stationary in the wide sense) are that mean
value and correlation (first and joint moments) are time invariant.
15 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
x (t ) x
R xx (t , ) R xx ( )
iv) Strongly stationary signals (or stationary in the strict sense) are that all
moments and joint moments (not only first moment but also high-order
moments) are time invariant.
x (t ) x
(first moment and joint moment)
R xx (t , ) R xx ( )
th
i x (t ) x
i th (ith moment and ith joint moment)
R xx (t , ) R xx ( )
v) Ergodic signal is stationary one (Mean value and correlation are time
invariant), moreover, its mean value and correlation are constant with
different samples of signal.
x (k ) x
(k )
Rxx ( ) Rxx ( )
k: Index of kth sample of signal
vi) Non-ergodic signal is stationary one (Mean value and correlation are time
invariant), however, its mean value and correlation are differed with
different samples of signal.
x (k ) x
(k )
R xx ( ) R xx ( )
vii) Gaussian signal (Normal distributed signal) is ergodic stationary one with
zero-mean and standard deviation
16 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
x (k ) 0
(k ) (k )
Rxx ( ) C xx ( ) 2
viii) Non-gaussian signal is ergodic stationary one with non zero-mean and
standard deviation
x (t ) x 0
C xx ( ) 2
Note 1:
Single random process: Signal of one physical quantity (phenomenon) at one
position
Multi-random processes: Signals of many physical quantities at different
positions
Sampling: Signal of random process at any time ( time interval)
Multi-dimensional process: are multi-variable function
Multi-variate process: Vector of many signals
Multi-variate and multi-dimensional process: Vector of many processes (signals)
in which each individual signal is multi-variable function
Ensemble: the collection of sample functions (any time interval) of one signal.
Thus random signal is the collection of these sample function of one signal
Note 2:
Process (Field): Display and illustration of one physical quantity at one position
(If measurement of the same physical quantity at one position differs from that
at another position due to its distribution and redistribution)
Sample: Display and illustration of a physical quantity at one position at any
time interval
17 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Record: Discrete display of a physical quantity at one position and one time
Signal: Electric display (continuous or discrete) of a physical quantity at one
position and one time
18 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
(5) The ensemble (collection of sampling records) of random signal can be divided by
two categories: Individual sample records (one process) and Multiple sample
records (many processes). The characteristics of individual and multiple sample
records of one and many processes can be expressed by figure hereinafter:
Coherence Function
Analysis
Frequency Response
Function (FRF)
Characteristics of individual and multiple sample records of random signals
Note 3:
Coherence function: is the relation between the power auto-spectral density and
the power cross-spectral density.
Frequency response function (gain factors and phase factors): is also the linear
relation between the power auto-spectral density and the power cross-spectral
density.
19 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
(6) Applications of statistical functions
From correlation functions
i) Similarity between 2 signals or 1 signal at different positions and time delays
ii) Prediction of signals in noise, influence of noise
iii) Identification of propagation directions and velocities
iv) Measurements of time delays
20 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Frequency Response Signal noise
Function (FRF) n(t)
Input signal Output signal
x(t) Hxy(f) y(t)
SISO system
Signal noise
n(t)
Input signal Output signal
x1(t) Hxy,1(f) y1(t)
Channel No.1
……………………..
Input signal Output signal Channel No.k
xk(t) yk(t)
Hxy,k (f)
21 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
3. DATA ACQUISITION, PROCESSING AND ANALYSIS
3.1. Introduction
(1) In order to understand and clarify the physical measurements, signal processing and
analysis for the buffeting experiments and other specified measurements as well in
the wind tunnel, it is important to understand the digital signal processing (DSP)
and measurement procedures and instruments in wind tunnels.
(2) This study hinges on some following points:
1) Instrumental systems their by function: data acquisition, A/D conversion, data
qualification and analysis
2) A/D conversion and sampling theorem for eliminating the aliasing errors
3) Data analysis techniques
(3) Three following main steps of the digital signal processing procedure will be
overviewed as follows:
22 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
3.2.1. Data acquisition
Transducer: Device and sensor to transform from any physical phenomena
(force, pressure and motions: displacement, velocity and acceleration) to
electric signals. Transducers commonly are employed two kinds of materials:
piezoelectric and strain-sensitive materials.
+) Piezoelectric materials: frm physical quantities to electric charge, such
as naturally polarized crystals like quartz and artificially polarized
ferroelectric ceramics like barium titanate.
+) Strain-sensitive materials: from physical quantities to resistance, such
as metallic like copper-nickel alloy and semiconductor like
monocrystalline silicon.
Signal conditioning: Change from the electric signals (charge and resistance)
to voltage.
Note: ADC can be stored under two types of codes: binary and ASCII codes
Binary code: By numbers 0 and 1 (8 bytes)
ASCII code: By numbers from 0 to 9 (1byte). However, almost data of
discrete signals (after ADC) has been stored under this ASCII code in
application, because it is easily red by any applied programs such as Matlab
and Fortran for data post-processing.
23 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
3.2.3. Data qualification
Many imperfections of received signals with noise can be reduced by the
digital or analogue filters.
Data qualification consists of: classification, validation and editing.
24 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Multiple sample record and multi-variate processes
Cross-correlation function computation
Cross-spectral density function computation
Coherence function computation
Multi sample record (of multi signals) = Collection of number of individual sample
records (of individual signal). Thus analysis of the multi sample record can be carried
out by analysis of the individual sample record and analysis of the pairs of correlated
two individual sample records.
Equally spaced time interval (T: sampling time or sampling period) of samples in
discrete signal:
Tn T0 n * T n=1,2,3… N
25 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
T0: Initial time (T0 = 0) Tn n * T
T: Sampling period
NT: Total time length of discrete signal; N *T
1
F: Sampling frequency (Hz); F
T
1
F0: Fundamental sampling frequency; F0
N *T
Noting that the Limit sampling frequency (the Nyquist frequency) for eliminating the
Aliasing Errors
1
FN
2 *T
26 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1 N 2
s x SQRT [ x n]
N 1 n 1
n=1,2,3…N (Root Mean Square
value)
Note: For the stationary ergodic data record (time-invariant and sampling-
independent), it is very convenient and common to transform the initial data
2 1 N 2
x' x' n
N 1 n 1
(Mean Square value or Variance)
1 N 2
s x ' x ' SQRT [ x' n ]
N 1 n 1
(Root Mean Square or Standard Deviation)
x'n n=1,2,3…N
Discrete Fourier Transform
N
1
X ( Fk )
[ x(n T ) * exp( j 2 F
n 1
k n T )]
27 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
X(Fk): DFT at computational frequency Fk
k: Index of discrete frequencies
Note:
1
f F0
NT
Fc Fs
K
F0
Fc : Cut-off frequency
Fs : Starting frequency
28 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N
k 1 k
X ( Fk ) X (
NT
)
[ x(n T ) * exp( j 2 NT n T )] ;k=0,2,3…K-1
n 1
N
k 1
X ( Fk ) X (
NT
)
[ x(n T ) * exp( j 2 k n / N )] ;k=0,2,3…K-1
n1
DFT
X(Fk)
X(F1)
X(Fi)
f(Hz)
F1=Fs Fi=Fs+i/NT F1=Fc
k=1 k=i k=K
N
1
k=1: X ( F1 ) X ( Fs )
[ x(n T ) * exp( j 2 n / N )]
n 1
1 1 N
k=2: X ( F2 ) X ( Fs ) [ x (n T ) * exp( j 2 2 n / N )]
NT n 1
N
2 1
k=3: X ( F3 ) X ( Fs
NT
)
[ x(n T ) * exp( j 2 3 n / N )]
n 1
29 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
4) Auto-correlation function
Auto-correlation function can be computed by i) Direct computation by definition and
ii) Indirect computation via FFT. Note that the second method is more efficiency
because of application of FFT algorithm, however, the first one is easier to compute
but more time consuming.
x'n n=1,2,3…N
s r * t
N r
1
R xx ( r * t )
N r
[ x (s) x
n 1
n nr ( s s )]
x'n n=1,2,3…N
30 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N 1 NT
1 2 21 2
Definition of mean power: S xx E[ x (t )]
N
n0
x n
NT x (t ) dt
0
x(t)
T 2T 3T 4T 5T 6T (N-1)T NT
N 1
1
S xx ( f ) | X n ( f ) |2 : Two-sided spectral density
NT n 0
1 N 1
X ( f ) x n * exp[ j 2nk / N ] ; k=0,1,2…N-1
N n 0
N 1
2
G xx ( f k ) 2S xx ( f k ) | X n ( f k ) | 2 : One-sided spectral density
NT n0
( x' 0)
31 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
x'n , n=1,2,3…N
3) Data Blocking: Taper data record by each data blocks using Window
functions
4) Fourier Transform: Compute the Fourier Transform at frequency fk,
k=1,2,3…N by using FFT technique
X ( f k ) , k=1,2,3…N
32 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
4. CORRELATION FUNCTION
4.1. Introduction
(1) The correction functions evaluate the statistical independence between the signals
(time-dependant) or stochastic processes (space-dependant). The cross correlation
function is a measure to tell us how much two processes or two signals are like each
other, whereas the auto correlation function tell us how much a process or a signal at
time t is like itself at time t+ (: time shift or time lag) or how much a process or a
signal at location (x,y,z) is like itself at another location (x+x,y+y,z+z). In addition,
to evaluate how much two processes or two signals are like each other, the correlation
coefficient function also is given.
For the number of samples is taken large enough, we have following approximations:
N
1
R xx (s)
N
[ x
n 1
(n) ( s) x( n s ) ( s s )] (Auto-correlation functions)
N
1
R xy (s )
N
[x
n 1
(n) ( s ) y ( n s ) ( s s )] (Cross-correlation
functions)
33 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
s: Spatial coordinates or time variable
N: Number of samples
s s s Span-wise
s
s
si
34 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N 1 N 1
1 1
xy ( s ) E[ x( n ) ]E[ y ( n s ) ] Lim N
N
[ x
n 0
( n) ( s )] Lim N
N
[ y
n 0
( n s ) ( s s ) ]
For the number of samples is taken large enough, we have following approximations:
N N
1 1
xx ( s )
N
n 1
[ x ( n ) ( s )]
N
[ x
n 1
( n s ) ( s s)] (Auto-covariance functions)
N N
1 1
xy (s )
N
n 1
[ x (n) ( s )]
N
[ y
n 1
( n s ) ( s s )] (Cross-covariance
functions)
s : Spatial interval or time shift
s: Spatial coordinates or time variable
N: Number of samples
If x, y are the zero-mean processes or signals, these mean that E[x(s)]=0 and E[y(s)]=0,
then the Root mean square (R.M.S) value must be replaced to the mean value (or
Expected value):
Auto-covariance function:
N N
1 2 1 2
xx ( s ) SQRT {
N
n 1
[ x ( n ) ( s )]} SQRT {
N
[ x
n 1
( n s ) ( s s )]}
Cross-covariance function:
N N
1 2 1 2
xy ( s ) SQRT {
N
n 1
[ x ( n ) ( s )]} SQRT {
N
[ y
n 1
( n s ) ( s s )]}
35 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
+) Time shift or spatial interval: s
Step 2: Sampling
+) Sampling signals x, y
Step 3: Computing covariance function
Step 4: Plotting xx, xy versus n [1,N]
4.5. Examples
Example 1: Correlation function
0.5 0.1
0 0
-0.5 -0.1
-1 -0.2
0 50 100 150 0 50 100 150
Number of sample Number of sample
Auto-correlation Rxx,Ryy by Xcorr(x) Cross-correlation Rxy by Xcorr(x,y)
0.6 0.2
0.4 0.1
0.2 0
0 -0.1
-0.2 -0.2
0 100 200 300 0 100 200 300
36 | Le Thai Hoa - Number
Notes onofStationary
sample Random Process and Digital
NumberSignal Processing
of sample
Example 2: Correlation of signal with noise
1.5
1.2
1
1
0.5
0.8
0
-0.5 0.6
-1
0.4
-1.5
0.2
-2
0
-2.5
-3 -0.2
0 1 2 3 0 0.2 0.4 0.6 0.8
Time (s) -3 Time (s)
x 10
37 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
5. FOURIER TRANSFORM
5.1. Introduction
(1) It is well known that the Fourier Series Transform and Fourier Spectral Transform
have been widely applied for almost kinds of natural and physical phenomena.
Applications and contributions of the Fourier Series and Fourier Transform concentrate
on the problem of the Digital Signal Processing (D.S.P) in the data processing and
analysis of measurements and the buffeting response prediction in which the spectral
representation cant be required.
(2) In summary, the Fourier Series and the Fourier Transform will be studied for such
purposes as follows:
1) Data processing and analysis of measurement processes in the Digital Signal
Processing (D.S.P)
2) Spectral representation in the buffeting response prediction
(3) Data measurements can be expressed under the continuous or discrete processes.
However, almost measured signals have been collected under the discrete signal for the
data post-processing and analysis. The discretization of measured signals is well
known as the sampling processes.
38 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
5.2. The Discrete Fourier Series
The Fourier series purposes to decompose any periodic or non-periodic signals into
combination of simple harmonic signals (sine and cosine functions). Post processing
and analysis on simple harmonic functions seem to be much easy than the original
signals. Mathematically, the Fourier series is known as the harmonic analysis.
N: Number of samples
T: Period (Time step of sampling) of a sample (s)
1
F: Frequency of a sample (Hz), F
T
NT: Fundamental period of series (s)
1
Fo: Fundamental frequency of series (Hz) Fo
NT
Discrete Fourier series is expressed as the sum of harmonic functions as follow:
a0 M 1
xn [a m cos( mn 0T ) bm sin( mn 0T )]
2 m 1
2 2
0 : Fundamental frequency of series (rad/s), 0
Fo NT
a0 M 1
xn [ a m cos( 2mn / N ) bm sin( 2mn / N )]
2 m1
39 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Discrete Fourier Coefficients:
N 1
2
am
N
x
n 0
n cos( 2mn / N ) ; 0 m M 1
N 1
2
bm
N
x
n 0
n sin( 2mn / N ) ; 1 m M 1
Computational procedure:
Step 1: Setting parameters
- Initial signal
- Number of samples N
- Number of series M
Step 2: Sampling
Step 3: Calculating Fourier Coefficients
- am, bm
Step 4: Simulating Fourier series and Plotting Simulated signal vs. number of
samples (n)
40 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Discrete Fourier Transform: (to frequency domain)
N
X ( k ) [ x ( nT ) e j k n T ]
n 1
N
1 j 2 f k n T
X ( fk )
2
[ x(nT ) e
n 1
]
Computational procedure:
Step 1: Setting parameters
- T (cycle sampling), N (numbers of samples)
- Freq. range: fs (starting freq.), fc (cut-off freq.)
- Number of freq. interval: K ( fc fs) / k
Step 2:
Loop 1: Frequency
For k=1 to K k k ( )
Loop 2: Sampling
N
For n=1 to N X (k ) x (nT )e jk ( ) nT
n 1
41 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
2 2
Putting 0 into formulas:
Fo NT
N
X (k 0 ) [ x ( nT )e jk0 nT ]
n 1
N 2
jk nT
X (k 2F0 ) [ x (nT )e NT
]
n1
N
X ( kF0 ) [ x( nT )e j 2kn / N ]
n 1
a0 K
h(t ) [ ak cos( k k t ) bk sin( kk t )]
2 k 1
2
ak Re{H ( k )}
N
2
bk Im{ H (k )}
N
Leakage Effect: Amplitudes will distribute on adjacent closed frequencies. This effects
occurs in cases that total sampling time NT does not coincide the integer multiple of
the sampling cycle T. To prevent the ‘leakage effect’ by Averaging Method or Window
Functions
42 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
5.4. The Fast Fourier Transform
It is very well-known that the Fast Fourier Transform (FFT) algorithm has been
powerfully used for solving the Discrete Fourier Transform. FFT is not a new
transform itself due to using the same DFT formula, however, its FFT algorithm is
much faster than the conventional DFT. In principle, the FFT algorithm eliminates the
component repetition to make the faster computation.
N 1
j 2kn / N
DFT formula: X ( kF0 ) [ x ( nT )e ]
n 0
N 1
X k x nW Nkn ; k=[0,N-1]; WN e j 2 / N
n 0
j
Note: e cos j sin = x + jy (Euler’s Formula)
e j 2 5 / 8 e j 2 7 / 8
2250 k=5 3150 k=7
2700 k=6
Complex plane (N=8) Complex plane
43 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Based on above-mentioned notes, we can easily obtain:
W Nm e ( j 2 / N ) m ; m[0,N-1]
44 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N
Number of components reduces to ( ) 2 < N 2 of original problem. Depending on N,
2
efficiency of FFT algorithm is different. For example, if N=2K then complex
N N 2
components in FFT is K log 2 N (in comparison of N components in DFT)
2 2
N
j 2 kn / N
X ( kF 0 ) [ x ( nT ) e
n 01
] ; k=[0:N-1]:
N 1
1 j 2 kn / N
x ( nT )
N
[ X ( kF )e
k 0
0 ] ; n=[0:N-1]: Inverse Transform
45 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
The sampling parameter: Sampling rate or sampling frequency Fs(Hz).
The sampling theorem may be stated as follows: ‘If continuous signals is sampled at a
sampling rate or sampling frequency that is twice higher than their highest frequency
component, then it is possible to recover and reconstruct the original signals from
samples ’
Fsampling 2 Fmax,signal
5.8. Examples
Example 1: D.F.S
1 1 1.2
0.9
0.8 1
0.8 Signal from
0.7 0.6 D.F.S 0.8
Nu m be r of s erie s M = 1
Nu m be r of s erie s M = 3
N u m be r of s erie s M = 5
0.6
0.4 0.6
0.5
0.2 0.4
0.4
0.3 0 0.2
1 1 1
0.8
0.8 0.8
N u m b e r o f s e rie s M = 1 0
N u m b e r o f s e rie s M = 2 0
N u m b e r o f s e rie s M = 5 0
0.6
0.6 0.6
0.4
0.4 0.4
0.2
0.2 0.2
0
0 0 -0.2
Initial signal (impulse function) and signal due to Discrete Fourier Series at various
number of series (M=1,3,5,10,10,50)
46 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Example 2: FFT vs. SPECTRUM
6
10
f1=8Hz
4
Amplitude
f1=33Hz
0
2
Noise
-10 0
0 0.2 0.4 0.6 0.8 0 20 40 60
time [t] frequency [Hz]
signal spectrum by using spectrum
20 25
20
10
15
10
0
5
No noise
-10 0
0 0.2 0.4 0.6 0.8 0 20 40 60
time [t] frequency [Hz]
Note:
Fourier Transform Spectrum
jt
CFT ( x) X ( j ) x(t )e dt CSPEC( x) X ( j ) x 2 (t )e jt dt
N 1 N 1
DFT ( x) X ( j ) x n e jnT DSPEC ( x) X ( j ) x 2 n e jnT
n 0 n 0
47 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Example 3: DFT
FFT command
1 0.6
0.8 0.4
0.6 0.2
Signal
0.4 0
0.2 -0.2
0 -0.4
0 0.2 0.4 0.6 0.8 0 20 40 60
Number of samples or time interval Frequency Hz
DFT by definition Spectrum command
0.6 8
0.4
DFT Magnitude
6
0.2
4
0
2
-0.2
-0.4 0
0 20 40 60 0 20 40 60
Frequency Hz Frequency Hz
Results: +) Fast Fourier Transform (FFT) is not a new form of Fourier transform, the
N
applied formula is exactly such an expression X (kF0 ) [ x (nT )e j 2kn / N ] , however,
n 1
H0=x*exp(-j*2*pi*k'*n/N);
n: pointer of sample, k: pointer of frequency, k must be transposed (k’)
H0=H0/N; H0 must be standardized by dividing by N (number of samples)
48 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Example 4: Amplitude and phase spectrum
Spectrum
1 0.6
0.8 0.4
Amplitude
0.6 0.2
Signal
0.4 0
0.2 -0.2
0 -0.4 fftshift(H)
0 0.2 0.4 0.6 0.8 0 20 40 60
Number of samples Frequency(Hz)
Phase spectrum Phase spectrum
4 20
2 15
Phase(rad)
Phase(rad)
0 10
-2 5
unwrap()
-4 Angle(H) 0
0 20 40 60 0 20 40 60
Frequency(Hz) Frequency(Hz)
1 3 2
100
0.5 2 0
50
0 1 -2
-0.5 0 -4 0
0 50 100 0 50 100 0 50 100 0 50 100
1 3 2
100
0.5 2 0
50
0 1 -2
-0.5 0 -4 0
0 50 100 0 50 100 0 50 100 0 50 100
49 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
6. POWER SPECTRAL DENSITY FUNCTION
6.1. Introduction
(1) The random signals can not be determined exactly. It means that the random
signals always differ from each other at the different observations, moreover, the
random signals contain the random parameters that can not be described by the means
of analytical quantities and determinant-parameter methods, but they are only able to
be described by the terms of statistical parameters that can differ from one random
signal to another.
(2) Thus the questions are that what terms of statistical parameters can be able to
describe the characteristics of the random signals. It is well known that the two most-
commonly-used means are:
i) amplitude distribution functions of random signal
ii) power spectral density of random signal (or equivalent as the correlation
function)
(3) In the term of amplitude distribution, the commonly-used statistical quantities are:
i) mean value (expectation); ii) mean square (variance when zero-mean signals) or root
mean square (standard deviation when zero-mean signals).
(4) It is broadly said that the square of DFT magnitudes of any function x(t) is
considered as power contribution of any frequency components in x(t) over the
frequency domain.
50 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N 1
1
- Mean value (Expectation): x E[ x ]
N
x
i 0
i
N 1
2 12 2
- Mean square : E[ x ]
x
N
x
i 0
i
N 1
2 1 2
- Root mean square : x E[ x ] sqrt (
N
x
i 0
i )
- Variance:
N 1 N 1
2 2 1
2 2 1 2 2
E[( x x ) ] E[ x ] [
x
N
x
i 0
x i][
N
x ]
i 0
i
y a x b
Linear relationship of y and x as y=ax+b, we easily obtain:
x2 a 2 x2
51 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
The instantaneous power of signal x(t) at any time t can be defined as:
Power | x(t ) |2
For the complex signals, we must use the mean power or expected power:
T /2 T /2
1
2 2 1
Mean power E[ x (t )] LimT x ( t ) dt x 2 (t ) dt
T T / 2 T T / 2
(when T is long enough)
NT
1 N 1 2 1 2 2
Discrete mean power S xx E[ x (t )] x n x (t ) dt
N n 0 NT 0
1 N 1 N 1 j 2kn / N 2 1 N 1 N 1 N 1
S xx 3 [ X ( kF0 )e ] 3 X k X m e j 2 ( k m ) n / N
N n 0 k 0 N k 0 m 0 n 0
N 1
1 2
S xx 2
N
X
k 0
k
52 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1 N 1 2
We have: S xx x k
N k 0
As a result, the relationship between the mean power of signal in the time domain and
power spectrum, or the mean power of signal can be presented in the term of the power
spectrum. This expression has been well known as the Parseval’s Theorem.
N 1
1 N 1
2 2
k 0
x Xk
k
N k 0
(Parseval’s Theorem)
1 2
Pxx (k ) Xk ; k=[0,N-1]
N
Pxx (k ) Pxx ( N k )
Note: In the same way that x k2 represents a measure of signal power at a point in
the time domain, Pxx(k) represents the measure of signal power at the point in
the frequency domain.
Therefore, the vector P xx=[Pxx(k)], k=[0:N-1] is considered as the measure of
estimation of the power spectral density (PSD) of signal x(t).
53 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N 1
1 2
Discrete PSD: 1) S xx 2
N
X
k 0
k (Frequency domain)
1 2
Pxx (k ) Xk ; m=[0,N-1]
N
N 1 N 1
1 2 2
Parseval’s Theorem: x k X k (Parseval’s Equality)
k 0 N k 0
NT
1 2
S
Continuous PSD: 1) xx x (t ) dt (Frequency domain)
NT 0
1 / 2T /T
2) S xx P xx ( f )df or S xx P xx ( ) d
1 / 2T /T
The relationship of the power spectrum to the auto-correlation: The discrete auto-
correlation of vector x from sampled values of signal x(t) with extended period has
been defined as follows:
N 1
1
R xx ( s )
N
x x
n 0
n n s ; s=[0:N-1]
54 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
N 1
1 2
R xx (0)
N
x
n0
n S xx Mean power
N 1
X k x n e j 2kn / N ; k=[0:N-1] (Definition of DFT)
n0
1 2
S R DFT {Rxx ( s )} Xk Pxx (k ) ; k=[0:N-1]
N
In the conclusion, the periodogram of vector x with periodic extension is the DFT of
the auto-correlation function of x, that means:
1 2
S R DFT {Rxx ( s )} X k Pxx ( k )
N
Role of the auto-correlation function gives us information on how much
dependence of given sample xn on nearby samples xn+1, xn+2, and so on.
The Cross Spectrum: involves two time-dependant signals (two time series). Suppose x
and y are vectors of length N sampled from two signals x(t) and y(t).
1
- The cross-periodogram:
Pxy(k) XkY'k ; k=[0:N-1]
N
55 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
1 N1
Sxy xn yn
- Mean cross-power: N n0
1 N1 1 N1
Sxy Rxy(0) 2 XnY'n Pxy(k)
N k0 N k0
Role of the cross spectrum gives us information on how the cross
correlation function is distributed over the frequency scale.
Coherence function: The magnitude of the cross periodogram Pxy(k), that is Pxy (k ) , is
the measure of the coherence of signals x(t) and y(t) at different frequencies.
2
Pxy ( f )
- Coherence function: Coh 2 xy ( f )
Pxx ( f ) Pyy ( f )
6.4. Examples
Example 1: FFT and SPECTRUM
0 0 0
0 0.5 1 0 50 0 50
-5 2 Spectrum
x 10 X
1 8 8
0.8
6 6
0.6 Difference in shape?
4 Different in value? 4
0.4
2 2
0.2
0 0 0
0
56 | Le Thai Hoa - Notes on 0.5 1 0 Process and Digital
Stationary Random 50 Signal
0 Processing 50
Time (s) Frequency (Hz) Frequency (Hz)
Note: 1) Using FFT command
H=FFT(x);
H=[H(1) 2*H(2:N/2)]/N; %Vector length limit and
standardization
2) Using DFT’s definition
H=x*EXP(-j*2*pi*k'*n/N); %DFT by definition
H=[H(1) 2*H(2:N/2)]/N;
3) Using SPECTRUM command
S=SPECTRUM(x);
S=[S(1) 2*S(2:N/2)]; %Vector length limit
4) Using spectrum’s definition
S=ABS(H).^2; %Square of absolute amplitude
S=[S(1) 2*S(2:N/2)]/ (N^2); % Vector length limit and
standardize
Digital Signal or
Random Process
57 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
normal signal uniform signal sine signal impulse signal
10 10 2 1
0 5 0 0.5
-10 0 -2 0
0 FFT 5 0 5 0 5 0 5
0.4 4 1.5 1
1
0.2 2 0.5
0.5
0 0 0 0
0 spectrum 50 0 50 0 50 0 50
40 40 200 40
20 20 100 20
0 0 0 0
0 50 0 50 0 50 0 50
8 8
10
6 6
0
4 4
-10
2 2
-20 0 0
0 5 0 50 0 50
4
3 3
3
2 2
2
1 1
1
0 0 0
0 50 0 50 0 50
58 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
7. COMPUTATION OF COHERENCE FUNCTION
7.1. Introduction
(1) Coherence function: Defined as the measure to evaluate the statistical
independence of two stochastic processes or two digital signals (but in the
frequency-domain)
(2) Coherence function: Computed by using the cross-spectrum (two signals) and
auto-spectrum (one signal) of two stochastic processes
(3) Correlation function: Similar to the coherence function, the correlation function
also is the measure to evaluate the statistical independence of two stochastic
processes or two digital signals (but in the time-domain)
Inter-relation or inter-influence
(2) The Auto-spectrum represents the Fourier transform of the Auto correlation
59 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
(3) The Cross-spectrum represents the Fourier transform of the Cross correlation
(4) The Cross-spectrum is the complex valued function whose magnitude and phase
are used in the signal processing to indicate the degree of correlation between two
signals
(5) The magnitude of the cross spectrum indicates whether frequency components of
one signal are associated with large or small amplitudes at the same frequency in
the second signal.
(6) The phase of the cross spectrum indicates whether the phase lag (tre) or lead (dan
truoc) of one signal with respect to the second signal for a given frequency
component.
(7) The cross-spectrum is used to determine the coherence function between two
signals
(8) Cross spectrum and coherence function can be determined by the MATLAB as
powerful engineering program for Digital Signal Processing
2
| S xy ( f ) | 2
| Cohxy ( f ) |
S xx ( f ) S yy ( f ) (Magnitude Squared Coherence Function)
Note: Here two discrete signals are taken into account. In cases, many pairs of two
discrete signals are counted, the summation must be used, called the average cross
spectrum and the average coherence function. Complex cross spectrum and
coherence function consists of two parts: i) magnitude and ii) phase
61 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
S u ,y ( )
Cohu ,y ( ) : The span-wise coherence
S u , 0 ( ) S u , y ( )
correlation.
7.4. Examples
Example 1: FFT vs. SPECTRUM
signal spectra by using FFT
20 8
6
10
4
0
2
-10 0
0 0.2 0.4 0.6 0.8 0 20 40 60
time [t] frequency [Hz]
signal spectrum by using spectrum
20 25
20
10
15
10
0
5
-10 0
0 0.2 0.4 0.6 0.8 0 20 40 60
time [t] frequency [Hz]
6
10
4
0
2
-10 0
0 0.2 0.4 0.6 0.8 0 20 40 60
Time[s] Frequency [Hz]
Input Signal
1
0.5
-0.5
0 20 40 60
62 | Le Thai Hoa - Notes on Stationary Random Process and Digital Signal Processing
Time[s]