Statistical Signal Processing

Chapter 1: INTRODUCTION
Instructor:

Phuong T. Tran, PhD.

(slides are borrowed from Prof. Natasha Devoyre, UIC)

Ton Duc Thang University

Faculty of Electrical and Electronics Engineering

Apr. 27 2019
 Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 2 / 19

 Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 2 / 19

 Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 2 / 19

 Course Introduction

Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 3 / 19

 Course Introduction

Example of detection

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 4 / 19

 Course Introduction

Example of estimation

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 5 / 19

 Course Introduction

Goals
Infer value of unknown state of nature based on noisy observations
Mathematically, optimally

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 6 / 19

 Course Introduction

Detection example 1: Digital

communications
Noise

Source Encoder Channel Decoder Destination

10001010100010
0 ↔ s0 (t) = sin(ω0 t)
1 ↔ s1 (t) = sin(ω1 t)

Detect (
s0 (t) + n(t) if ‘0’ sent
r (t) =
s1 (t) + n(t) if ‘1’ sent

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 7 / 19

 Course Introduction

Detection example 2: Radar

communication
Send s(t) = sin(ωc t), 0 ≤ t ≤ T
Receive
———————————————————————————–

Hypothesis H0
r (t) = n(t), 0 ≤ t ≤ T

Detect?

Hypothesis H1
r (t) = Vr sin((ωc + ωd )(t − τ ) + θr )
+ n(t), τ ≤ t ≤ T + τ
Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 8 / 19
 Course Introduction

Further examples

Sonar: enemy submarine

Image processing: detect an aircraft from infrared images

Biomedicine: cardiac arryhthmia from heart beat sound wave

Control: detect occurence of abrupt change in system to be

controlled

Seismology: detect presence of oil deposit

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 9 / 19

 Course Introduction

Difference between detection

and estimation

Detection:

Discrete set of hypotheses

Right or wrong

Estimation:

Continuous set of hypotheses

Almost always wrong - minimize error instead

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 10 / 19

 Course Introduction

Estimation example 1:
communications

Pulse amplitude modulation (PAM)

Analog source Sampler Transmitter

Receiver?

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 11 / 19

 Course Introduction

Estimation example 2: Radar

Send s(t) = sin(ωc t), 0 ≤ t ≤ T
Receive
———————————————————————————–

Hypothesis H0
r (t) = n(t), 0 ≤ t ≤ T

Estimate?

Hypothesis H1
r (t) = Vr sin((ωc + ωd )(t − τ ) + θr )
+ n(t), τ ≤ t ≤ T + τ
Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 12 / 19
 Course Introduction

Our method

Will treat everything generally, with a unified mathematical

representation

Bias towards Gaussian noise and linear observation - parameter

model

Examples mainly drawn from communications / radar

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 13 / 19

 Course Introduction

Aside: “Classical” vs.

“Bayesian”

Classical
Hypotheses/parameters are fixed, non-random

Bayesian
Hypotheses/parameters are treated as random variables with
assumed priors (or a priori distribution)

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 14 / 19

 Course Introduction

Course outline
Chapter 1: INTRODUCTION
Lecture 1: Course introduction
Lecture 2: Review of probability theory
Chapter 2: DETECTION THEORY
Lecture 3: Hypotheses testing
Lecture 4: Deterministic signals in noise
Lecture 5: Random signals
Lecture 6: Statistical decision theory
Chapter 3: ESTIMATION THEORY
Lecture 7: Minimum variance unbiased estimators
Lecture 8: Cramer-Rao lower bound
Lecture 9: Linear models
Lecture 10: General minimum unbiased estimation
Lecture 11: Best linear unbiased estimators
Lecture 12: Maximum likelihood estimators
Lecture 13: Least squares estimation
Lecture 14: Bayesian estimation
Lecture 15: Kalman filtering (self-study)
Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 15 / 19
 Grading Policy

Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 16 / 19

 Grading Policy

Grading policy

Progressive assessment (40%):

Attendance and homework
Midterm test
Simulation exercises (for bonus points)

Final assessment (60%):

Term paper report

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 17 / 19

 Textbooks and References

Outline

1 Course Introduction

2 Grading Policy

3 Textbooks and References

Apr. 27 2019 EE702030 - Lecture 1: Course Introduction 18 / 19

 Textbooks and References

Textbooks and references

Textbooks:
Steven M. Kay.
Fundamentals of Statistical Signal Processing - Volume II:
Detection Theory, 1e.
Prentice-Hall PTR, 1998.
Steven M. Kay.
Fundamentals of Statistical Signal Processing - Volume I:
Estimation Theory, 1e.
Prentice-Hall PTR, 1993.

References:
Steven M. Kay.
Fundamentals of Statistical Signal Processing - Volume III: Detection Theory, 1e.

Prentice-Hall PTR, 2013.

T. K. Moon and W. C. Stirling, Pearson
Mathematical Methods and Algorithms for Signal Processing.
Pearson, 1999.
Harry L. Van Trees.
Detection, Estimation, and Modulation Theory, Part I: Detection, Estimation,
and Filtering Theory, 2nd edition.
Apr. 27John
2019Wiley & Sons, 2013. EE702030 - Lecture 1: Course Introduction 19 / 19