Time Series Analysis & Classification

T. Medkour & H. Brairi

National School of Artificial Intelligence

Outline

1. Motivation

2. Short Time Fourier Transform

3. Wavelet Transform

4. Applications
Motivation
Example: Fourier Analysis of Localised Waves
3
1
−1
−3 Simultaneous Signals: sin(40πx) and sin(170πx)

0.0 0.2 0.4 0.6 0.8 1.0

Time

Sequential Signals: sin(40πx) and sin(170πx)

sin(40πx) sin(170πx)
1.5
0.5
−1.0

0.0 0.5 1.0 1.5 2.0

Time
Example: Fourier Analysis of Localised Waves
Periodogram

200 400
0

0.0 0.1 0.2 0.3 0.4 0.5

Frequency
Periodogram

100 200
0

0.0 0.1 0.2 0.3 0.4 0.5

Frequency
Limitations of Fourier Analysis

▶ Fourier analysis decomposes a signal into sinusoids of different

frequencies.

▶ Excellent for stationary signals (global frequency content).

▶ Problem: Poor at analyzing non-stationary or transient signals.

▶ Time-localization is lost: when did the frequency component occur?

▶ Example: Heartbeat with an occasional arrhythmia - Fourier sees all

frequencies but doesn’t tell when.
Short Time Fourier Transform
Short Time Fourier Transform (STFT)

▶ Introduce a window function w(t) localized in time.

▶ Windowed Fourier Transform:

∫ ∞
X (t, ω) = x(τ )w(τ − t)e −iωτ dτ
−∞

▶ Provides a compromise: some time and frequency localization.

Example: STFT Analysis of Localised Waves
STFT (Spectrogram) − Simultaneous
0.2
Frequency (kHz)

0.15
0.1
0.05
0
0 0.2 0.4 0.6 0.8 1

Time (s)

STFT (Spectrogram) − Sequential

0.2
Frequency (kHz)

0.15
0.1
0.05
0
0 0.2 0.4 0.6 0.8 1

Time (s)
Heisenberg uncertainty:

With STFT we cannot have arbitrarily good resolution in both time and
frequency.
Key issue:
▶ Fixed window size - poor at low frequencies (long signals), overkill at
high frequencies (short signals).
▶ We lose frequency localization for what we gain in time localization.
Underlying this is a trade-off between time and frequency localization
▶ The STFT cannot separate events of a distance smaller than the window
width, that is to localize the two frequencies and the transient
phenomena
Example: STFT Analysis of Localised Waves
STFT (Spectrogram) − Sequential − Wide Window
0.2
Frequency (kHz)

0.15
0.1
0.05
0
0 0.2 0.4 0.6 0.8 1

Time (s)

STFT (Spectrogram) − Sequential − Middle Window

0.2
Frequency (kHz)

0.15
0.1
0.05
0
0 0.2 0.4 0.6 0.8 1

Time (s)

STFT (Spectrogram) − Sequential − Short Window

0.2
Frequency (kHz)

0.15
0.1
0.05
0
0 0.2 0.4 0.6 0.8 1

Time (s)
Wavelet Transform
What is a Wavelet?

▶ A wavelet is a function ψ(t) with compact support and specific

mathematical properties:
∫
• Zero mean: ψ(t)dt =0
∫
• Finite energy: |ψ(t)|2 dt < ∞

▶ Acts like a "small wave" - localized in time.

▶ Used to construct a basis for representing signals efficiently.

Scale-varying idea: zoom in/out to match features in the signal.

Wavelet Families: Key Properties

▶ Haar (1909):
• Oldest wavelet; simple and compact
• Discontinuous
• One vanishing moment
▶ Daubechies (DbN):
• Compact support; smooth (for N > 1)
• N vanishing moments
• Widely used in signal processing and compression
▶ Mexican Hat:
• Continuous wavelet; second derivative of a Gaussian
• Excellent time-frequency localization
• No compact support
▶ Coiflets:
• Compact support
• Vanishing moments for both wavelet and scaling function
• Good for feature extraction
▶ Symmlets:
• Symmetrized Daubechies wavelets
• Nearly symmetric; better reconstruction properties
• Optimized for smoothness and frequency response
Families of Wavelets
Continuous Wavelet Transform (CWT)

▶ CWT analyzes signal x(t) with scaled and shifted wavelets:

∫ ( )
1 t −b
W (a, b) = x(t) √ ψ dt
|a| a

▶ a: scale (like frequency), b: position (like time)

▶ Large scale (a): captures low frequency trends
▶ Small scale (a): captures high frequency detail
Discrete Wavelet Transform (DWT)

▶ Efficient version of CWT with discrete a = 2j , b = k2j

▶ Analyzes signal at dyadic scales and positions

▶ Multiresolution analysis (MRA): signal decomposed into approximations

and details

▶ Fast Wavelet Transform (FWT) allows efficient computation O(n)

Mathematical Structure of DWT

Multiresolution Analysis (MRA):

▶ Sequence of nested spaces: · · · ⊂ V−1 ⊂ V0 ⊂ V1 ⊂ · · ·

▶ Vj : Vector space spanned by the approximation space scaling functions

ϕj,k (t) = 2j/2 ϕ(2j t − k) at resolution 2−j

▶ Wj : detail space spanned by wavelet functions ψj,k (t) = 2j/2 ψ(2j t − k)

such that Vj+1 = Vj ⊕ Wj
Filter Bank Implementation

DWT uses two filters:

▶ Low-pass filter h[n]: averages (approximation coefficients)

▶ High-pass filter g[n]: differences (detail coefficients)

Given signal X [n]:

∑
Aj+1 [n] = h[k − 2n]Aj [k]
k
∑
Dj+1 [n] = g[k − 2n]Aj [k]
k

Downsampling by 2 after filtering halves the number of coefficients at each

level.
How to Compute DWT (Haar Example)

▶ Suppose we have 8 data points: x = [1, 0, −3, 2, 1, 0, 1, 2]

▶ Step 1: Compute averages and differences in pairs:
[ ] [ ]
(1 + 0) (−3 + 2) (1 + 0) (1 + 2) 1 −1 1 3
A1 = √ , √ , √ , √ = √ ,√ ,√ ,√
2 2 2 2 2 2 2 2
[ ] [ ]
(1 − 0) (−3 − 2) (1 − 0) (1 − 2) 1 −5 1 −1
D1 = √ , √ , √ , √ = √ ,√ ,√ ,√
2 2 2 2 2 2 2 2
▶ Step 2: Repeat on the averages A1 to get A2 , D2 , and so on.
Result: Signal is represented by one final average and a sequence of detail
coefficients.
Applications
Wavelets vs Fourier: A Summary

▶ Fourier: fixed resolution; great for stationary signals

▶ Wavelets: adaptive resolution; better for transient, non-stationary signals
▶ Fourier basis: global sines and cosines
▶ Wavelet basis: localized and multiscale
▶ Wavelets allow sparse representations of real-world data
Application: Image Compression (JPEG 2000)

▶ Wavelets decompose images into smooth and detailed parts.

▶ Low-frequency subbands store general appearance (approximation).
▶ High-frequency subbands encode edges and fine detail.
▶ Most high-frequency coefficients can be quantized or discarded.
▶ JPEG 2000 uses discrete wavelet transform (e.g., CDF 9/7 wavelets).

Result: High compression ratio with minimal visible distortion.

Application: Biomedical Signal Analysis

▶ ECG/EEG signals contain both smooth rhythms and sharp spikes.

▶ Wavelet transforms isolate features at different scales.
▶ Denoising: thresholding wavelet coefficients suppresses noise.
▶ Feature extraction: detect QRS complexes, epileptic spikes.
▶ Used in sleep studies, arrhythmia detection, and more.
Application: Geophysical and Seismic Data

▶ Wavelets detect changes in seismic waveforms.

▶ Discontinuities correspond to layer boundaries underground.
▶ Can improve interpretation of reflection signals.
▶ Adaptive resolution helps localize short-lived tremors.

Example: Used in petroleum exploration for identifying reservoir structures.

Application: Financial Time Series

▶ Wavelets capture volatility at multiple time horizons.

▶ Used for denoising stock returns, detecting anomalies.
▶ Multiscale analysis reveals short-term fluctuations vs. long-term trends.
▶ Also used in option pricing models and risk analysis.

Example: Wavelet variance decomposition of S&P 500 returns.

Application: Numerical Solutions to PDEs

▶ Wavelets represent solutions to differential equations compactly.

▶ Adaptive grid refinement: use more points where solution changes
rapidly.
▶ Sparse matrix representations reduce memory usage.
▶ Examples include fluid dynamics, heat transfer, and electromagnetism.

Tool: Wavelet-Galerkin methods for efficient PDE solvers.