Simone Milani

Room 216
BIOMETRICS
DEI A A.A. 2021/2022
Phone: 049 827 7641 LECTURE 03
E-mail:
simone.milani@dei.unipd.it BASE NOTIONS - FINGERPRINTS
 Basics of Digital Signal Processing

DIGITAL SIGNAL PROCESSING BASICS

Biometrics 21/22-Milani 2
 DIGITAL SIGNAL IN BIOMETRICS MEASUREMENTS

After acquisition, digital signals can be modelled by time-series or matrix of integer values

Audio example

1
𝐹" = Sampling frequency 𝑥 𝑛 = 𝑥! 𝑡 𝑡 = 𝑛𝑇 = 𝑥(𝑛𝑇)
𝑇
Image example

𝐼(0,0) ⋯ 𝐼(0, 𝑁 − 1)
𝐼 𝑥, 𝑦 = ⋮ ⋱ ⋮
Biometrics 21/22-Milani 𝐼(𝑀 − 1,0) ⋯ 𝐼(𝑀 − 1, 𝑁 − 1) 3
 QUANTIZATION OF SAMPLES

¨ Every sample is quantized and represented with a finite number of bits (8, 12, o 16 bit)

¨ Most images use 8 bit/sample: every sample is an integer number from 0 to 255 depending
on light intensity; for color images, digitalization is done on the three components R,G,B

¨ Audio precision can vary 8,16,32, 48 bits; depends also on sampling frequency

Es.
254=11111110
150=10010110

4=00000010
Biometrics 21/22-Milani 4
 BINARY SIGNALS

¨ Sometimes whenever shape and geometric details are involved, binary images are used

¨ Usually they can be obtained thresholding a standard image or applying some segmentation

¨ Examples: fingerprints, silhouettes for gait analysis

Before After After
Gabor filter Gabor filter thresholding

¨ The signal is a matrix whose cell can assume binary values {0,1}

𝐼(0,0) ⋯ 𝐼(0, 𝑁 − 1)
𝐼= ⋮ ⋱ ⋮ 𝐼 𝑥, 𝑦 ∈ {0,1}
𝐼(𝑀 − 1,0) ⋯ 𝐼(𝑀 − 1, 𝑁 − 1)

Biometrics 21/22-Milani 5
 EXAMPLES OF BINARIZATION (1/2)

¨ Thresholding
1 𝐼 𝑢, 𝑣 ≥ 𝑇
𝐵 𝑢, 𝑣 = .
0 𝐼 𝑢, 𝑣 < 𝑇
where T can be computed in different ways.

Otsu’s method: minimize intra-class variance 𝜎#$%&! = 𝑝' 𝜎' + 𝑝( 𝜎( where 𝑝' 𝑝( is the
probability (percentage) that the pixel is lower (higher) than T. 𝜎' 𝜎( is the variance for
the class of 1s (0s).

+
min 𝜎)$%&! = max 𝜎)$%*& = max(𝜎 − 𝜎)$%&! ) = max 𝑝' 𝑝( 𝜇( − 𝜇'

1. Compute histogram and probabilities of 𝐼(𝑢, 𝑣)

2. Set 𝑝' , 𝑝( and 𝜇' , 𝜇(
3. For 𝑇 = 1, … , 255
compute the new 𝑝' 𝑝( and 𝜇' 𝜇(
compute 𝜎)$%*&
If 𝜎)$%*& is max, 𝐭𝐡𝐞𝐧
𝑇,-% = 𝑇

Biometrics 21/22-Milani 6
 EXAMPLES OF BINARIZATION (2/2)

¨ Segmentation of foreground objects

Simple case: define background and

Foreground with respect to colors

Background

Foreground

With knn refinement

Moe complex solutions: gradient based, watershed, using neural networks

Biometrics 21/22-Milani
 OTHER BIT DEPTHS
¨ Genomic signal processing for DNA/RNA sequences
Signal can be seen a function
𝒔 𝑛 : ℝ ↦ {𝐴, 𝐺, 𝐶, 𝑇}

or a set of 4 binary functions (Voss representation)

1 if 𝑠 𝑛 = 𝐴
𝑠%̂ 𝑛 = >
0 otherwise
1 if 𝑠 𝑛 = 𝐺
𝑠̂& 𝑛 = >
0 otherwise

1 if 𝑠 𝑛 = 𝐶
𝑠̂' 𝑛 = >
0 otherwise
1 if 𝑠 𝑛 = 𝑇
𝑠(̂ 𝑛 = >
0 otherwise

q Typing biometrics: user’s typing patterns (speed, the duration of a single keypress, and how
long it takes between releasing a key to pressing the next).

signals taking values in the keyboard keys

Biometrics 21/22-Milani 8
 DEPTH AND 3D SIGNALS (1/2)

¨ 3D sensors are employed as well

¨ The acquired signal is usually a depth map D= 𝑑(𝑥, 𝑦)

¨ Depth value is an integer parameterizing the distance of the pixel from

the camera itself.

¨ From depth representation it is possible to infer a 3D modelling of faces

or body

Biometrics 21/22-Milani 9
 DEPTH AND 3D SIGNALS (2/2)
ply
format ascii 1.0 PLY format
A mesh model need to be specified element vertex 8
property float x
by the following elements property float y
property float z
property uchar red
property uchar green
• Vertex property uchar blue
element face 7
• Edges property list uchar int vertex_index
• Faces element edge 5
property int vertex1
• Normals property int vertex2
property uchar red
property uchar green
Types of meshes: property uchar blue
end_header
0 0 0 255 0 0
0 0 1 255 0 0
0 1 1 255 0 0
0 1 0 255 0 0
• Generic (rare) 1 0 0 0 0 255
• Triangular 1 0 1 0 0 255
1 1 1 0 0 255
• Rectangular 1 1 0 0 0 255
3 0 1 2
3 0 2 3
4 7 6 5 4
OpenGL use generic meshes, 4 0 4 5 1
4 1 5 6 2
But convert them into triangular 4 2 6 7 3
4 3 7 4 0
0 1 255 255 255
1 2 255 255 255
2 3 255 255 255
3 0 255 255 255
2 0 0 0 0 10
3DAR - static code 659674
 3D TIME SERIES
Whenever the identification involves the time modelling of a set of 2D/3D coordinates, n-
dimensional time series can be involved

𝒔 𝑛 : ℝ ↦ ℝ$

Example: joints position in skelethon for gait modelling

§ A skeleton is a framework of rigid body“bones” connected by articulated joints

§ Used as an (invisible?) armature for the articulated bodies

§ Joint has 0-6 degrees of freedom (DoF): three coordinates and 3 rotation angles.

3DAR - static code 659674 11

 JOINTS AND DOF

§ Rotational joints
§ Translational joints
§ Compound (mix the previous two motions)

Saddle
Hinge
back and forth & up
Rotation along
and down motion
one axis
2DOF
1DOF

Ball-and-socket Sliding
Rotation along Translation on a
all 3 axes plane
3DOF 2DOF

S. Milani - Augmented and Virtual Reality 12

 JOINTS FROM ACQUIRED SIGNALS

§ Usually time series of 2D coordinates (n=2)

§ One time serie for each joint; traditionally, 15 markers are used for a 30-dimensional time
serie

§ Acquired via different means

i. Giroscopic sensors, accelerometers, … (IMU) -> specil equipment for diagnostics or
information leackage from devices
ii. RGB cameras
iii. Depth cameras
S. Milani - Augmented and Virtual Reality https://www.nbc.com/saturday-night-live/video/motion-capture/n13489 13
 FOURIER TRANSFORM

¨ Continuous-Time Fourier Transform (CTFT) 𝜔 = 2 𝜋𝑓

referred as Fourier spectrum or simply spectrum
/ /

𝑋! 𝑓 = M 𝑥! 𝑡 𝑒 .01 % 𝑑𝑡 = M 𝑥! (𝑡)𝑒 .0+23 % 𝑑𝑡

./ ./
/
¨ Inverse CTFT 1
𝑥! 𝑡 = M 𝑋! 𝑓 𝑒 0+2 %3 𝑑𝑓
2𝜋
./
/

¨ Discrete Fourier Transform (DFT) 𝑋 𝑓 = S 𝑥 𝑛 𝑒 .0+ 23$" = 𝑋(𝑓) 𝑒 05(3)

$4./
Magnitude (dB) 𝑋! 𝑓

Phase (rad) 𝜃 𝑓 = arg 𝑋! 𝑓

/
1
q Inverse Discrete Fourier Transform (IDFT) 𝑥[𝑛] = M 𝑋 𝑓 𝑒 0+2 $"3 𝑑𝑓
2𝜋
./

Biometrics 21/22-Milani 14
 FOURIER PROPERTIES

Biometrics 21/22-Milani 15
 FOURIER TRANSFORM: 1D EXAMPLES

𝑠𝑖𝑛/𝑐𝑜𝑠 𝑔𝑎𝑢𝑠𝑠𝑖𝑎𝑛 𝑟𝑒𝑐𝑡

"
𝑒 )*+ rec(𝑡 /𝑇)
cos(2𝜋𝑓! 𝑡 )

1 T sinc(𝑇𝑓)
[𝛿 𝑓 − 𝑓! + 𝛿 𝑓 − 𝑓! ] "
2 𝑒 )*,

Biometrics 21/22-Milani 16
 FOURIER TRANSFORM: 1D EXAMPLES

Biometrics 21/22-Milani 17
 PERIODICITIES

Sampling in time =.periodic repetition in frequency

𝑋! 𝜔
𝑥! 𝑡

𝑋 𝜔
𝑥 𝑛 = 𝑥- 𝑛𝑇.

Ts 4Ts
𝜔# = 2𝜋/𝑇#

It is possible to sample frequencies as well = repetition in time domain

8!.(
1
𝑋[𝑢] = S 𝑥 𝑛 𝑒 .0+ 29$ Ω = 𝜔𝑇: = 2𝜋𝑢
𝑁
$4' Normalized frequency
Biometrics 21/22-Milani 18
 DFT FOR IMAGES

Continuous / /

𝐼! 𝑓; , 𝑓< = M M 𝑖! (𝑥, 𝑦) 𝑒 .0+2(=";>=#<) 𝑑𝑥𝑑𝑦

./ ./

with its inverse

/ /

𝑖! 𝑥, 𝑦 = M M 𝐼! (𝑓; , 𝑓< ) 𝑒 0+2(=";>=#<) 𝑑𝑓; 𝑑𝑓<

./ ./

Discrete
8.( ?.(
+2
.0 8? (9;>@<)
𝐼 𝑢, 𝑣 = S S 𝑖(𝑥, 𝑦)𝑒
;4' <4'

with its inverse

8.( ?.(
1 +2
0 8? (9;>@<)
𝑖 𝑥, 𝑦 = S S 𝐼(𝑢, 𝑣)𝑒
𝑁𝑀
;4' <4'

Biometrics 21/22-Milani 19
 2D DFT: EXAMPLES

Rect 2D

Log-magnitude of spectrum

Fingerprint detail FFT

magnitude phase

𝑤 𝑥, 𝑦 = 𝐴 cos 2 𝜋𝑓! (𝑥 cos 𝜃 + 𝑦 𝑠𝑖𝑛𝜃)

Biometrics 21/22-Milani 20
 FILTERING

Filering is widely used on biometric signals

𝑦 𝑛 = S ℎ 𝑛 − 𝑖 𝑥 𝑖 = ℎ ∗ 𝑥[𝑛]
)4'

1 𝑛=0
where h[n] is the response of the filter to the impulsive input 𝛿 𝑛 =.
0 𝑛≠0

Ideally, it is like computing the correlation between x[𝑖] and ℎ. [𝑖] with delay n
ℎ ∗ 𝑥[𝑛]
𝑥[𝑖] 𝑥[𝑖]

h[𝑖] h_[𝑖]

Example: 𝑥[𝑛]
convolution of two rect x h[𝑛]
y[𝑛]

Biometrics 21/22-Milani 21
 FILTERING IN FREQUENCY DOMAIN

Filtering in time =.samplewise multiplication in frequency

𝐻 𝜔 = 𝐷𝐹𝑇(ℎ)
X 𝜔 = 𝐷𝐹𝑇 𝑥

Y 𝜔 = 𝐻 𝜔 𝑋(𝜔)

Usually amplitude response 𝐻 𝜔 ∈ ℂ and can be represented in abs 𝐻 𝜔 and phase ∠𝐻 𝜔

Amplitude usually measured in dB. = 20 𝑙𝑜𝑔%2 |𝐻 𝜔 |. Phase in radiants

Biometrics 21/22-Milani 22
 INTERPRETATION OF FILTERING

Filter responses are therefore maximum whenever input signal and response are
aligned and very similar.

i.e. filters catch some specific structure in a signal

E.g. Average filters

2𝜋𝑛
𝑥' 𝑛 = sin
100

𝑥 𝑛 = 𝑥' $ + 𝑒 𝑛
AWGN

𝑦 𝑛 = ℎ 𝑛 ∗ 𝑥[𝑛]

Biometrics 21/22-Milani 23
 FILTERING FOR BIOMETRICS
Filtering is very helpful in ridge extraction for fingerprints
a) pores are brighter
b) ridges can be broken due to cuts or creases;
c) adjacent ridges may appear joined due to moist skin or pressure.

Gabor filters Before Gabor filter After Gabor filter Thresholded

Also used for audio, selection of specific frequencies - denoising

t
Biometrics 20/21 1s 24
 SPECTROGRAMS
Signals can be represented by spectrograms using Short-time (or term) Fourier Transform (STFT)

𝑋A 𝑓 = S 𝑥 𝑛 𝑤[𝑛 − 𝑑𝑅]𝑒 .0+ 23$"

$4./
' '
If Constant-Overlap-Add (COLA) propetry is verified B 𝑤 𝑛 − 𝑑𝑅 = 1. ∀𝑛 . we have B 𝑋$ 𝑓 = 𝑋(𝑓)
$%&' $%&'
It is possible to represent it as an image where rows=frequencies, columns=different windows d,
pixel intensities=|𝑋3 𝑓 |

Final Fantasy VIII –

Compression of Time

Biometrics 21/22-Milani 25
 SPECTROGRAMS-EXAMPLES

5 male speakers uttering

“puhujantunnistus” (in
finnish: speaker recognition).

Which are the same?

Biometrics 21/22-Milani 26