Digital Image Processing

Applications of Image Processing

Dr. Hetal N. Patel

List of Applications
Target detection
2. Object recognition
3. Feature extraction and matching
4. Optical Character Recognition Systems
5. Writer identification
6. Digital watermarking
7. Medical and industrial applications
8. Image Morphing and
9. Image Fusion
1.

1/11/2015

Object Recognition
Object recognition is an amazing human feat.
Humans effortlessly recognize objects within
a fraction of a second.
Unlike computers, humans can easily
recognize a face, understand spoken words,
and distinguish different styles of
handwriting.
Object recognizing is a study of how
machines can observe the environment,
learn to distinguish patterns of interest and
make reasonable decisions about the
categories of patterns.

1/11/2015

Automated object-recognition
systems
Living organism
Biological
sensor

information

Cognitive
process
identification

ENVIRONMENT

Electronic
sensor

data

Algorithm

Automated pattern recognition

1/11/2015

Automated Pattern Recognition

Automated pattern recognition systems use

algorithms to process data collected either
electronically through sensors or transcribed
by a human.
Pattern recognition algorithms

Data

Description

Features

Classification

1/11/2015

identification

Approaches to object recognition

Object-recognition approaches

Decision theoretic

Classical
approach

Parametric Non-parametric
method
method

Structural

Neural network
approach

Parametric
method

1/11/2015

Non-parametric
method

Applications of object recognition

Applications of object recognition in
different fields:
1. Medicine
2. Forensics
3. Remote sensing
4. Industrial applications

1/11/2015

Medicine
ECG pattern recognition
EEG pattern recognition
Mammorgram analysis
Lung sound classification and analysis

1/11/2015

Forensic
Fingerprint recognition
Face recognition
Iris recognition
Vein-pattern recognition

1/11/2015

Remote sensing
Cloud pattern analysis
Crop forecast

1/11/2015

10

Industrial applications
Character recognition
Automated inspection

1/11/2015

11

The conventional personal identification

techniques can be broadly classified into two
types:
Knowledge based:
1. Password
2. Personal identification number
Token based
1. Passport
2. Driving license
3. ID card
Traditional knowledge-based and token-based
approaches are unable to satisfy the
requirements of an electronically
interconnected information society.

1/11/2015

12

Face recognition
Face detection
Eigen faces
Neural network approach

1/11/2015

13

Face Detection
Faces are diverse, semi-rigid, semi-flexible,
culturally significant and part of our
individual entity.
A variety of face detection techniques are
employed from correlation, neural networks,
eigen templates, Bayesian models and flexible
models.
Given an arbitrary image, the goal of face
detection is to determine whether or not
there are any faces in the image and if
present, return the image location and
extent of each face.

1/11/2015

14

Detecting faces in a single image

Four categories:
1. Knowledge-based methods
2. Feature-invariant approaches
3. Template matching methods, and
4. Appearance-based methods

1/11/2015

15

Knowledge-based methods
Rule based,
Rules capture the relationships between
facial features
The rules are derived from a researchers
knowledge of human faces.

1/11/2015

16

Feature based methods

Facial features such as eyebrows, eyes
,nose, mouth and hair-line are commonly
extracted using edge detectors
Human skin color has been used and
proven to be an effective feature in many
applications.

1/11/2015

17

Template matching
A standard face pattern is manually
predefined, or parameterized by a
function.
Correlation values with the standard
patterns are computed for the face
contour, eyes, nose and mouth
independently.

1/11/2015

18

Appearance based methods

Statistical analysis and machine learning is
used to find the relevant characteristics of
face and non-face images.
Bayesian classification or maximum
likelihood can be used to classify a
candidate image location as a face or nonface.

1/11/2015

19

Eigenfaces
Face vectors can be obtained by
computing the eigen vectors of the
images autocorrelation matrix.
These eigen vectors are known as eigen
faces.
Face image consists of different patterns
such eyes, nose, mouth etc. These
patterns are separated by specific
distances.

1/11/2015

20

Neural network approach

The advantage of using neural networks for

face detection is the feasibility of training a
system to capture the complex classconditional density of face patterns.
The neural-network architectures employed
by researchers in the field of face
recognition include (i) Kohonen Self
Organizing Feature Map (SOFM), (ii) autoassociative network, and (iii) probabilistic
decision-based neural network.

1/11/2015

21

Challenges in face recognition

1.
2.

3.
4.

5.

Facial Expression
Image Orientation
Imaging Conditions
Pose
Presence of Glass

1/11/2015

22

Feature extraction and matching

When the input data to an algorithm is too large to

be processed and it is suspected to be notoriously
redundant then the input data will be transformed
into a reduced representation set of features (also
named features vector).

Transforming the input data into the set of features is

called feature extraction.

If the features extracted are carefully chosen it is

expected that the features set will extract the
relevant information from the input data in order to
perform the desired task using this reduced
representation instead of the full size input
1/11/2015

23

Optical Character Recognition

Systems

OCR systems consist of five major stages

Pre-processing
Segmentation

Feature

Extraction
Classification
Post-processing

1/11/2015

24

Pre-processing

The raw data is subjected to a number of preliminary

processing steps to make it usable in the descriptive
stages of character analysis. Pre-processing aims to
produce data that are easy for the OCR systems to
operate accurately. The main objectives of preprocessing are :

Binarization
Noise reduction
Stroke width normalization
Skew correction
Slant removal

1/11/2015

25

Binarization

Document image binarization (thresholding)

refers to the conversion of a gray-scale image into
a binary image. Two categories of thresholding:

Global, picks one threshold value for the entire

document image which is often based on an
estimation of the background level from the
intensity histogram of the image.

Adaptive (local), uses different values for each

pixel according to the local area information

1/11/2015

26

Noise Reduction - Normalization

Noise reduction improves the quality of
the document. Two main approaches:
Filtering (masks)
Morphological Operations (erosion,
dilation, etc)
Normalization provides a tremendous
reduction in data size, thinning extracts
the shape information of the characters.

1/11/2015

27

1/11/2015

28

Skew Correction

Skew Correction methods are used to

align the paper document with the
coordinate system of the scanner. Main
approaches for skew detection include
correlation, projection profiles, Hough
transform.

1/11/2015

29

Slant Removal
The slant of handwritten texts varies
from user to user. Slant removal methods
are used to normalize the all characters
to a standard form.
Popular deslanting techniques are:
Calculationof the average angle of nearvertical elements

1/11/2015

30

Slant Removal

1/11/2015

31

Segmentation

Text Line Detection (Hough Transform, projections, smearing)

Word Extraction (vertical projections, connected component analysis)

Word Extraction 2 (RLSA)

1/11/2015

32

Segmentation

Explicit Segmentation
In explicit approaches one tries to identify
the smallest possible word segments
(primitive segments) that may be smaller
than letters, but surely cannot be segmented
further. Later in the recognition process
these primitive segments are assembled into
letters based on input from the character
recognizer. The advantage of the first
strategy is that it is robust and quite
straightforward, but is not very flexible.

1/11/2015

33

Feature Extraction

In feature extraction stage each character is represented as a

feature vector, which becomes its identity. The major goal of
feature extraction is to extract a set of features, which
maximizes the recognition rate with the least amount of
elements.

Due to the nature of handwriting with its high degree of

variability and imprecision obtaining these features, is a
difficult task. Feature extraction methods are based on 3
types of features:

Statistical
Structural
Global transformations and moments

1/11/2015

34

Statistical Features

Representation of a character image by

statistical distribution of points takes care of
style variations to some extent.

The major statistical features used for

character representation are:

Zoning
Projections and profiles
Crossings and distances

1/11/2015

35

Zoning

The character image is divided into NxM

zones. From each zone features are
extracted to form the feature vector. The
goal of zoning is to obtain the local
characteristics instead of global
characteristics

1/11/2015

36

Zoning Density Features

The number of foreground pixels, or the
normalized number of foreground pixels, in
each cell is considered a feature.
Darker squares indicate higher density of zone
pixels.

1/11/2015

37

Zoning Direction Features

Based on the contour of the character
image
For each zone the contour is followed
and a directional histogram is obtained by
analyzing the adjacent pixels in a 3x3
neighborhood

1/11/2015

38

Zoning Direction Features

Based on the skeleton of the character image

Distinguish individual line segments
Labeling line segment information
Line segments are coded with a direction
number 2 = vertical line segment 3 = right
diagonal line segment 4 = horizontal line segment
5 = left diagonal line segment
Line type normalization
Formation of feature vector through zoning

1/11/2015

39

Projection Histograms

The basic idea behind using projections is that character images,

which are 2-D signals, can be represented as 1-D signal. These
features, although independent to noise and deformation, depend
on rotation.

Projection histograms count the number of pixels in each column

and row of a character image. Projection histograms can separate
characters such as m and n .

1/11/2015

40

Profiles

The profile counts the number of pixels (distance)

between the bounding box of the character image
and the edge of the character. The profiles describe
well the external shapes of characters and allow to
distinguish between a great number of letters, such as
p and q.

1/11/2015

41

Profiles
Profiles can also be used to the contour of the
character image
Extract the contour of the character
Locate the uppermost and the lowermost points
of the contour
Calculate the in and out profiles of the contour

1/11/2015

42

Structural Features

Characters can be represented by structural features

with high tolerance to distortions and style variations.
This type of representation may also encode some
knowledge about the structure of the object or may
provide some knowledge as to what sort of
components make up that object.

Structural features are based on topological and

geometrical properties of the character, such as
aspect ratio, cross points, loops, branch points,
strokes and their directions, inflection between two
points, horizontal curves at top or bottom, etc.

1/11/2015

43

Structural Features

1/11/2015

44

Structural Features

A structural feature extraction method for

recognizing Greek handwritten characters
[Kavallieratou et.al2002]
Three types of features:
Horizontal and Vertical projection histograms
Radial histogram
Radial out-in and radial in-out profiles

1/11/2015

45

Global Transformations -Moments

The Fourier Transform (FT) of the contour of the

image is calculated. Since the first ncoefficients of
the FT can be used in order to reconstruct the
contour, then these ncoefficients are considered to be
a n-dimesional feature vector that represents the
character.

Central, Zenrike moments that make the process

of recognizing an objectscale, translation, and
rotation invariant. The original imagecan be
completely reconstructed from the moment
coefficients

1/11/2015

46

1/11/2015

47

Writer identification

Handwriting analysis involves two

phases:
1. The hardwareink, paper, pens, pencils, typewriter,
printers

2. Visual examination of the writing

1/11/2015

48

Handwriting Characteristics
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.

Line quality
Word and letter spacing
Letter comparison
Pen lifts
Connecting strokes
Beginning and ending strokes
Unusual letter formation
Shading or pen pressure
Slant
Baseline habits
Flourishes or embellishments
Diacritic placement
1/11/2015

49

Digital Watermarking

1.
2.

3.
4.

What is watermarking?
Digital watermarking is a technique for inserting
information (the watermark) into an image, which can
be later extracted or detected for variety of
purposes including identification and authentication
purposes.
Applications of watermarking:
Copyright Protection : Invisibly mark products
Manage distribution of assets :Apply unique
watermark key to each copy of a distributed
video/image
Embed all necessary data in a single image
Naturally expands to video watermarking
1/11/2015

50

Common Watermarking methods

Simple
Spatial Domain Modification made to the luminance
values

Transformed Domain
DCT
DWT
SVD

Product of 3 matrices A = UVT

U ,V are orthogonal matrices: UTU= I,VTV = I
= diag (1, 2, ...).
The diagonals of are called the singular values of A
The columns of U are called the left singular vectors of A and
The columns of V are called the right singular vectors of A.

1/11/2015

51

Classification

An effective watermark should be:

Robust to common manipulations

Unobtrusive so that it does not affect visual
quality

Categorize based on:

Capacity
Complexity
Invertibility
Robustness
Security
Transparency
Verification
1/11/2015

52

Robustness

Fragile
Detection fails with even minor modification
Useful in tampering detection
Common in simple additive watermarking

Robust
Detection is accurate even under modification
Need for robustness dependent on use of
data

1/11/2015

53

Verification or detection methods

Non-blind
The watermarking scheme requires the use of
the original image

Semi-Blind
The watermarking scheme requires the
watermark data and/or the parameters used to
embed the data

Blind
If the watermarking scheme does not require the
original image or any other data
1/11/2015

54

Watermarking basic procedure

1/11/2015

55

Types of Watermark

1/11/2015

56

Results

1/11/2015

57

1/11/2015

58

Conclusion
Watermarking is still a challenging research
field with many interesting problems :
Robustness to both geometric and nongeometric attacks with blind detection.
Other attacks such as protocol attacks and
cryptographic attacks
Correct recovery of multiple-bit message
Public-key detection

1/11/2015

59

Image Morphing

60

Contents
Brief Introduction
What is morphing?
Image processing techniques
Image morphing algorithms
Applications of image morphing

61

Brief Introduction:
Morphing derived from the word metamorphosis.
The word metamorphosis is of the Greek origin, and is composed of two words:
- Meta: meaning between or after, and
- Morphosis: the way form or structure changes.
Metamorphosis means to change shape, appearance
or form.

62

What is morphing?
Morphing can be defined as:
- Transition from one object to another.
- Process of transforming one image into another.
Morphing can be defined as an animated transformation
of one image into another image.

An animation technique that allows you to blend two

still images, creating a sequence of in between
pictures that when played in Quick Time,
metamorphoses the first image into the second.

63

 Example:

64

How is morphing done?

As the metamorphosis proceeds ,
- The first image is gradually distorted and is faded out.

- The second image starts out totally distorted

toward the first and is faded in.

65

Image processing techniques

Morphing involves image processing techniques like:
- Warping.
- Cross dissolving.

66

Cross-dissolving
A simple way to transform an image to an other is to do crossdissolving.
It is just a pixel by pixel interpolation to fade from one image to an
other.
A cross-dissolve is a sequence of images which
implements a gradual fade from one to the other.
Cross dissolving means that one image fades to another image using
linear interpolation.

67

Cross-dissolving (contd.)

The result is usually bad because the two images are often not aligned and just
cross-dissolving gives a double-image effect.

This problem is particularly apparent in the middle frame, where both input
images contribute equally to the output.
In order to overcome this problem, warping is used to align the two images
before cross dissolving.

68

Example of image morphing process

Step 1) The original images of a human face and a lions face,
scaled to the
same size (fig.1).
Note that the distance between the eyes and the mouth
is
significantly longer in the lion's picture than in the
girls picture.

Fig.1 The original images.

69

Example (contd.)
Step 2) Perform coordinate transformations on the partitioned
images to
match the feature points of two images.
Here, matching is done within the eyes and the mouths for these two
images.
We can find that the girls face becomes longer, and
the lion's
face becomes shorter.

Fig.2 The modification of images.

70

Example (contd.)
Step 3) Cross-dissolve the two images to generate a new image. The morph
result looks like a combination of these two wrapped faces.

The new face has two eyes and one mouth, and it possesses the features
from both the girls and the
lion's faces.

Fig.3 The image transformation.

71

Warping
Warping is the difficult step of morphing.
There are a lot of techniques available for warping.
Warping determines the way pixels from one image are
correlated with corresponding pixels from the other image.
Two ways to warp an image:
- Forward mapping.
- Reverse mapping.

72

Warping v/s Cross-dissolving

KETUL & RAJENDRA

73

Image morphing algorithms

There are several image morphing algorithms as given below:

1) Field/Line Morphing
2) Mesh Warping
3) Thin Plate Spline Based Image Warping
4) Multilevel Free-form Deformation (MFFD)

74

1) Field/Line Morphing
The algorithm uses lines to relate the features in source image
to features in the destination image.
It uses reverse mapping for warping the image.
Correspondence achieved using feature line(s) in source and
destination images.

Source Image

Destination Image
75

Field/Line Morphing (contd.)

Two step process
Step I : Interpolating the lines:
- Interpolate the coordinates of the end points of
lines.

every pair of

Step II : Warping the Images:

- Each of the source images has to be deformed

needed frame.

towards the

-The deformation works pixel by pixel is based

the reverse mapping. This algorithm is
Neely Algorithm.

on
called Beier-

76

Beier-neely algorithm (Pixel position)

1) Compute position of pixel X in destination image relative to the line
drawn in destination image.
(x,y) (u,v)
Q

Q
X
v

u
P

Destination Image

Source Image
77

Beier-neely algorithm (Pixel position)

2) Compute coordinates of pixel in source image whose position relative to
the line drawn in source image is (u,v).
(u,v) (x,y)
Q
Q
X

u
P

Destination Image

Source Image
78

2) Mesh Warping
The mesh-warping algorithm relates features with non-uniform mesh in the
source and destination images, i.e., the images are broken up into small regions that
are mapped onto each other for the morph.
To illustrate the 2-pass mesh warping algorithm, consider the image sequence .
Here, refer the source and the target images as IS and IT, respectively. The source
image is associated with mesh MS.
It specifies the coordinates of control points.
A second mesh MT specifies their corresponding positions in the target image.

79

Mesh Warping (contd.)

Notice that landmarks such as the eyes, nose and lips lie below corresponding
grid lines in both meshes.
MS and MT used to define spatial transformation that maps all point in IS onto
IT.
All intermediate frames in the morph sequence are the product of a four-step
process:
For each frame f do
Linearly interpolate mesh M, between MS and MT
Warp IS to
I1, using meshes MS and M
Warp IT to I2, using meshes MT and M
Linearly interpolate image If, between I1 and I2
End

80

3) Thin Plate Spline Based Image Warping

Thin-plate Spline is a conventional tool for surface interpolation over scattered
data.
It is an interpolation method that finds a "minimally bended" smooth surface that
passes through all given points.

81

Thin Plate Spline Based Image Warping

(contd.)
The figure is a simple example of coordinate transformation using TPS. It starts
form two sets of points for which it is assumed that the correspondences are known
(a).
The TPS warping allows an alignment of the points and the bending of the grid
shows the deformation needed to bring the two sets on top of each other (b).

82

Marker controlled segmentation

technique for Medical applications

Content
Introduction
Algorithm
Results

Introduction

Original Image

Watershed segmentation without

Marker

Original Image

After Segmentation

Watershed segmentation

Algorithm
Read the original image I
Morphological reconstruction(using dilation) of the I
Complement of reconstructed image Ic
Marker= I Ic
Extended and imposed minimum to obtained the
markers
Compute watershed transform of the markers

Morphological
Reconstruction(Dilation)

Result (I)

Original Image

Result (II)

Dilated image

Reconstructed image

Result (III)

Complement of
reconstructed image

Result (IV)

Marker = I-Ic

Result (V)

Final Marker

Result (VI)

After watershed

Border line

Result (VII)

Pseudo color image

Segmented object with

different color

Image Fusion

1/11/2015

97

Outline
Introduction
Objective
Single sensor image fusion
Multi sensor image fusion
Image registration
Image fusion
Different image fusion methods
Applications
conclusion

Introduction
To FUSE means to join together
The process of combining relevant information from
two or more images into a single image.
The resulting image will be more informative than any
of the input images.
Create new images that are more suitable for the
purposes of human/machine perception, and for
further image-processing tasks such as segmentation,
object detection or target recognition
For example, visible-band and infrared images may be
fused to aid pilots landing aircraft in poor visibility.

Objective
Require high spatial and high spectral resolution
in a single image simultaneously
Techniques allow the integration of different
information sources
The fused image can have complementary spatial
and spectral resolution characteristics
Fusion techniques can distort the spectral
information of the multispectral data while
merging
Examples: A camera equipped visible image
capturing as well as infrared sensors

Single sensor image fusion

Problem with Single sensor

A visible-band sensor for camera is appropriate for
daylight scenes but is not suitable for poorly
illuminated situations found during night, or under
adverse conditions such as in fog or rain

Multi-sensor image fusion

An infrared camera is supplementing the
digital camera
Individual images are fused => fused
image

Advantages of multi-sensor image

fusion
Extended range of operation
Reduced uncertainty
Increased reliability
Robust system performance
Compact representation of
information(by means of color coding)

Block diagram for image fusion

Sensors

Registration

Fusion

Image registration
Concept of signal conditioning
Process of aligning two or more images of the
same scene (base image & input image)
Objective-bring the input image into alignment
with the base image by applying a spatial
transformation

Spatial transformation maps locations in one image to

new locations in another image

This difference is due to

Lens and other internal sensor distortions, or
differences between sensors and sensor types

Two synthetic images, each with

same anatomy but different values in
corresponding regions

Algorithm for image registration

Generic Block diagram of image

fusion

Evolution in image fusion

Simple image fusion
attempts

Pyramid-decompositionbased image fusion

Wavelet-transformbased image fusion

Different fusion methods

Spatial
Domain

Transform
Domain

Averaging,

Lapacian
Pyramid Based

Brovey Method

Curvelet
Transform
Based

Principal
Component
Analysis (PCA)

Wavelet
Transform

IHS Based
Methods

High Pass
Filtering

Problems with spatial domain fusion

Produce spatial distortion
Classification problem required for
further processing
Reduced contrast
So use Multiresolution analysis

original image is divided into no. of different

resolution images (image pyramid)
lower resolution imagery does not affect the
higher resolution imagery

Applications
Image Classification
Aerial and Satellite imaging
Medical imaging
Robot vision
Concealed weapon detection
Multi-focus image fusion
Digital camera application
Battle field monitoring