Chuyên

h ê đề dà
dành
h cho
h lớ
lớp cao học
h

IMAGE PATTERN RECOGNITION

WITH NEURAL NETWORK
Dr. Lê Dũng
g

School of Electronics and Telecommunications

Hanoi University of Science and Technology

Hà nội
ộ 11/2011
 huyên đề dành cho lớp cao học
C 2

TABLE OF CONTENT
Part I: Design a real application using image pattern
recognition with neural network
1. Automatic Envelopes Classification System in the post office
2. Skin color detector with Neural Network

Part II: Image Pattern Recognition

+ Digital Image and Image Acquisition
+ Image Enhancement
+ Image
I Segmentation
S t ti
+ Image Pattern Recognition

Part III: Recognition with Neural Network

+ Theory of Neural Network
+ Using Neural Network for Pattern Recognition

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học
C 3

MỤC TIÊU CỦA CHUYÊN ĐỀ

™ Hiểu thế nào là “Mẫu

Mẫu ảnh
ảnh” (Image Pattern) và nguyên
lý nhận dạng mẫu ảnh (Image Pattern Recognition).

™ Nguyên tắc tạo và chuẩn hóa mẫu ảnh phù hợp

cho việc nhận dạng bằng mạng Nơron.
Nơron

™ Nắm được lý thuyết cơ bản về mạng Nơron trong

nhận dạng mẫu ảnh.

Î Định
Đị h hướng
h ớ cho
h việc
iệ ứng
ứ d
dụng kỹ thuật
th ật nhận
hậ
dạng mẫu ảnh bằng mạng nơron.

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học
C 4

TÀI LIÊU THAM KHẢO CHÍNH

‰ “Digital Image Processing”
Barnd Jähne
Spring Verlag 1995
‰ “Neural Networks for Pattern Recognition”
Bishop, C.M.
Oxford University Press, 1995.
‰ “Neural Network Design”
Martin T. Hagan, Howard B. Demuth, Mark Beale
Thomson Learning, 1996.
‰ “Gradient-based learning applied to document recognition”
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P
Proceedings of the IEEE,
IEEE vol.
vol 86,
86 1998.
1998

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 5

AUTOMATIC ENVELOPES CLASSIFICATION SYSTEM

(Project : 010-2001-TCT-RDP-BC-26 at the Research Institute of Post and Telecommunication 2000-2001)

Collected envelopes
Control & Operation System Centre
control
Recognition result
(postcode)
Classification of
envelope types Automatic My control
Handwritten Postcode design
Recognition

Standard Image 640x480

envelopes Mechanical
CCD Light System to do
camera classification
Separate,
p , Index &
Put each of
envelopes Conveyer belt
on conveyer belt

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 6

DESIGN THE PROCESS OF FUNCTIONS

Error (reject)
xxxxx
Preprocessing Recognition
Segmentation
(Neural network)
28117
Image WxH
Classification OK
CCD
Camera Index i
Light

Envelope
Multi-layers Feed-Forward
Neural Network
Conveyer belt

‹ Preprocessing ‹ Segmentation ‹ Recognition

- Grayscale convert - Segment the postcode area - Patterns normalization
- Enhancement - Segment
g 5 areas for - Recognition using
- Binary convert each code number a neural network.
- Subtraction - Segment 5 number patterns - Check result
- Noise filter (OK or reject)
- Find envelopep frame
- Rotate & Crop Postcode area

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 7

NEURAL NETWORK FOR PATTERN RECOGNITION

16x16
Input layer

Input layer : 256

Normalized
Pattern 16x16
Layer 2
Layer 2 : 106
5x5
16 7x7 16 (Feature map)
3x3 3x3 Layer 3
Activation function
Log-Sigmoid 7 7 Layer 3 : 68
1 (Sub-sampling)
a=
1 + e − n net 10 Output layer Output layer : 10
(classification)
0 1 2 3 4 5 6 7 8 9 Result
Total : 440 neurons

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 8

SOFTWARE MODULE (3) : RECOGNIZING POSTCODE

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 9

SKIN COLOR DETECTOR

FOR A REAL-TIME HAND GESTURE RECOGNITION SYSTEM

2D Color
Camera

Skin color
Skin color detection
detector image

Hand feature extraction

Hand gesture recognition

Hand tracking
g

Human-Robot interaction
Human-Computer interaction

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 10

THE OVERVIEW OF MY COMBINING METHOD

Step 1 Step 2 Result
Image frame
PRNN

The
Collecting statistical
t ti ti l
skin color Training
model Building
Training data set
+ Skin color patterns
+ Non-skin color patterns

Elliptical
p skin color
The originality of the method: model
Step 1: Using the statistical skin color model
Æ Fast detection with not high accuracy (coarse work) Multi-layers Feed-
Æ Check all pi
pixels
els of the image frame (coarse data) Forward Neural
Step 2: Using the pattern recognition neural network Network

ÆDetect with high accuracy (fine work)

ÆL
ÆLearn tto correctt th
the ddetection
t ti errors off the
th statistical
t ti ti l skin
ki color
l model.
d l

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 11

DEFINE A PATTERN FOR SKIN COLOR PRNN*

Skin detection result based on
the color information of a pixel
Non-skin

Pattern for PRNN

with 9 pixels

Skin
Skin Non-skin
Under examination
pixel and its
8 neighborhood pixels

* PRNN – Pattern
P R
Recognition
i i Neural
N l Network
N k

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

 huyên đề dành cho lớp cao học Part I : Design a real application
C 12

TỔNG KẾT CÁC KHÁI NIỆM TRONG PHẦN I

¾ Khái niệm
iệ mẫu
ẫ ảnh
ả h trong
t nhận
hậ dạng.
d

¾ Khái niệm tạo và chuẩn hoá mẫu ảnh

¾ Khái niệm
ệ nhận
ậ dạng
ạ g

¾ Khái niệm mạng nơron và huấn luyện mạng nơron

¾ Khái niệm thư viện mẫu ảnh để huấn luyện

¾ Khái niệm huấn luyện mạng nơron

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRONICS AND TELECOMMUNICATION