DESIGN OF SINGLE LAYER NEURAL NETWORKS

SYSTEM USING VHDL FOR CHARACTER

RECOGNITION
VIVIEN WONG WEI PING
This thesis is submitted in partial fulfillment of the requirements for
the Degree in Bachelor of Electrical (Electronics) Engineering
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
2002 ii
DECLARATION
“It is hereby declared that all materials in this thesis are the
effort of my own work and materials which are not the
effort of my own work has been clearly acknowledged.”
Signature : _____________
Name of Author : Vivien Wong Wei Ping
Date : 21 February 2002 iii
Specially dedicated to
My beloved parents and sister who have
encouraged, guided and inspired me throughout my journey of education iv
ACKNOWLEDGEMENT
I would like to take this opportunity to express my deepest gratitude to my
supervisor of this project, Prof. Dr. Mohamed Khalil B. Haji Mohd Hani who has
relentlessly and tirelessly assisted me in completing this project. He has given me
support and insight in doing this project and has patiently listened and guided. My
utmost thanks also goes to my family who has given me support throughout my
academic years.
It is to my advantage that I have received help and support from friends and the
staff in the faculty labs. I would like to thank Mr. Cheang Choon Hoe who has willingly
offered his help despite his tight schedule. My thanks to the lab technicians, En.Zul and
Abang Din who has been lending a helping hand with dealing with lab facilities. It is of
my greatest thanks and joy that I have met with these people. Thank you. v
ABSTRACT
This projects aims to build a neural network hardware system using VHDL
implementation mainly to recognize characters. Neural network is a collection of many
neurons to form a network that is able to learn. The neural networks system contains one
input layer, one hidden layer and one output layer. There is no feedback in the network
and input signal propagates through the network on layer-by-layer basis. The neural
networks system is capable of separating nonlinear variables and to recognize characters.
The main objective of the project is to obtain a tangible neural networks
hardware system that incorporates a multiplier, adder, register and activation function.
This neural network hardware system will be able to learn and recall and hence
recognize characters through interfacing with a data collector and personal computer
using the graphical user interface (GUI). Output of recognized characters will be
displayed in 7-segments as integers on the UP1 board.
A neural network system for character recognition will be implemented in
hardware using VHDL. The several logic and arithmetic operations will be done in
VHDL based on the back propagation algorithm. This particular system requires the use
of combinational and sequential logic synthesis. The input vector is obtained from the
personal computer through a GUI and sent into the UP1 board through the data collector.
Output will be displayed on 7 segments of the UP1 board. vi
ABSTRAK
Projek ini bertujuan membina sebuah sistem rangkaian neural dalam bentuk
perkakasan dengan menggunakan implementasi melalui VHDL untuk mengenal huruf
abjad. Rangkaian neural merupakan koleksi neuron-neuron yang berkebolehan belajar.
Rangkaian neural ini mempunyai satu lapisan input, satu lapisan terlindung dan satu
lapisan output. Tiada suapbalik dalam rangkaian ini dan isyarat input akan bergerak dari
satu lapisan ke lapisan yang berikutnya. Rangkaian neural ini berupaya membahagikan
pembolehubah tak linear untuk mengenal huruf abjad.
Objektif utama projek ini adalah untuk membina satu sistem yang mengandungi
pendarab, penambah dan pendaftar serta fungsi activasi. Sistem rangkaian neural ini
berupaya belajar dan mengingat untuk mengenal huruf adjad melalui persemukaan
pengumpul data dan komputer dengan menggunakan Pesemukaan Grafik Pengguna.
Ouput akan dipaparkan pada ruas tujuh dalam bentuk integer pada bod UP1.
Sistem rangkaian neural untuk pengenalan huruf dalam bentuk perkakasan akan
diimplementasikan menggunakan VHDL berdasarkan pada algoritma penjanaan
belakang. Sistem ini memerlukan sintesis logik kombinasi dan bersiri masa. Vektor
input diperoleh daripada komputer melalui Persemukaan Grafik Pengguna dan dihantar
ke bod UP1 melalui pengumpul data. Output akan dipaparkan pada ruas tujuh pada bod
UP1. vii
CONTENTS
CHAPTER TOPIC PAGE
Title i
Declaration ii
Dedication iii
Acknowledgment iv
Abstract v
Abstrak vi
Contents vii
List of Tables xi
List of Figures xii
List of Appendices xiv
CHAPTER I PROJECT OVERVIEW
1.1 Introduction 1
1.2 Objectives 2
1.3 Work Scope 2
1.4 Project Contribution 3
CHAPTER II ARTIFICIAL NEURAL NETWORKS
2.1 Chapter Overview 4
2.2 Theories and Concepts of Neural Networks 4
2.2.1 Human Nervous System 4
2.2.2 Basic Components of a Biological Neuron 6
2.2.3 Function of a Neuron 7
 2.2.4 Speed Benchmarking 7
2.3 Artificial Neurons 7 viii
2.3.1 Basic Elements of An Artificial Neuron 8
2.3.2 Activation Function 9
2.3.3 Bias Value 10
CHAPTER III DESIGN METHODOLOGY AND CAD TOOLS
3.1 Chapter Overview 11
3.2 Design Process 11
3.3 Implementation with VHDL using Maxplus II 12
3.3.1 General Comment on VHDL Syntax 14
3.3.2 The Standard Logic 15
3.4 Computer Aided Design 15
3.4.1 Compiler 16
3.4.2 Simulator 16
3.4.3 Programmer 17
3.5 Supported Design Implementation using VHDLmg 18
3.5.1 Digital System Modular Design 19
CHAPTER IV DIGITAL CHARACTER RECOGNITION BASED ON NEURAL
NETWORKS
4.1 Chapter Overview 20
4.2 System Configuration and Functionality 20
4.3 Character Recognition System 22
4.3.1 Clock Divider 24
4.3.2 Counter 24
4.3.3 Register 5 25
4.3.4 BCD to 7 Segment Decoder 26
CHAPTER V NEURAL PROCESSOR DESIGN
5.1 Chapter Overview 27
5.2 Concepts of Design of Neural Processor 27
5.2.1 Single Layer Neuron Network 27 ix
5.2.2 Network Architecture of Neural Processor 28
5.2.3 Weights Format 30
5.2.4 Digital Hard Limit Activation Function 30
5.3 Digital Design of Neural Processor 31
5.4 Data Path Unit of Single Neuron-Neuroprocessor 32
5.4.1 Shift Register 34
5.4.2 Multiplier 34
5.4.3 8 bit Adder 35
5.4.4 Latch 35
5.4.5 Comparator 36
5.4.6 ROM 36
5.5 Controller Design 37
5.5.1 Digital Controller Implementation 38
CHAPTER VI INTERFACE DESIGN-DATA COLLECTOR
6.1 Chapter Overview 40
6.2 Interface Design-Data Collector 40
6.3 Data Path Unit of Data Collector 41
6.3.1 Latch 42
6.3.2 Multiplexer 42
6.3.3 Address Generator 43
6.3.4 35 bit Shift Register 43
6.3.5 Address Decoder 44
6.3.6 Address Detector 44
6.4 Data Collector Controller 45
6.4.1 Data Collector Controller Digital Implementation 47
CHAPTER VII SOFTWARE INTERFACE DESIGN
7.1 Chapter Overview 48
7.2 Graphical User Interface 48
7.3 Handshaking Protocol and Parallel Port Programming 50
7.3.1 Parallel Port Assignments 51 x
7.3.2 Port Outputs 52
7.3.3 Port Inputs 53
7.4 Visual Basic Handshaking Protocol –‘Send Pattern’ Code 53

CHAPTER VIII HARDWARE DESIGN AND IMPLEMENTATION

8.1 Chapter Overview 56
8.2 Field Programmable Gate Array 56
8.2.1 UP1 Education Board 57
8.2.2 EPF10K20 Device 57
8.3 Pin Assignments 58
8.4 Parallel Port Interface 59
CHAPTER IX TESTS, RESULTS AND SIMULATIONS
9.1 Chapter Overview 61
9.2 Early Stage Tests 61
9.3 Results and Simulations 63
9.3.1 Character Recognition System 63
9.3.2 Neural Processor 64
9.3.3 Interface Design-Data Collector 65
CHAPTER X SUGGESTIONS AND CONCLUSION
10.1 Chapter Overview 67
10.2 Recommendations For Future Work 67
10.2.1 Multiplexed Neural Networks System 67
10.2.2 Weights Length 68
10.2.3 Floating Point Unit 68
10.3 Conclusion 69
REFERENCE 70
Appendix A 71
Appendix B 88
Appendix C 92
Appendix D 100
Appendix E 127 xi
LIST OF TABLES
TABLE TITLE PAGE

7.1 Parallel Port Address 51

8.1 Parallel Port Pin Assignments 58
8.2 Parallel Port Signals, Arranged by Pin Number 60 xii
LIST OF FIGURES
FIGURE TITLE PAGE
2.1 Human Nervous System with Motor and Sensory Organs 5
2.2 Basic Components and Signal Flow in a Biological Neuron 6
2.3 Model Of Artificial Neuron 8
2.4 Activation Function Graphs 10
3.1 Basic processes in Digital Systems Design 12
3.2 Compiler Tool in Maxplus II 16
3.3 Simulator Tool in Maxplus II 17
3.4 Programmer Tool in Maxplus II 17
3.5 Conceptual Diagram of VHDL Module Generator 18
4.1 Final Character Recognition System 20
4.2 Input and Output of Character Recognition System 21
4.3 Top Level Block Diagram of Character Recognition System 23
4.4 Top Level Internal Architecture of Character Recognition System 23
4.5 Clock Divider implemented using Flip-flops 24
4.6 Block Diagram of Counter vin_count 25
4.7 Block Diagram of Register 25
4.8 Input and Output ports of BCD to 7 Segment Decoder 26
5.1 Single Layer Neuron Network 28
5.2 Network Architecture 5 Neuron Neural Processor 29
5.3 Weight Matrix to Calculate Output 29
5.4 Digital Implementation of 5 Neurons Neural Processor 31
5.5 Complete Functional Block of Data Path Unit of Single Neuron 32
5.6 Basic Functional Block Diagram of Single Neuron 33 xiii
5.7 Shift Register Module 34
5.8 AND gate for Multiplication 34
5.9 8 bit signed adder 35
5.10 Latch to Capture Input Value 35
5.11 Transfer Function Implementation 36
5.12 ROM module 36
5.13 State Diagram of Controller 37
5.14 Digital Controller of Neural Processor 39
5.15 Digital System of Single Neuron and Controller 39
6.1 Complete Functional Block Diagram of Data Collector 41
6.2 8:1 Multiplexer 42
6.3 Address Generator Implemented using 3 bit Counter 43
6.4 35 bit Shift Register module 43
6.5 Address Decoder implemented using 3:8 decoder 44
6.6 State Diagram of Controller of Data Collector 45
6.7 Internal Architecture of Data Collector Controller 48
7.1 GUI main software window 49
7.2 Input Test Pattern Code Generation 50
7.3 Pin Assignments 52
7.4 Port Assignments 52
7.5 Flow Chart of Handshaking Protocol 54
7.6 Visual Basic Send Pattern Code with Description 55
8.1 Flex_Expan A, Flex_Expan B, Flex_Expan C Numbering Convention 59
9.1 Excel Sheet on Test of Letter ‘K’ 62
9.2 Simulation Results of Character Recognition System 63
9.3 Input Test Pattern ‘Z’ 64
9.4 Simulation Results of 5 Neuron Neural Processor for letter ‘K’ 65
9.5 Simulation Results of Data Collector 66 xiv
LIST OF APPENDICES
APPENDIX TITLE PAGE
A. VHDL source code 71
B. Excel Simulation Files 88
C. Memory Initialization Files 92
D. Visual Basic Code 100
E. Test Patterns 127