Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
3Activity
0 of .
Results for:
No results containing your search query
P. 1
INTEGRATED HANDWRITING RECOGNITION SYSTEM USING ARTIFICIAL…

INTEGRATED HANDWRITING RECOGNITION SYSTEM USING ARTIFICIAL…

Ratings: (0)|Views: 174|Likes:
Published by AnaBBMaria

More info:

Published by: AnaBBMaria on Nov 26, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

05/31/2014

pdf

text

original

 
IntelliSec – The 1st International Workshop on Intelligent Security Systems11-24th November 2009, Bucharest, Romania
INTEGRATED HANDWRITING RECOGNITION SYSTEM USINGARTIFICIAL NEURAL NETWORKS
RAILEANU,
 
Ana-Maria; CARSTOIU Dorin
 Abstract: In this study we set purpose to prove the high degreeof security that can be oferred by using ANN-s as the base of abiometric system. Neural networks , based upon a feed-forward architecture are being used in problem solving as
 
universalapproximators in concrete associations such as classification(including nonlinear separable classes), prediction,compression.The error backpropagation algorithm has beenused to train the multi-layered perceptron network. The resultsshowed that errors can be reduced by increasing the number of learning epochs and the number of input charaters up until a point and that, of course, there is room for improvement.Key words: neural networks, biometrics,character recognition
1.
 
INTRODUCTION
 A biometric system is essentially a patternrecognition system, which makes a personal identification bydetermining the authenticity of a specific physiological orbehavioral characteristics possessed by the user. Patternrecognition , as a branch of artificial intelligence is aimingidentification of similarity relationships between abstractrepresentations of objects or phenomena., for recognition is toclassify data entry as belonging to certain classes usingclassification criteria based on information previously built.An important issue in designing a practical system isto determine how an individual is identified. Biometrics datesback to the ancient Egyptians, who measured people to identitythem. Keeping to the basics, we submit to your attention theideea of identifying someone by his handwriting. Every personwho desires to entry a secured perimeter is obliged to write arandom text , which will be compared with previously takensamples of his handwriting. Depending on the results ,consisting of a percentage which illustrates the similarity , theperson shall, or shall not be allowed the entry.Biometrics devices have three primary components:an automated mechanism that scans and captures a digital / analog image of a living personal characteristics; anotherhandles compression, processing, storage and comparison of image with the stored data; the third interfaces with applicationsystems.Pattern recognition is a branch of artificialintelligence aiming identification of similarity relationshipsbetween abstract representations of objects or phenomena. Forrecognition is to classify forms (= recognition) data entry(forms) as belonging to certain classes using classificationcriteria based on information previously built.This study is dealing with the first part of thebiometric system, illustrated by the useage of artificial neuralnetworks which are, as their name indicates, computationalnetworks which attempt to simulate the networks of nerve cell(neurons) of the biological central nervous system.The neuralnetwork is in fact a novel computer architecture and a novelalgorithmization architecture relative to conventionalcomputers. It allows using very simple computationaloperations (additions, multiplication and fundamental logicelements) to solve complex, mathematically ill-definedproblems. A conventional algorithm will employ complex setsof equations, and will apply to only a given problem andexactly to it. The ANN will be computationally andalgorithmically very simple and it will have a self-organizingfeature to allow it to hold for a wide range of problems.A multitude of types of neural networks have beenproposed over time. Actually, neural networks have been sointensely studied (for example by IT engineers, electronicsengineers, biologists and psychologists) that they have receiveda variety of names. Scientists refer to them as „Artificial NeuralNetworks”, „Multi-Layered Perceptron” and „Parallel-Distributed Processors”. Despite this fact ,there is a small groupof classical networks which are used, mainly networks whichuse the BackPropagation algorithm, Hopfield networks,„Competitive” networks and those networks which use Spikyneurons.Knowledge can be classified by its degree of generality. At the basic level are signals which contain usefuldata as well as parasite elements (noise). Data consists of elements which can raise a potential interest. We must considerthe fact that processed data lead to obtaining information,driven by a specific interest. When a piece of information issubjected to a certain specialization then we face knowledge.Knowledge based systems, depending on the purpose and ontheir type, can rationalize on their own, having as a startingpoint signals, data, pieces of information; further more, in theseknowledge based systems we may be dealing withmetaknowledge.Here are a number of resons why we should studyneural networks:1.
 
They are a viable alternative to the computationalparadigm based upon the utilization of a formalmodel and the design of algorithm whose behaviourdo not alter during use.2.
 
They incorporate results which derive from differentfields of study, for the purpose of ontaining simplecalculus architectures.3.
 
They model human intelligence, helping us to betterunderstand the way the human brain works.4.
 
They can offer a better rejection of errors, being ableto have a good performance even if the data entrieshave been flawed.ANNs have multiple representational forms but the mostcommon are the mathematical. For each artificial neuron, themathematical form consists of a function
()
gx
of the inputvector
 x
, where
x (x1; x2; ... ; xi)
=
. Each input
i
 x
 is weighted according to its weight
12
w (w;w; ... ;w)
i
=
. K is the post-processingfunction that is finally applied. This results in the followingequation for a single neuron:
()()
iii
gxKwx
=
(1)When interpreting the results we must take intoconsideration the fact that in handwritten text we face thevariability due to the loss of synchronism between the musclesof the hand as well as the variation of one’s style due to severalfactors, including but not limited to: education, mood,etc.Reading handwriting is a very difficult task consideringthe diversities that exist in ordinary penmanship. However,progress is being made. Early devices, using non-reading inksto define specifically-sized character boxes, read constrainedhandwritten entries.
 
IntelliSec – The 1st International Workshop on Intelligent Security Systems11-24th November 2009, Bucharest, Romania
2.
 
RELATED WORK
Obviously, if one is to solve a set of diffrential equations,one would not use an ANN. But problems of recognition,filtering and control would be problems suited for ANNs. Asalways, no tool or discipline can be expected to do it all. Andthen, ANNs are certainly at their infancy. They started in the1950s; and widespread interest in them dates from the early1980s. So, all in all, ANNs deserve our serious attention.One field that has developed from Character Recognitionis Optical Character Recognition (OCR). OCR is used widelytoday in the post offices, banks, airports, airline offices, andbusinesses. Address readers sort incoming and outgoing mail,check readers in banks capture images of checks for processing,airline ticket and passport readers are used from accounting forpassenger revenues to checking database records, and formreaders are used to read and process up to 5,800 forms per hour.OCR software is also used in scanners and faxes that allow theuser to turn graphic images of text into editable documents.Newer applications have even expanded outside the limitationsof just characters. Eye, face, and fingerprint scans used in high-security areas employ a newer kind of recognition.Optical Character Recognition has even advancedinto a newer field - Handwritten Recognition, which of courseis also based on the simplicity of Character Recognition.The basic principles of the artificial neural networks(ANNs) were first formulated by McCulloch and Pitts in 1943,in terms of five assumptions, as follows:1.
 
The activity of a neuron (ANN) is all-or-nothing.2.
 
A certain fixed number of synapses larger than 1 mustbe excited within a given interval of neural addition fora neuron to be excited.3.
 
The only significant delay within the neural system isthe synaptic delay.4.
 
The activity of any inhibitory synapse absolutelyprevents the excitation of the neuron at that time.5.
 
The structure of the interconnection network does notchange over time.The Hebbian Learning Law (Hebbian Rule) due to DonaldHebb (1949) is also a widely applied principle. The HebbianLearning Law states that:”When an axon of cell A is near-enough to excite cell B and when it repeatedly and persistentlytakes part in firing it, then some growth process or metabolicchange takes place in one or both these cells such that theefficiency of cell A [Hebb, 1949] is increased" (i.e. - theweight of the contribution of the output of cell A to the abovefiring of cell B is increased).1.
 
Historically, the earliest ANNs are The Perceptron,proposed by the psychologist Frank Rosenblatt(Psychological Review, 1958).2.
 
The Artron (Statistical Switch-based ANN) due to R.Lee (1950s).3.
 
The Adaline (Adaptive Linear Neuron, due to B.Widrow, 1960). This artificial neuron is also known asthe ALC (adaptive linear combiner), the ALC being itsprincipal component. It is a single neuron, not anetwork.4.
 
The Madaline (Many Adaline), also due to Widrow(1988). This is an ANN (network) formulation based onthe Adaline above.5.
 
The Back-Propagation network - A multi-layerPerceptron-based ANN, giving an elegant solution tohidden-layers learning [Rumelhart et al., 1986 andothers].6.
 
The Hopfield Network, due to John Hopfield (1982).7.
 
The Counter-Propagation Network [Hecht-Nielsen,1987] | where Kohonen's Self-Organizing
 
Mapping(SOM) is utilized to facilitate unsupervisedlearning(absence of a „teacher").The other networks, such as ART, Cognitron,LAMSTAR, etc. incorporate certain elements of thesefundamental networks, or use them as building blocks, usuallywhen combined with other decision elements,statistical or deterministic and with higher-level controllers.The Adaptive Resonance Theory (ART) wasoriginated by Carpenter and Grossberg (1987a) for the purposeof developing artiffcial neural networks whose manner of performance, especially (but not only) in pattern recognition orclassification tasks, is closer to that of the biological neuralnetwork (NN).Since the purpose of the ART neural network is toclosely approximate the biological NN, the ART neuralnetwork needs no „teacher" but functions as anunsupervised self-organizing network. Its ART-I version dealswith binary inputs. The extension of ART-I known as ART-II[Carpenter and Grossberg, 1987b] deals with both analogpatterns and with patterns represented by different levels of grey.The cognitron, as its name implies, is a network designed mainly with recognition of patterns in mind. To dothis, the cognitron network employs both inhibitory andexcitory neurons in its various layers. It was first devised byFukushima (1975), and is an unsupervised network such that itresembles the biological neural network in that respect.The LAMSTAR (LArge Memory STorage AndRetrieval) network is not a specific network but a system of networks for storage, recognition, comparison and decision thatin combination allow such storage and retrieval to beaccomplished.
 
3.
 
POSSIBILITY OF HARDWAREIMPLEMENTATION
 
Most of the physical implementation of neural systems arebased on the mathematical model due of McCulloch and Pitts(1943). The main issues raised by the synthesis of artificialsystems which simulate actual behavior are the number andnature of biological real features, starting with the connectivitymatrix of elements whose size increases with the square of theirnumber, and processing time, which must be independent of thesize of the network.Complex neural networks produce temporal variations of network parameters and can perform some more sophisticatedmathematical operations than mere summary of the signals.Consequently, elements of processing are organized in severallayers of input, output and one or more hidden layers. ANNphysical implementation should incorporate as many aspects of physiological, and operational characteristics of themathematical models as possible.We can highlight three main physical modeling of neuronsand default artificial neural networks,considering theadvantages and limitations of technology:a) Analog modeling of the amplifier gain control and resistivesynapses;b) ANN modeling with semi-parallel shift registers;c) electro-optical modeling of ANN.Main trends in ANN approach form of an integratedcircuit semiconductor (IC) are to increase density componentsof the circuit per unit area. The limited level of integration isdetermined by the matrix of connectivity whose size increaseswith the square of the number of the dynamic processing units(neurons).In essence, the matrix of connectivity is phisically madethrough a network of perforations arranged in an insulatingmaterial,in which conductive material is injected (usuallypolycrystalline silicon). On the two sides of the insulatingmaterial are secured two sets of metal interconnections, whichcorrespond to inputs (dendrites) and outputs (axon) of amplifiers (neurons). Physically achieving positive andnegative synapses is made by doubling each neuron. The value
 
IntelliSec – The 1st International Workshop on Intelligent Security Systems11-24th November 2009, Bucharest, Romaniaof each resistor is determined by the section of the hole andcorresponding inverse synaptic efficacy. The circuit integrationcan be achieved both by standard bipolar technology, themodest level of integration, as well as CMOS technology.In order to avoid the difficulty brought about by theimposed compromise between the high level of connectivityand synaptic contacts inaccessibility we could use a scheme of neural network implemented using CCD (Charge CoupledDevice) type microelectronic circuits beacouse CCD shiftregisters can store discrete groups of electrons in well definedposition, which can then be quickly moved by applying externalpotential, keeping their local value.Fig. 1. A neural network architecture implemented with CCDshift register typeThe circuit shown in the previous figure mostly avoids thelimitations imposed by the high degree of connectivitycharacterized by a synaptic matrix easily accessible andmodifiable. On the other hand, the arrangement proposed ispartially sacrificing the parallelism and the asynchronous signalprocessing which gave rise to the original idea. It is true thatrelatively high speed CCD circuits partially compensates non-parallel data processing. Currently, there can be implementedshift register containing up to 2000 CCD circuits operating atfrequencies of 10MHz.
 
4.
 
PROPOSED SOFTWAREARCHITECTURE
In order to obtain positive results a network-typefeedforward was chosen for implementing the integratedsystem,consisting of 150 neurons in the input layer, 250neurons in the hidden layer and 16 neurons in the output layer,which represent the characters of the alphabet in binary code,each uniquely represented on 16 bits.Inserting hidden layers enhances the capacity of representationof the feed forward networks but raise difficulties in terms of learning as „delta” type algorithmscan not be directly applied. This was one of the main reasonsfor the stagnation of the development of feedforward networkswith supervised learning between 1969 (when Papert andMinsky highlighted the limits of single-level networks) and1985 (when the BackPropagation algorithm, developed inparallel by several researchers became known).In determining the number of neurons in each layer thefollowing had been taken into account:-
 
Both entry level and output level should have asmany units as needed to represent input datarespectively output.
-
 
The number of hidden units should be just enoughto solve the problem, but not higher than necessary.The number of hidden units is based either ontheoretical results concerning the capacity of representation of the architecture (such as the caseof the current chosen network) or heuristic rules(eg. for a network with N input units and M outputunits and one hidden layer we can choose for thelast one the size as
 M
).
 
If the number of hidden layer neurons is too small, thenetwork fails to form an adequate internal representation of data training and thus the classification error will be high. Witha number too large, the network learns very well the trainingdata but it turns out to be incapable of obtaining a goodgeneralization obtaining high levels of error for the test data.Therefore the input vector consists of 150 partsrepresenting the matrix elements ,size 10x15 pixel binaryrepresentation. The matrix size was chosen considering theaverage values of the characters represented, with a minimumof noise introduced.Used for network learning algorithm is the well-knownBackpropagation, proposed in 1986 by Rumelhart, Hinton andWilliams for setting weights and hence for the training of multi-layer perceptrons.Here's how learning arises: it initializes the network’sweights with some random numbers, usually between -1 and 1.The next step consists of applying the set of entry data andcalculating the exit (this step is called "step forward"). Thecalculation brings one result completely different from ourtarget, because all the weights had had random values. At thispoint the error of each neuron is calculated, which usuallymeets the formula:
Target - Effective
output. This error is thenused to modify the weights so that the error has becomeincreasingly smaller. The process is repeated until when theerror is minimal.The learning rate (
η
) is to speed up or slow down thelearning process, if appropriate. We have decided upon alearning rate of 150 as being the appropiate for this system, butwe are
 
allowing the user to modify it in the range of 1 to 200and configure it according to his own needs.The detection of the symbols is a very important part of the program. This is based on the premise that we are dealingonly with black and white images, where white RGB(255,0,0,0) = space and black RGB (255,255,255,255) =character bitmap image, with any resolution. It is alsoconsidered that the image contains only characters, any anotherform of existence (line of the table, edge, etc..) are considerednoise.Fig. 2. Neural Network Chosen ArchitectureWe should mention also that each set of training consistsof an image and a text file containing the desired output.Concerning the user’s ability to customize the application,we have granted the possibility to choose an activationfunction. To fully understand the mechanism we shouldacknowledge that in biologically-inspired neural networks, theactivation function is usually an abstraction representing therate of action potential firing in the cell. In its simplest form,this function is binary-that is, either the neuron is firing or not.The function looks like
φ(
v
i
) =
(
v
i
), where
is the Heaviside

Activity (3)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads
mohammadthalif liked this

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->