IntelliSec – The 1st International Workshop on Intelligent Security Systems11-24th November 2009, Bucharest, Romania
Obviously, if one is to solve a set of diffrential equations,one would not use an ANN. But problems of recognition,filtering and control would be problems suited for ANNs. Asalways, no tool or discipline can be expected to do it all. Andthen, ANNs are certainly at their infancy. They started in the1950s; and widespread interest in them dates from the early1980s. So, all in all, ANNs deserve our serious attention.One field that has developed from Character Recognitionis Optical Character Recognition (OCR). OCR is used widelytoday in the post offices, banks, airports, airline offices, andbusinesses. Address readers sort incoming and outgoing mail,check readers in banks capture images of checks for processing,airline ticket and passport readers are used from accounting forpassenger revenues to checking database records, and formreaders are used to read and process up to 5,800 forms per hour.OCR software is also used in scanners and faxes that allow theuser to turn graphic images of text into editable documents.Newer applications have even expanded outside the limitationsof just characters. Eye, face, and fingerprint scans used in high-security areas employ a newer kind of recognition.Optical Character Recognition has even advancedinto a newer field - Handwritten Recognition, which of courseis also based on the simplicity of Character Recognition.The basic principles of the artificial neural networks(ANNs) were first formulated by McCulloch and Pitts in 1943,in terms of five assumptions, as follows:1.
The activity of a neuron (ANN) is all-or-nothing.2.
A certain fixed number of synapses larger than 1 mustbe excited within a given interval of neural addition fora neuron to be excited.3.
The only significant delay within the neural system isthe synaptic delay.4.
The activity of any inhibitory synapse absolutelyprevents the excitation of the neuron at that time.5.
The structure of the interconnection network does notchange over time.The Hebbian Learning Law (Hebbian Rule) due to DonaldHebb (1949) is also a widely applied principle. The HebbianLearning Law states that:”When an axon of cell A is near-enough to excite cell B and when it repeatedly and persistentlytakes part in firing it, then some growth process or metabolicchange takes place in one or both these cells such that theefficiency of cell A [Hebb, 1949] is increased" (i.e. - theweight of the contribution of the output of cell A to the abovefiring of cell B is increased).1.
Historically, the earliest ANNs are The Perceptron,proposed by the psychologist Frank Rosenblatt(Psychological Review, 1958).2.
The Artron (Statistical Switch-based ANN) due to R.Lee (1950s).3.
The Adaline (Adaptive Linear Neuron, due to B.Widrow, 1960). This artificial neuron is also known asthe ALC (adaptive linear combiner), the ALC being itsprincipal component. It is a single neuron, not anetwork.4.
The Madaline (Many Adaline), also due to Widrow(1988). This is an ANN (network) formulation based onthe Adaline above.5.
The Back-Propagation network - A multi-layerPerceptron-based ANN, giving an elegant solution tohidden-layers learning [Rumelhart et al., 1986 andothers].6.
The Hopfield Network, due to John Hopfield (1982).7.
The Counter-Propagation Network [Hecht-Nielsen,1987] | where Kohonen's Self-Organizing
Mapping(SOM) is utilized to facilitate unsupervisedlearning(absence of a „teacher").The other networks, such as ART, Cognitron,LAMSTAR, etc. incorporate certain elements of thesefundamental networks, or use them as building blocks, usuallywhen combined with other decision elements,statistical or deterministic and with higher-level controllers.The Adaptive Resonance Theory (ART) wasoriginated by Carpenter and Grossberg (1987a) for the purposeof developing artiffcial neural networks whose manner of performance, especially (but not only) in pattern recognition orclassification tasks, is closer to that of the biological neuralnetwork (NN).Since the purpose of the ART neural network is toclosely approximate the biological NN, the ART neuralnetwork needs no „teacher" but functions as anunsupervised self-organizing network. Its ART-I version dealswith binary inputs. The extension of ART-I known as ART-II[Carpenter and Grossberg, 1987b] deals with both analogpatterns and with patterns represented by different levels of grey.The cognitron, as its name implies, is a network designed mainly with recognition of patterns in mind. To dothis, the cognitron network employs both inhibitory andexcitory neurons in its various layers. It was first devised byFukushima (1975), and is an unsupervised network such that itresembles the biological neural network in that respect.The LAMSTAR (LArge Memory STorage AndRetrieval) network is not a specific network but a system of networks for storage, recognition, comparison and decision thatin combination allow such storage and retrieval to beaccomplished.
POSSIBILITY OF HARDWAREIMPLEMENTATION
Most of the physical implementation of neural systems arebased on the mathematical model due of McCulloch and Pitts(1943). The main issues raised by the synthesis of artificialsystems which simulate actual behavior are the number andnature of biological real features, starting with the connectivitymatrix of elements whose size increases with the square of theirnumber, and processing time, which must be independent of thesize of the network.Complex neural networks produce temporal variations of network parameters and can perform some more sophisticatedmathematical operations than mere summary of the signals.Consequently, elements of processing are organized in severallayers of input, output and one or more hidden layers. ANNphysical implementation should incorporate as many aspects of physiological, and operational characteristics of themathematical models as possible.We can highlight three main physical modeling of neuronsand default artificial neural networks,considering theadvantages and limitations of technology:a) Analog modeling of the amplifier gain control and resistivesynapses;b) ANN modeling with semi-parallel shift registers;c) electro-optical modeling of ANN.Main trends in ANN approach form of an integratedcircuit semiconductor (IC) are to increase density componentsof the circuit per unit area. The limited level of integration isdetermined by the matrix of connectivity whose size increaseswith the square of the number of the dynamic processing units(neurons).In essence, the matrix of connectivity is phisically madethrough a network of perforations arranged in an insulatingmaterial,in which conductive material is injected (usuallypolycrystalline silicon). On the two sides of the insulatingmaterial are secured two sets of metal interconnections, whichcorrespond to inputs (dendrites) and outputs (axon) of amplifiers (neurons). Physically achieving positive andnegative synapses is made by doubling each neuron. The value