Connectionist Systems

TABLE OF CONTENTS
Page No.
ABSTRACT
1. INTRODUCTION 1-7
1.1 !at is Connectionism "
1.# !at a$e Connectionist Systems
#. CONNECTIONIS% AND ARETIFICIAL LIFE &-1&
#.1 E'o()tion o* A$ti*icia( Systems
#.# A$ti*icia( Li*e
#.+ ,oa(s o* A$ti*icia( systems
+. E-OLUTION OF CONNECTIONIST S.STE%S 1/-#0
+.1 P$inci1(es Fo$ Im1(ementing Connectionist
Systems
+.# T!e ECOS *$ame2o$3
4. APPLICATIONS OF CONNECTIONIST
S.STE%S
#7-+5
4.1 Connectionist 6ase7 Decision S)11o$t
Systems an7 E81e$t Systems
4.# E'o('ing Connectionist Systems *o$ On-
(ine
CONCLUSION
BIBLIO,RAP9.
APPENDI: ; POER POINT SLIDES
ABSTRACT
The rapid proliferation of computers through all organizations makes it
necessary to look at manufacturing in a new way to meet rapidly changing
market conditions and the business environment. By using networking within
the organization manufacturing units can work at high efficiencies and make
decisions that affect their operations in a most informed way.
However, the extension of this networking to physical levels in the shop
floor needs to be given special attention since this is often the weak link in the
information chain. Organisations have also to look at the networking to the
outside world which will have a great bearing on their ability to respond to
opportunities, threats and regulatory policies impinging on their business.
C9APTER 1
INTRODUCTION
1. Int$o7)ction <-
Connectionist systems !a'e in *act 7e(i'e$e7 a non-c(assica(
e81(anation o* systematicity. It 6een so )nc(ea$ 2!et!e$ t!ese mo7e(s
act)a((y s!o2 t!at connectionism meets t!e c!a((enge" 9e$e= O)$ 'ie2 is
t!at t!e most im1o$tant $eason !as 6een o6sc)$ity in t!e conce1t o*
systematicity itse(*. . Some $esea$c!e$s c(aime7 t!at t!e systematicity
a$g)ment in *a'o$ o* t!e c(assica( a11$oac! is a t$a7itiona( one= 6)t 1$o'i7e7
no $e*e$ences to 1$e'io)s occ)$$ences= an7 as *a$ as 2e can 7ete$mine t!e$e
is no occ)$$ence o* t!e a$g)ment o$ t!e conce1t in t!e cogniti'e science
(ite$at)$e. Conse>)ent(y= 7es1ite t!e cont$o'e$sy c$eate7 6y t!e 1a1e$= t!e$e
2as sim1(y no c(ea$ conce1t o* systematicity a'ai(a6(e= an7 it 2as enti$e(y
)nc(ea$ 2!at 3in7 o* mo7e(ing= i* any= co)(7 7emonst$ate systematicity.
%ost connectionist mo7e(s t!at 2e$e s)6se>)ent(y c(aime7 to s!o2
t!at t!e systematicity c!a((enges can 6e met 7i7 not !a'e as t!ei$ 1$ima$y
e81(anato$y ta$get t!e 1!enomenon o* systematicity no 2on7e$= since t!at
1!enomenon 2as so i((-7e*ine7 to 6egin 2it!? Conse>)ent(y= s)ggestions t!at
a 1a$tic)(a$ mo7e( may !a'e im1(ications *o$ t!e systematicity 7e6ate 2e$e
)s)a((y ma7e as an a77ition to t!e main a$g)ments o* t!e 2o$3= an7
ty1ica((y 7i7 not in'o('e 7etai(e7 attention to e8act(y 2!at systematicity is
an7 2!at it 2o)(7 ta3e *o$ a connectionist mo7e( to e81(ain it. It is t!)s not
s)$1$ising t!at 7e*en7e$s o* t!e c(assica( a11$oac! $emaine7 )ncon'ince7 6y
connectionist c(aims to 6e e81(aining systematicity.
In t!is s)**ocating *og o* 'ag)eness an7 con*)sion= Ro6e$t 9a7(ey@s
2o$3 !as a$$i'e7 (i3e a g)st o* *$es! ai$. 9a7(ey !as 1ai7 'e$y c(ose attention
to e8act(y 2!at systematicity is an7 2!at it 2o)(7 ta3e *o$ connectionist
mo7e(s to ac!ie'e it. 9e !as gi'en 2!at is 1e$!a1s t!e *i$st com1$e!ensi'e=
1$ecise ana(ysis o* 2!at a goo7 n)m6e$ o* t!e most 2e((-3no2n connectionist
mo7e(s in t!is a$ea !a'e $ea((y ac!ie'e7 in t!is $ega$7. In 1a$tic)(a$= !e !as
a$g)e7 1(a)si6(y t!at 2e !a'e no $eason to 6e(ie'e t!at e8isting connectionist
mo7e(s !a'e e8!i6ite7 anyt!ing mo$e t!an 2!at !e te$ms @>)asi-
systematicity=@ 2!e$eas !)mans e8!i6it at (east @st$ong systematicity@.
e !a'e no >)a$$e(s 2it! any o* 9a7(ey@s maAo$ conc()sions. Rat!e$=
2e *in7 9a7(ey@s 2o$3 a )se*)( s1$ing6oa$7 into t!e contin)e7 in'estigation
o* !)man cogniti'e ca1acities an7 t!e 1o2e$ o* connectionist mo7e(s. In
1a$tic)(a$= 9a7(ey is to 6e commen7e7 *o$ c(a$i*ying systematicity to t!e
1oint 2!e$e it is a $e(ati'e(y st$aig!t*o$2a$7 mo7e(ing e8e$cise to
7emonst$ate 6eyon7 any >)estion t!at non-c(assica( connectionist net2o$3s
can e8!i6it an inte$esting *o$m o* systemati-city.
In t!is commenta$y 2e 2i(( 6$ie*(y 7esc$i6e one s)c! mo7e(= 2!ic!
s)ccess*)((y !an7(es 9a7(ey@s st$ong case. Be*o$e 1$ocee7ing= !o2e'e$= 2e
2is! to 1oint o)t t!at it is 6y no means c(ea$ t!at c(assica( a$c!itect)$es a$e
ca1a6(e o* e81(aining t!e act)a( em1i$ica( *acts o* systematicity. T!e$e is no
>)estion t!at c(assica( a$c!itect)$es e8!i6it systematicity o* some *o$m= an7
t!e$e is no >)estion t!at !)man cogniti'e ca1acities e8!i6it systematicity o*
some *o$mB 6)t a$e t!ese *o$ms t!e same"
Dete$mining 2!et!e$ o$ not c(assica( a$c!itect)$es e81(ain t!e act)a(
em1i$ica( *acts $e>)i$es C1D 'e$y c(ose attention to 1syc!o(ogica( 7ata to
o6tain a c(ea$ i7ea t!e e8act 2ay in 2!ic! !)man 3in7 o* systematicity t!at
is entai(e7 6y c(assica( cogniti'e a$c!itect)$es. In C'an ,e(7e$ an7 2e a$g)e7
t!at 6eca)se ste1s C1D an7 C#D !a'e not 6een ca$$ie7 o)t *o$ most as1ects
o* cognition= 2e 7o not no2 3no2 t!at c(assica( a$c!itect)$es 7o e81(ain
systematicityB *)$t!e$= in at (east one a$ea Ca *o$m o* sim1(e 7e7)cti'e
in*e$enceD c(assica( a$c!itect)$es mani*est(y *ai( to im1(ement.
1. T!e te$m @systematicity@ !as a11ea$e7 'e$y occasiona((y= 6)t 2!en it
7i7 it meant somet!ing e(se enti$e(y see= name(y 1$o7)cti'ity= is o*
co)$se 'e$y *ami(ia$ e81(ain t!e act)a( em1i$ica( *acts. %o$e gene$a((y=
c(assica( a$c!itect)$es a11ea$ to !a'e a 1$o6(em A$c!itect)$es may 6e
too sensiti'e to conte8t to e81(ain t!e em1i$ica( 1!enomena= c(assica(
a$c!itect)$es may not 6e sensiti'e eno)g!= since t!e act)a( em1i$ica(
*acts 7o in7icate some meas)$e o* 2!at mig!t 6e t!o)g!t o* as conte8t-
sensiti'ity. T!e )1s!ot is t!at 2!i(e it is an inte$esting o1en >)estion
2!et!e$ connectionism can e81(ain= in 7etai(= t!e em1i$ica( *acts o*
systematicity= it is a(so an inte$esting o1en >)estion 2!et!e$ c(assica(
a$c!itect)$es can 7o soB *)$t!e$= any *ai()$e o* connectionism in t!is
$ega$7 7oes not a)tomatica((y (en7 s)11o$t to t!e c(assica( com1etito$.
1.1 !at is connectionism"
Connectionism= A$ti*icia( Li*e= an7 Dynamica( Systems a$e a((
a11$oac!es to cognition 2!ic! a$e $e(ati'e(y ne2 an7 !a'e 6een c(aime7 to
$e1$esent s)c! 1a$a7igm s!i*ts. E)st !o2 7ee1 t!e s!i*ts a$e $emains an o1en
>)estion= 6)t eac! o* t!ese a11$oac!es ce$tain(y seems to o**e$ no'e( 2ays o*
7ea(ing 2it! 6asic >)estions in cogniti'e science.
T!e meta1!o$ o* t!e 6$ain as a 7igita( com1)te$ 2as eno$mo)s(y
im1o$tant in t!ese t!eo$ies. Among ot!e$ t!ings= as t!e cogniti'e
1syc!o(ogist !as U($ic Neisse$ 1ointe7 o)t= t!e com1)te$ !e(1e7 to
$ationa(iFe t!e st)7y o* cognition itse(* 6y 7emonst$ating t!at it 2as 1ossi6(e
to st)7y cognition in an e81(icit= *o$ma( manne$ Cas o11ose7 to t!e
6e!a'io$a( 1syc!o(ogists o* t!e ea$(ie$ e$a 2!o a$g)e7 t!at inte$na(
1$ocesses o* t!o)g!t 2e$e not 1$o1e$ o6Aects o* st)7yD.
T!e c(ass o* mo7e(s 2!ic! *a((s )n7e$ t!e connectionist )m6$e((a is
(a$ge an7 7i'e$se. B)t a(most a(( mo7e(s s!a$e ce$tain c!a$acte$istics.
P$ocessing is ca$$ie7 o)t 6y a C)s)a((y (a$geD n)m6e$ o* C)s)a((y 'e$y sim1(eD
1$ocessing e(ements. T!ese e(ements= ca((e7 no7es o$ )nits= !a'e a 7ynamics
2!ic! is $o)g!(y ana(ogo)s to sim1(e ne)$ons. Eac! no7e $ecei'es in1)t
C2!ic! may 6e e8citato$y o$ in!i6ito$yD *$om some n)m6e$ o* ot!e$ no7es=
$es1on7s to t!at in1)t acco$7ing to a sim1(e acti'ation *)nction= an7 in t)$n
e8cites o$ in!i6its ot!e$ no7es to 2!ic! it is connecte7. Detai(s 'a$y ac$oss
mo7e(s= 6)t most a7!e$e to t!is gene$a( sc!eme.
In t!e net2o$3= t!e tas3 is to ta3e 'is)a( in1)t an7 $ecogniFe 2o$7sG
in ot!e$ 2o$7s= to $ea7. T!e$e a$e se'e$a( 3ey c!a$acte$istics 2o$t! noting.
Fi$st= Cacti'ationD o* t!e )nits is o*ten non(inea$= an7 t!is non(inea$ity !as
'e$y im1o$tant conse>)ences *o$ 1$ocessing. Among ot!e$ t!ings= t!e
non(inea$ity a((o2s t!e systems to $es1on7 )n7e$ ce$tain ci$c)mstances in a
7isc$ete= 6ina$y-(i3e manne$= yie(7ing c$is1 catego$ica( 6e!a'io$. In ot!e$
ci$c)mstances= t!e system is ca1a6(e o* g$a7e7= contin)o)s $es1onses.
Secon7= 2!at t!e system H3no2sI is= to a (a$ge e8tent= ca1t)$e7 6y t!e
1atte$n o* connectionsG2!o ta(3s to 2!omGas 2e(( as t!e 2eig!ts
associate7 2it! eac! connection C2eig!ts se$'e as m)(ti1(ie$sD. T!i$7= $at!e$
t!an )sing sym6o(ic $e1$esentations= t!e 'oca6)(a$y o* connectionist systems
consists o* 1atte$ns o* acti'ations ac$oss 7i**e$ent )nits. Fo$ e8am1(e= to
1$esent a 2o$7 as a stim)()s to a net2o$3= 2e 2o)(7 $e1$esent it as a 1atte$n
o* acti'ations ac$oss a set o* in1)t )nits. T!e e8act c!oice o* $e1$esentation
mig!t 'a$y 7$amatica((yB at one e8t$eme= a 2o$7 co)(7 6e $e1$esente7 6y a
sing(e= 7e7icate7 in1)t )nit Ct!)s acting 'e$y m)c! (i3e an atomic sym6o(DB
at t!e ot!e$ e8t$eme= t!e enti$e ensem6(e o* in1)t )nits mig!t 1a$tici1ate in
t!e $e1$esentation= 2it! 7i**e$ent 2o$7s !a'ing 7i**e$ent 1atte$ns o*
acti'ation ac$oss a s!a$e7 set o* )nits. ,i'en t!e im1o$tance o* t!e 2eig!te7
connections in t!ese mo7e(s= a 3ey >)estion is= !o 7ete$mines t!e 2eig!ts"
P)t in mo$e t$a7itiona( te$ms= !o 1$og$ams t!e net2o$3s" In ea$(y
mo7e(s= t!e connecti'ity 2as 7one 6y !an7 Can7 t!is $emains t!e case *o$
2!at a$e sometimes ca((e7 Hst$)ct)$e7 connectionist mo7e(sD. 9o2e'e$= o*
t!e e8citing 7e'e(o1ments 2!ic! ma7e connectionism so att$acti'e to many
2as t!e 7e'e(o1ment o* (ea$ning a(go$it!ms 6y 2!ic! t!e net2o$3s co)(7 6e
H1$og$amme7I on its o2n. In ot!e$ 2o$7s= t!e net2o$3s co)(7 t!emse('es
(ea$n t!e 'a()es *o$ t!e 2eig!ts.
%o$eo'e$= t!e sty(e o* (ea$ning 2as in7)cti'e< Net2o$3s 2o)(7 6e
e81ose7 to e8am1(es o* a ta$get 6e!a'io$ C*o$ e8am1(e= t!e a11$o1$iate
$es1onses to a set o* 'a$ie7 stim)(iD. T!$o)g! (ea$ning= t!e net2o$3 2o)(7
(ea$n to a7A)st t!e 2eig!ts in sma(( inc$ementa( ste1s in s)c! a 2ay t!at o'e$
time= t!e net2o$3Js $es1onse acc)$acy 2o)(7 im1$o'e.
I7ea((y= t!e net2o$3 2o)(7 a(so 6e a6(e to gene$a(iFe its 1e$*o$mance
to no'e( stim)(i= t!e$e6y 7emonst$ating t!at it !a7 (ea$ne7 t!e )n7e$(ying
*)nction 2!ic! $e(ate7 o)t1)ts to in1)ts Cas o11ose7 to me$e(y memo$iFing
t!e t$aining e8am1(esD. CIt s!o)(7 6e note7 t!at t!e ty1e o* (ea$ning
7esc$i6e7 a6o'eGso-ca((e7 Hs)1e$'ise7 (ea$ningIGis 6)t one o* a n)m6e$ o*
7i**e$ent ty1es o* (ea$ning t!at a$e 1ossi6(e in connectionist net2o$3s. Ot!e$
(ea$ning 1$oce7)$es 7o not in'o('e any 1$io$ notion o* Hco$$ect 6e!a'io$I at
a((. T!e net2o$3 mig!t (ea$n instea7= *o$ e8am1(e= t!e co$$e(ationa( st$)ct)$e
)n7e$(ying a set o* 1atte$ns.D No2 (et )s t)$n to some o* t!e inte$esting
1o1e$tiess o* t!e net2o$3s= as 2e(( as t!e cont$o'e$sies 2!ic! s)$o)n7 t!em.
1.# !at a$e connectionist systems <-
T!e 1)$1ose o* t!is e81(aination is to 7isc)ss t!e common ass)m1tions
o* connectionist systems t!eo$ies an7 connectionism= to e'a()ate t!e )ni>)e
st$engt!s o* eac!= an7 to e81(o$e 2!et!e$ t!ese a$e act)a((y t!e same
a11$oac! to 7e'e(o1ment. In t!e *i$st section o* o)$ cont$i6)tion to t!is
iss)e= 2e note im1o$tant commona(ities 6et2een t!e t2o 3in7s o* t!eo$ies
an7 conc()7e t!at t!e$e may 6e no in-1$inci1(e 7i**e$ences in t!e $ange o*
1!enomena t!at eac! can e81(ain. In t!e *o((o2ing sections= 2e note t!at
t!e$e a$e= nonet!e(ess= im1o$tant 7i**e$ences 6et2een t!e t2o a11$oac!es in
t!e goa(s o* t!e t!eo$ists. e a$g)e t!at t!ese 7i**e$ences a$e c$itica( an7
s)ggest t!at t!e t2o a11$oac!es a$e 'a()a6(e com1(ements to one anot!e$.
Im1o$tant(y= 6ot! stan7 to ma3e a )ni>)e cont$i6)tion to a mo$e com1(ete
t!eo$y o* 7e'e(o1ment.
T!e enemy o* my enemy is my *$ien7. T!is c!estn)t gi'es )s a
stan7a$7 6y 2!ic! to A)7ge t!e simi(a$ities o* t!e t2o 3in7s o* acco)nts.
Connectionist systems an7 connectionism acco)nts o* 7e'e(o1ment stan7 in
o11osition to t!e c(assic sym6o( system 'ie2 o* cognition. In t!e t$a7itiona(
'ie2= t!e co$e o* !)man cognition $esi7es in sym6o(ic $e1$esentations.
Re1$esentations stan7 *o$ e'ents in t!e 2o$(7 an7 a$e o1e$ate7 on 6y
inte$na( 1$ocesses t!at mani1)(ate t!ei$ 7isc$ete an7 en7)$ing *o$ms. Since
no one !as an i7ea !o2 s)c! sym6o(ic $e1$esentations mig!t 6e initia((y
*o$me7= t!eo$ies t!at ta3e t!is c(assic $e1$esentationa( stance en7 )1 as
nati'ist t!eo$ies o* 7e'e(o1ment. Bot! connectionist an7 connectionists
systems t!eo$ies o* 7e'e(o1ment 2e$e *o)n7e7 in o11osition to t!is c(assic
cogniti'ist i7ea an7 a$g)e7 instea7 t!at cognition 2as an eme$gent
1!enomenon= g$o)n7e7 in (o2e$= sim1(e$= an7 non-sym6o(ic 1$ocesses. T!)s=
in t!ei$ ea$(y 7ays= 6ot! connectionists systems an7 connectionist mo7e(s
esc!e2e7 t!e notion o* $e1$esentations.
Anot!e$ meas)$e 6y 2!ic! connectionist an7 connectionists systems
t!eo$ies a$e at (east simi(a$ is in t!e *act t!at t!ey a$e mat!ematica((y
$e(ate7. Des1ite t!e 7i**e$ent names in t!e t2o 3in7s o* acco)ntsG(ea$ning
$)(es 'e$s)s time e'o()tion e>)ations= attention 2eig!ts 'e$s)s coo1e$ati'e(y
acti'e $e1$esentations= (atent $e1$esentations 'e$s)s memo$y in1)t t!e
mat!ematica( notions 6e!in7 t!ese i7eas a$e 'e$y m)c! t!e same. In7ee7=
many connectionist mo7e(s a$e connectionists systems an7 a$e $o)tine(y
ana(yFe7 an7 c!a$acte$iFe7 in te$ms o* t!ei$ att$acto$ states. 9o2e'e$= t!e
$e(ation 6et2een connectionists systems an7 connectionist mo7e(s is not one
o* e>)i'a(ence. T!e$e a$e many connectionists systems 2!ic! a$e not
connectionist mo7e(s Can7 s!a$e none o* t!ei$ 1$o1e$tiesD an7 t!e$e a$e some
connectionist mo7e(s t!at a$e not connectionists systems. Nonet!e(ess= i* one
ta3es t!e mat!ematics as 7e*ining o* sameness o* t!eo$ies= t!ese a$e t!eo$ies
o* t!e same gene$a( c(ass.
C9APTER#
CONNECTIONIS% AND ARTIFICIAL LIFE
Pe$io7ica((y in science t!e$e a$$i'e on t!e scene 2!at a11ea$ to 6e
7$amatica((y ne2 t!eo$etica( *$ame2o$3s. C!a$acte$istic o* s)c! c!anges in
1e$s1ecti'e is t!e $ecasting o* o(7 1$o6(ems in ne2 te$ms. By a(te$ing t!e
so()tions 2!ic! 2e$e o6sc)$e7 6y 1$io$ 2ays o* t!in3ing a6o)t t!ings.
conce1t)a( 'oca6)(a$y 2e )se to t!in3 a6o)t 1$o6(ems= 2e may 7isco'e$
Connectionism= A$ti*icia( Li*e= an7 Dynamica( Systems a$e a(( a11$oac!es to
cognition 2!ic! a$e $e(ati'e(y ne2 an7 !a'e 6een c(aime7 to $e1$esent s)c!
1a$a7igm s!i*ts. E)st !o2 7ee1 t!e s!i*ts a$e $emains an o1en >)estion= 6)t
eac! o* t!ese a11$oac!es ce$tain(y seems to o**e$ no'e( 2ays o* 7ea(ing 2it!
6asic >)estions in cogniti'e science. !i(e t!e$e a$e signi*icant 7i**e$ences
among t!ese t!$ee a11$oac!es an7 some com1(ementa$ity= t!ey a(so s!a$e a
g$eat 7ea( in commonan7 an7 t!e$e many $sea$c!e$s 2!o 2o$3
sim)(taneo)s(y in a(( t!$ee.
T!e goa( o* t!is c!a1te$ 2i(( 6e *i$st= to t$ace t!e !isto$ica( $oots o*
t!ese t!$ee a11$oac!es in o$7e$ to )n7e$stan7 2!at moti'ates t!emB an7
secon7= to consi7e$ t!e 2ays in 2!ic! t!ey may eit!e$ o**e$ no'e( so()tions to
o(7 1$o6(em= o$ to e'en to $e7e*ine 2!at t!e 1$o6(ems in cognition a$e.
9isto$ica( conte8t <
!en $esea$c!e$s *i$st 6egan to t!in3 a6o)t cognition in
com1)tationa( te$ms= it seeme7 nat)$a( an7 $easona6(e to )se t!e 7igita(
com1)te$ as a *$ame2o$3 *o$ )n7e$stan7ing cognition. T!is (e7 to cogniti'e
mo7e(s 2!ic! !a7 a n)m6e$ o* c!a$acte$istics 2!ic! 2e$e s!a$e7 2it!
7igita( com1)te$s< 1$ocessing 2as ca$$ie7 o)t 6y 7isc$ete o1e$ations
e8ec)te7 in se$ia( o$7e$B t!e memo$y com1onent 2as 7istinct *$om t!e
1$ocesso$B an7 1$ocesso$ o1e$ations co)(7 6e 7esc$i6e7 in te$ms o* $)(es o*
t!e so$t t!at 2e$e *o)n7 in 1$og$amming (ang)ages.
T!ese ass)m1tions )n7e$(ay a(most a(( o* t!e most im1o$tant cogniti'e
t!eo$ies an7 *$ame2o$3s )1 t!$o)g! t!e 1/75Js= as 2e(( as a (a$ge n)m6e$ o*
contem1o$a$y a11$oac!es.
T!e meta1!o$ o* t!e 6$ain as a 7igita( com1)te$ 2as eno$mo)s(y
im1o$tant in t!ese t!eo$ies. Among ot!e$ t!ings= as t!e cogniti'e
1syc!o(ogist !as U($ic Neisse$ 1ointe7 o)t= t!e com1)te$ !e(1e7 to
$ationa(iFe t!e st)7y o* cognition itse(* 6y 7emonst$ating t!at it 2as 1ossi6(e
to st)7y cognition in an e81(icit= *o$m a( manne$ Cas o11ose7 to t!e
6e!a'io$a( 1syc!o(ogists o* t!e ea$(ie$ e$a 2!o a$g)e7 t!at inte$na(
1$ocesses o* t!o)g!t 2e$e not 1$o1e$ o6Aects o* st)7yD. B)t as $esea$c!
2it!in t!is *$ame2o$3 1$og$esse7= t!e a7'ances a(so $e'ea(e7 s!o$tcomings.
By t!e (ate 1/75Js= a n)m6e$ o* 1eo1(e inte$este7 in !)man cognition 6egan
to ta3e a c(ose$ (oo3 at some o* 6asic ass)m1tions o* t!e c)$$ent t!eo$ies.
In 1a$tic)(a$= some 1eo1(e 6egan to 2o$$y t!at t!e 7i**e$ences 6et2een
6$ains an7 7igita( com1)te$s mig!t 6e mo$e im1o$tant t!an !it!e$to
$ecogniFe7. Psyc!o(ogists s)c! as Dona(7 9e66= B)t it 2as not )nti( t!e
ea$(y 1/&5Js t!at connectionist a11$oac!es ma7e signi*icant *o$ays into
mainst$eam cogniti'e 1syc!o(ogy. T!e 2o$7-1e$ce1tion mo7e( o* Da'e
R)me(!a$t an7 Eay %cC(e((an7= 1)6(is!e7 in 1/&1= !a7 a 7$amatic im1actB
not on(y 7i7 it 1$esent a com1e((ing an7 com1$e!ensi'e acco)nt o* a (a$ge
6o7y o* em1i$ica( 7ata= 6)t (ai7 o)t a conce1t)a( *$ame2o$3 *o$ t!in3ing
a6o)t a n)m6e$ o* 1$o6(ems 2!ic! !a7 seeme7 not to *in7 $ea7y e81(anation
in t!e 9)man In*o$mation P$ocessing a11$oac!. In 1/&0= t!e 1)6(ication o*
a t2o-'o()me co((ection= 6y R)me(!a$t an7 %cC(e((an7= an7 t!e PDP
Resea$c! ,$o)1= ca((e7 Pa$a((e( Dist$i6)te7 P$ocessing< E81(o$ations in t!e
%ic$ost$)ct)$e o* Cognition= se$'e7 to conso(i7ate an7 *(es! o)t many
7etai(s o* t!e ne2 a11$oac! C'a$io)s(y ca((e7 PDP= ne)$a( net2o$3s= o$
connectionismD. Be*o$e 7isc)ssing t!e so$ts o* iss)es 2!ic! moti'ate7 t!is
a11$oac!= (et )s 6$ie*(y 7e*ine 2!at 2e mean 6y connectionism.
#.1 E'o()tion o* A$ti*icia( Systems <-
Ea$(y mo7e(s o* !)man in*o$mation 1$ocessing ty1ica((y )ti(iFe7 a 3in7
o* Htem1(ate matc!ingI a11$oac! in many tas3s= es1ecia((y t!ose *o$ 2!ic!
1atte$n $ecognition 1(aye7 an im1o$tant $o(e. it! t!e a7'ent o* t!e
com1)te$ meta1!o$= t!is a11$oac! ga'e 2ay to t!e )se o* $)(es *o$ ana(ysis.
T!ese ass)m1tions 2e$e c!a((enge7 6y a n)m6e$ o* *in7ings. Fo$
e8am1(e= it 2as *o)n7 t!at s)6AectsJ a6i(ity to 1e$cei'e a sing(e (ette$
C1$esente7 'e$y $a1i7(y on a com1)te$ sc$eenD 2as in*()ence7 6y t!e conte8t
in 2!ic! it occ)$$e7< A (ette$ co)(7 6e 6ette$ i7enti*ie7 i* it a11ea$e7 in a
$ea( 2o$7= com1a$e7 to a11ea$ing in iso(ation o$ em6e77e7 in a non-2o$7
(ette$ st$ing. In t!e 7omain o* s1eec!= it 2as 7isco'e$e7 t!at t!e 1e$ce1tion o*
am6ig)o)s so)n7s 2as a(so a**ecte7 6y t!ei$ conte8t. T!)s a so)n7 2!ic!
2as 1e$ce1t)a((y mi72ay 6et2een a H3I an7 a HgI 2o)(7 6e !ea$7 6y
(istene$s a(te$nati'e(y as a H3I i* 2!at *o((o2e7 2as H...issI a HgI i* *o((o2e7
6y H...i*tI. T!is 2as ca((e7 a Hconte8t e**ect.I F)$t!e$mo$e= t!e$e 2as
g$o2ing e'i7ence t!at t!e !)man cogniti'e system 2as a6(e t 1$ocess at
m)(ti1(e (e'e(s in 1a$a((e(= $at!e$ t!an 6eing $est$icte7 Cas 2as t!e 7igita(
com1)te$D to e8ec)ting a sing(e inst$)ction at a time.
#.1.1 9o2 to acco)nt *o$ $eg)(a$ity" R)(es o$ associations"
One o* t!e 6asic c!a((enges in cogniti'e science is !o2 to acco)nt *o$
6e!a'io$. I* !)man 6e!a'io$ 2e$e eit!e$ enti$e(y $an7om= o$ (imite7 to a
*i8e7 $e1e$toi$e o* actions 2!ic! co)(7 6e memo$iFe7= t!en t!e$e 2o)(7 6e
(itt(e to e81(ain. !at ma3es t!e 1$o6(em so inte$esting is t!at 6e!a'io$ is
1atte$ne7= an7 is o*ten 1$o7)cti'e C2e gene$a(iFe t!ese 1atte$ns to no'e(
ci$c)mstancesD.
A nat)$a( 2ay to acco)nt *o$ t!e 1atte$ne7 nat)$e o* cognition is to
ass)me t!at )n7e$(ying t!ese 6e!a'io$ is a set o* $)(es. R)(es 1$o'i7e a
com1act an7 e(egant 2ay to acco)nt *o$ t!e a6st$act an7 1$o7)cti'e nat)$e
o* 6e!a'io$. R)(es a(so o**e$ a 2ay to ca1t)$e system-(e'e( 1$o1e$tiesGt!at
is= t!e$e e8ists t!e 1ossi6i(ity t!at $)(es can inte$act in com1(e8 2ays.
T!e 1$o6(em t!at can a$ise= !o2e'e$= is t!at some !)man 6e!a'io$s
a$e o*ten on(y 1a$tia((y gene$a( an7 1$o7)cti'e. A goo7 e8am1(e o* t!is Can7
one 2!ic! !as 6een 2e((-st)7ie7 an7 t!e to1ic o* consi7e$a6(e 7e6ateD is t!e
*o$mation o* t!e 1ast tense in Eng(is! 'e$6s. In 1/&0= Da'i7 R)me(!a$t an7
Eames %cC(e((an7 1)6(is!e7 t!e $es)(ts o* a connectionist sim)(ation in
2!ic! t!ey t$aine7 a net2o$3 to 1$o7)ce t!e 1ast tense *o$ms o* Eng(is!
1$esent tense 'e$6s.
Lea$ning in'o('e7 g$a7)a((y c!anging t!e 2eig!ts in t!e net2o$3 so t!at t!e
net2o$3Js o'e$a(( 1e$*o$mance im1$o'e7. R)me(!a$t an7 %cC(e((an7
$e1o$te7 t!at t!e net2o$3 2as not on(y a6(e to maste$ t!e tas3 Ct!o)g! not
1e$*ect(yD= 6)t t!at it a(so e8!i6ite7 t!e same U-s!a1e 1e$*o$mance *o)n7 in
c!i(7$en. T!ey s)ggeste7 t!at t!is 7emonst$ate7 t!at (ang)age 1e$*o$mance
2!ic! co)(7 6e 7esc$i6e7 as $)(e-go'e$ne7 mig!t in *act not a$ise *$om
e81(icit $)(es.
T!is 1a1e$ gene$ate7 consi7e$a6(e cont$o'e$sy= 2!ic! contin)es to t!e
1$esent time. Ste'en Pin3e$ an7 A(an P$ince 2$ote a 7etai(e7 an7 !ig!(y
c$itica( $es1onse in 2!ic! t!ey >)estione7 many o* t!e met!o7o(ogica(
ass)m1tions ma7e 6y R)me(!a$t an7 %cC(e((an7 in t!ei$ sim)(ation= an7
c!a((enge7 R)me(!a$t an7 %cC(e((an7Js conc()sions. In *act= many o* Pin3e$
an7 P$inceJs c$iticisms a$e 1$o6a6(y co$$ect= an7 2e$e a77$esse7 in
s)6se>)ent connectionist mo7e(s o* 1ast tense *o$mation. A g$eat many
s)6se>)ent sim)(ations !a'e 6een ca$$ie7 o)t 2!ic! co$$ect 1$o6(ems in t!e
R)me(!a$t an7 %cC(e((an7 mo7e(.
T!ese sim)(ations !a'e in t)$n gene$ate7 an on-going 7e6ate a6o)t
ne2 iss)es. In $ecent 2$itings= Pin3e$ an7 P$ince !a'e s)ggeste7 t!at
a(t!o)g! 1e$!a1s a connectionist-(i3e system is $es1onsi6(e *o$ 1$o7)cing t!e
i$$eg)(a$s= t!e$e a$e >)a(itati'e 7i**e$ences in t!e 2ay $eg)(a$ mo$1!o(ogy is
1$ocesse7 2!ic! can on(y 6e e81(aine7 in te$ms o* $)(es. T!is !as 6ecome
3no2n as t!e H7)a( mec!anismI acco)nt. P$o1onents o* a sing(e mec!anism
a11$oac! a$g)e t!at a net2o$3 can in *act 1$o7)ce t!e *)(( $ange o*
6e!a'io$s 2!ic! c!a$acte$iFe $eg)(a$ an7 i$$eg)(a$ 'e$6s.
#.1.# Rec)$sion an7 com1ositiona(ity <-
T!e notion Hcom1ositiona(ityI is c(ose(y $e(ate7= an7 $e*e$s to t!e
st$)ct)$a( $e(ations!i1 6et2een 7i**e$ent e(ements 2!ic! can a$ise 2!en
$ec)$sion occ)$s Cas 2e(( as in ot!e$ ci$c)mstancesD. T!)s= one mig!t say
t!at t!e No)n P!$ase H...s1i7e$ on t!e c!ai$I is com1ose7 o* a no)n 2!ic! is
mo7i*ie7 6y a 1$e1ositiona( 1!$ase= 2!ic! in t)$n is com1ose7 o* a
1$e1osition *o((o2e7 6y a no)n 1!$ase= etc. T!e t$ee 7iag$am in Fig)$e # is a
ty1ica( notation )se7 6y (ing)ists to ca1t)$e s)c! st$)ct)$a( $e(ations!i1s.
In a $ec)$$ent net2o$3= t!e inte$na( Co$ H!i77enID )nits *ee76ac3 on
t!emse('es. T!is 1$o'i7es t!e net2o$3 2it! a 3in7 o* memo$y. T!e *o$m o*
t!e memo$y is not (i3e a ta1e $eco$7e$= !o2e'e$B it 7oes not (ite$a((y $eco$7
1$io$ in1)ts. Instea7= t!e net2o$3 !as to (ea$n itse(* !o2 to enco7e t!e in1)ts
in t!e inte$na( state= s)c! t!at 2!en t!e state is *e7 6ac3 it 2i(( 1$o'i7e t!e
necessa$y in*o$mation to ca$$y o)t 2!ate'e$ tas3 is 6eing (ea$ne7.
Un7e$stan7ing e8act(y !o2 s)c! net2o$3s 2o$3 is an inte$esting 1$o6(em in
its o2n $ig!t= an7 t!e$e a$e st$ong connections 6et2een t!ei$ so()tions an7
t!e 7ynamica( systems 2e 7isc)ss 6e(o2.
#.# ARTIFICIAL LIFE <-
Connectionist mo7e(s a$e 1o2e$*)( in7)ction engines. T!at is= t!ey (ea$n
6y e8am1(e= an7 )se t!e statistics o* t!ose e8am1(es to 7$i'e (ea$ning. T!e
att$action o* t!e a11$oac! is t!at a(t!o)g! (ea$ning is statistica((y 7$i'en= t!e
o)tcome o* t!at (ea$ning 1$ocess is a system 2!ose 3no2(e7ge is
gene$a(iFa6(e to no'e( instances. B)t t!e$e a$e se'e$a( $es1ects in 2!ic!
connectionist mo7e(s seem 7e*icient. F)$t!e$mo$e= t!ese 7e*iciencies a$e
simi(a$ to t!ose *o)n7 in a(most a(( a$ti*icia( inte((igence mo7e(s. 9e$e a$e
t!$ee e8am1(es o* t!e most st$i3ing s!o$tcomings.
Disem6o7ie7 inte((igence. An eno$mo)s amo)nt o* t!e cogniti'e 6e!a'io$ o*
6io(ogica( o$ganisms is tig!t(y co)1(e7 to t!e 6o7ies in 2!ic! t!e 6e!a'io$ is
mani*est. T!e 2ay 2e t!in3 a6o)t s1ace C*o$ e8am1(eD is !ig!(y 7e1en7ent on
1$o1e$ties o* o)$ 'is)a( system. T!e 2ay 2e t!in3 a6o)t o)$se('es an7 t!e
2o$(7 7e1en7s on !o2 2e e81e$ience it= an7 o)$ e81e$ience is 'ast(y
7i**e$ent *$om t!at o* a *is!= o$ a 6i$7= o$ a cat. Bot! t$a7itiona( AI mo7e(s
an7 connectionist mo7e(s en7 to igno$e t!e $o(e o* 6o7iesB t!ese mo7e(s a$e in
*act 7isem6o7ie7 *$om t!e sta$t. Passi'e 's. acti'e...t!e im1o$tance o* goa(s.
A ne)$a( net2o$3 is a 1assi'e t!ing. Be*o$e it (ea$ns anyt!ing= (e*t to its o2n
7e'ices= it 7oes not 7o anyt!ing 'e$y inte$esting. E'en a*te$ (ea$ning= *e2
connectionist mo7e(s 7is1(ay any 6e!a'io$ 2!ic! is inte$na((y gene$ate7. In
gene$a(= most connectionist an7 AI systems a$e $eacti'e. O$ i* not= t!ei$ goa(s
a$e 1$e1$og$amme7 an7 7ete$mine7 6y an o)tsi7e agency Ct!ei$
1$og$amme$D. .et e'en a 'e$y 1$imiti'e 6io(ogica( o$ganism 7is1(ays goa(-
7i$ecte7 6e!a'io$. A snai(= (e*t a(one on a ta6(e= 2i(( 2an7e$ a$o)n7 in sea$c!
o* *oo7. A 6a6y= (e*t to 1(ay in its c$i6= 2i(( s1ontaneo)s(y ma3e noises= mo'e
a$o)n7= an7 *in7 t!ings to am)se itse(*. Pe$!a1s mo$e im1o$tant(y= 2!en an
e8te$na( stim)()s 7oes im1inge= t!e 6a6yJs $eactionGan7 !o2 m)c! it
1$ocesses t!at stim)()sG7e1en7s on 2!et!e$ t!e o6Aect is inte$esting Ca
mot!e$Js *ace is 'ast(y mo$e inte$esting t!an a 6oo3D. P)t sim1(y= 6io(ogica(
o$ganisms !a'e an agen7a.
Socia( 's. asocia( cognition. A(most a(( AI an7 connectionist mo7e(s 'ie2
cognition as an essentia((y in7i'i7)a( 1!enomenon< it occ)$s 2it!in t!e s3)((.
T!)s= t!ese mo7e(s *oc)s on com1etencies s)c! as c!ess= 1$o6(em so('ing=
1atte$n $ecognition= etc. P(ace7 a(one in t!e Sa!a$a Co$ e'en a mo$e
!os1ita6(e en'i$onmentD= an in7i'i7)a( !)man 2o)(7 not 7is1(ay any o* t!e
6e!a'io$s 2e ta3e as c!a$acte$istica((y !)man C6)i(7ing com1)te$s=
t$a'e(ing to t!e sta$s= c$eating s3ysc$a1e$s= etc.D. A t$emen7o)s amo)nt o*
o)$ cogniti'e ca1acity 7e1en7s on e8te$na( a$ti*acts< t!e 1!ysica( an7 socia(
st$)ct$es 2e c$eate in o$7e$ to !e(1 )s so('e 1$o6(ems 2!ic! co)(7 not 6e
so('e7 6y one 1e$son a(one.
#.#.1. Ea$(y A$ti*icia( Li*e< -e!ic(es
In 1/&4= t!e 6io(ogist -a(entino B$aiten6e$g 1)6(is!e7 a s!o$t
monog$a1! ca((e7 -e!ic(es< E81e$iments in Synt!etic Psyc!o(ogy. T!e 6oo3
consiste7 o* 1# s!o$t Ht!o)g!t-e81e$iments=I in 2!ic! B$aiten6e$g in'ite7
t!e $ea7e$ to imagine 7i**e$ent 1$imiti'e 'e!ic(es. Eac! 'e!ic(e 2as sim1(y a
6(oc3 o* 2oo7 2it! a 1ai$ o* 2!ee(s in t!e $ea$= senso$s C2!e$e !ea7(ig!ts
2o)(7 6eD= an7 connections *$om senso$s to t!e moto$s 2!ic! 7$o'e eac!
2!ee(. T!e e8act nat)$e o* t!e senso$s an7 t!ei$ connection to t!e moto$s
'a$ie7 2it! eac! 'e!ic(e. B$aiten6e$g t!en consi7e$e7 !o2 t!e 7i**e$ent
'e!ic(es mig!t 6e!a'e 2!en 1(ace7 on a s)$*ace= 1ossi6(y 2it! ot!e$ 'e!ic(es=
an7 e81ose7 to a stim)()s= s)c! as a (ig!t so)$ce. Some o* t!e 'e!ic(es mo'e7
to2a$7 t!e (ig!t an7 t!en at t!e (ast min)te= 'ee$e7 a2ay. Ot!e$s s1e7
agg$essi'e(y to2a$7 it an7 smas!e7 in to it. Ot!e$s ci$c(e7= 2a$i(y.
T!e Conce1ts 7esc$i6ing t!e 'a$io)s 'e!ic(es 6o$e names s)c! as
HLo'e=I H9ateI= H-a()esI= HLogic.I An7 in7ee7= o6se$'e7 *$om o)tsi7e= it
2as not 7i**ic)(t to imagine t!ese 'e!ic(es as animate c$eat)$es= moti'ate7 6y
ange$= o$ a**ection= an7 e'en mo$e com1(e8 $easoning 1$ocesses. O* co)$se=
t!e ci$c)its insi7e 2e$e act)a((y >)ite sim1(e. B$aiten6e$gJs 1ointGan7 one
2!ic! 2as o* inte$est to many connectionists as 2e((Gis t!at sim1(e systems
o*ten gi'e $ise to om1(e8 6e!a'io$s. 9is monog$a1! is a 1o2e$*)( an7
g$a1!ic 2a$ning a6o)t t!e Hatt$i6)tion 1$o6(emI< It is easy to att$i6)te
mo$e t!an is 2a$$ante7 to a mec!anism= es1ecia((y i* 2e a($ea7y !a'e
$econcei'e7 notions a6o)t 2!at mec!anisms m)st )n7e$(ie a gi'en 6e!a'io$.
Resea$c!e$s *$om a 2i7e $ange o* 7isci1(ines met to e8c!ange 'ie2s on
2!at 2as 6ecoming a t!eme o* g$o2ing inte$est in a n)m6e$ o* 7i**e$ent
scienti*ic comm)nities< A$ti*icia( Li*e= o$ A(i*e= as it is mo$e 1o1)(a$(y
3no2n. A(t!o)g! t!e met!o7s an7 s1eci*ic goa(s o* t!e 7i**e$ent
s)6comm)nities 'a$ie7= t!e$e 2e$e a(so a n)m6e$ o* 1e$s1ecti'es 2!ic! 2e$e
s!a$e7.
Eme$gentism < Anot!e$ cent$a( insig!t 2!ic! )n7e$(ies m)c! o* t!e 2o$3 in
A(i*e is t!e notion o* eme$gentism< many systems !a'e 6e!a'io$s
GIeme$gent 1$o1e$tiesIG2!ic! $es)(t *$om t!e co((ecti'e 6e!a'io$ o* t!e
systemJs com1onents $at!e$ t!an *$om t!e action o* any sing(e com1onent.
F)$t!e$mo$e= t!ese 6e!a'io$s a$e o*ten )nantici1ate7 Can7 in t!e case o*
a$ti*icia( systems= )n1(anne7 o$ )n1$og$amme7D. E8am1(es o* eme$gentism
a6o)n7 in nat)$e. In7ee7= o)$ 'e$y 6o7ies a$e a com1e((ing e8am1(e. O)$
155 t$i((ion o$ so ce((s inte$act in com1(e8 2ays to 1$o7)ce co!e$ent acti'ityB
no sing(e ce((Go$ e'en sing(e g$o)1 o* ce((sG1$e7icts o$ acco)nts *o$ t!e
!ig!est (e'e( 6e!a'io$. Socia( o$ganiFations a$e anot!e$ e8am1(e. No matte$
!o2 a)toc$atic t!e socia( st$)ct)$e in'o('e7= com1(e8 inte$1e$sona(
7ynamics )s)a((y gi'e $ise to g$o)1 6e!a'io$s 2!ic! co)(7 not !a'e 6een
1$e7icte7 in a7'ance. %any A(i*e $esea$c!e$s !a'e come to t!e conc()sion
t!at eme$gentism is a !a((ma$3 o* (i*e. A$ti*icia( systems 2ic! e8!i6it
eme$gentism C1a$tic)(a$(y 6e!a'io$s 2!ic! in some 2ay $esem6(e t!ose o*
6io(ogica( (i*e*o$msD a$e es1ecia((y inte$esting. One e8am1(e o* 2!at seems
(i3e a 'e$y sim1(e system t!at 7is1(ays inte$esting eme$gentism comes *$om
2!at a$e ca((e7 ce(()(a$ a)tomata. T!ese a$e systems 2!ic! a$e 6)i(t o)t o*
C)s)a((yD t2o-7imensiona( g$i7s. At any gi'en 1oint in time= eac! ce(( in t!e
g$i7 can ass)me one o* a sma(( n)m6e$ o* statesB most sim1(y= ON CHa(i'eID
o$ OFF CH7ea7ID. At eac! tic3 o* t!e c(oc3= ce((s may c!ange t!ei$ state=
acco$7ing to a sim1(e set o* $)(es 2!ic! )s)a((y 7e1en7 on t!e states o* a
ce((Js eig!t imme7iate neig!6o$s.
E'o()tion o* A$ti*icia( Systems <-
%ost a$ti*icia( systems a$e 6)i(t 6e some e8te$na( 6eing. Bio(ogica(
systems !o2e'e$= e'o('e. F)$t!e$mo$e= 6io(ogica( c!ange !as 6ot! a $an7om
e(ement= in t!e *o$m o* $an7om genetic 'a$iationsB an7 a(so a >)asi-7i$ecte7
e(ement= inso*a$ as 'a$iants 2!ic! a$e 6ette$ a7a1te7 to t!ei$ en'i$onment
o*ten 1$o7)ce mo$e o**s1$ing= t!)s a(te$ing t!e genetic ma3e)1 o*
s)ccee7ing gene$ations. T!is insig!t 1$om1te7 com1)te$ scientist Eo!n
9o((an7= in a 1/7K monog$a1! ca((e7 A7a1tation in Nat)$a( an7 A$ti*icia(
Systems= to 1$o1ose 2!at !e ca((e7 t!e ,enetic A(go$it!m= o$ ,A. T!e ,A
2as inten7e7 to ca1t)$e some o* t!e c!a$acte$istics o* nat)$a( e'o()tion 6)t
in a$ti*icia( systems. Imagine= *o$ e8am1(e= t!at one !as a 1$o6(em 2!ic! can
6e 7esc$i6e7 in te$ms o* a set o* yesLno >)estions= an7 t!e goa( is to *in7 t!e
$ig!t set o* ans2e$s. T!e so()tion may 6e !a$7 i* t!e$e a$e inte$actions
6et2een t!e ans2e$s to 1a$tic)(a$ >)estionsB in7ee7= t!e$e may 6e m)(ti1(e
so()tions= 7e1en7ing on !o2 7i**e$ent >)estions a$e ans2e$e7. T!e ,A
2o)(7 mo7e( t!is 6y const$)cting an a$ti*icia( Hc!$omosomeBI t!is is $ea((y
A)st a 'ecto$ o* 1Js an7 5Js= eac! 6it 1osition stan7ing *o$ t!e ans2e$ C1Myes=
5MnoD to a 7i**e$ent >)estion. e 6egin 2it! a 1o1)(ation o* $an7om(y
const$)cte7 c!$omosomes.
Eac! c!$omosome $e1$esents a 1ossi6(e so()tion to t!e 1$o6(em= so 2e
can e'a()ate it to see !o2 2e(( it 7oes. T!is 7ete$mines t!e c!$omosomeJs
H*itnessI Con ana(ogy 2it! e'o()tiona$y *itnessD. A ne2 gene$ation is
const$)cte7 6y 1$e*e$entia((y $e1(icating t!ose c!$omosomes 2!ic!
1e$*o$me7 6ette$= 2!i(e at t!e same time $an7om(y s2itc!ing a sma((
1e$centage o* t!e 1Js an7 5Js. CBette$ $es)(ts a$e o6taine7 i* one a(so a((o2s
Hc$oss-o'e$I to occ)$ 6et2een c!$omosomes= so t!at 1a$t o* one c!$omosome
mig!t 6e com6ine7 2it! 1a$t o* anot!e$.D T!e ne2 gene$ation is t!en teste7=
an7 its *itness is )se7 to 7ete$mine t!e com1osition o* a t!i$7 gene$ation.
An7 so on )nti( a 6est so()tion is *o)n7.
T!e ,A !as m)c! in common 2it! nat)$a( e'o()tion. It is es1ecia((y
1o2e$*)( 2!en t!e$e a$e !ig!-o$7e$ inte$actions 6et2een t!e many 7i**e$ent
s)6-1a$ts to a 1$o6(em. A(t!o)g! t!e o$igina( ,A ma3es sim1(i*ying
ass)m1tions 2!ic! a$e >)estiona6(e Ce.g.= t!e$e is not t!e
genoty1eL1!enoty1e 7istinction *o)n7 in nat)$eD it is 2i7e(y )se7 in A(i*e=
sometimes in conA)nction 2it! ne)$a( net2o$3s. T!e ne8t sim)(ation gi'es an
e8am1(e o* t!is.
2.2 ,oa(s o* A$ti*icia( Systems <-
e note7 a6o'e t!at connectionist mo7e(s !a'e a 7ist)$6ing 1assi'ity
2!ic! is >)ite )n(i3e 6io(ogica( o$ganisms. One co)(7 const$)ct a net2o$3 in
a7'ance in o$7e$ to en7o2 it 2it! 2!at (oo3s (i3e inte$na((y-gene$ate7
6e!a'io$s= 6)t t!is is !a$7(y t!e so()tion *o)n7 in nat)$e. Instea7= many o*
t!e 6e!a'io$s 2e t!in3 o* as goa(-7i$ecte7G s)c! as t!e sea$c! *o$ *oo7=
7esi$e to mate an7 6ea$ o**s1$ing= $es1onses to 7ange$Ga$e )s)a((y
a7a1ti'e= an7 t!ey a$e t!e 1$o7)ct o* e'o()tion o* o)$ s1ecies an7 not ta)g!t
to )s. T!)s= it seems mo$e a11$o1$iate to mo7e( s)c! 6e!a'io$s )sing an
e'o()tiona$y a11$oac! s)c! as t!e ,A. One can e'en 7o t!is 2it! ne)$a(
net2o$3s.
T!e A(i*e a11$oac! is sti(( *ai$(y yo)ng= an7 m)c! o* t!e 2o$3 !as a
1$e(imina$y c!a$acte$. As a co$$ecti'e to 1$e'io)s mo7e(ing a11$oac!es=
t!e$e is no >)estion t!at t!e A(i*e 1e$s1ecti'e is 'a()a6(e. T!e em1!asis on
eme$gentism= t!e $o(e o* t!e en'i$onment= t!e im1o$tance o* an o$ganismJs
6o7y= t!e socia( nat)$e o* inte((igence= an7 t!e 1e$s1ecti'e o**e$e7 6y
e'o()tion a$e notions 2!ic! go 2e(( 6eyon7 A(i*e. F)$t!e$= 6y t$ying to
)n7e$stan7 (i*e= not necessa$i(y as it is= 6)t as it co)(7 6e= 2e 6$oa7en o)$
notion o* 2!at co)nts as inte((igent 6e!a'io$. T!is e81an7e7 7e*inition may
in t)$n gi'e )s *$es! 2ays o* t!in3ing a6o)t t!e 6e!a'io$ o* mo$e t$a7itiona(
C6io(ogica(D (i*e*o$ms. B)t it is a(so c(ea$ t!at A(i*e !as a (ong 2ay to go. As is
t$)e o* many mo7e(ing *$ame2o$3s= t!e 6$i7ge 6et2een t!e initia( HtoyI
mo7e(s an7 mo$e com1(ete an7 $ea(istic mo7e(s is a 7i**ic)(t one to c$oss.
C9APTER +
E-OLUTION OF CONNECTIONIST S.STE%S
T!e com1(e8ity an7 7ynamics o* $ea(-2o$(7 1$o6(ems= es1ecia((y in
enginee$ing an7 man)*act)$ing= $e>)i$e so1!isticate7 met!o7s an7 too(s *o$
6)i(7ing on-(ine= a7a1ti'e inte((igent systems CISD. S)c! systems s!o)(7 6e
a6(e to g$o2 as t!ey o1e$ate= to )17ate t!ei$ 3no2(e7ge an7 $e*ine t!e mo7e(
t!$o)g! inte$action 2it! t!e en'i$onment. T!is is es1ecia((y c$)cia( 2!en
so('ing AI 1$o6(ems s)c! as a7a1ti'e s1eec! an7 image $ecognition= m)(ti-
mo7a( in*o$mation 1$ocessing= a7a1ti'e 1$e7iction= a7a1ti'e on-(ine cont$o(=
inte((igent agents on t!e .
+.1 P$inci1(es Fo$ Im1(ementing Connectionist Systems
Se'en maAo$ $e>)i$ements o* t!e 1$esent IS Ct!at a$e a77$esse7 in t!e
ECOS *$ame2o$3 1$esente7 (ate$D a$e (iste7 6e(o2 N#=+K=+0=+&O<
C1D IS s!o)(7 (ea$n *ast *$om a (a$ge amo)nt o* 7ata C)sing *ast t$aining=
e.g. one-1ass t$ainingD.
C#D IS s!o)(7 6e a6(e to a7a1t inc$ementa((y in 6ot! $ea( time= an7 in an
on-(ine mo7e= 2!e$e ne2 7ata is accommo7ate7 as t!ey 6ecome
a'ai(a6(e. T!e system s!o)(7 to(e$ate an7 accommo7ate im1$ecise an7
)nce$tain *acts o$ 3no2(e7ge an7 $e*ine its 3no2(e7ge.
C+D IS s!o)(7 !a'e an o1en st$)ct)$e 2!e$e ne2 *eat)$es C$e(e'ant to t!e
tas3D can 6e int$o7)ce7 at a (ate$ stage o* t!e systemJs o1e$ation. IS
s!o)(7 7ynamica((y c$eate ne2 mo7)(es= ne2 in1)ts an7 o)t1)ts= ne2
connections an7 no7es. T!at s!o)(7 occ)$ eit!e$ in a s)1e$'ise7= o$ in
s1e$'ise7mo7e= )sing one mo7a(ity o$ anot!e$= accommo7ating 7ata=
!e)$istic $)(es= te8t= images= etc.
C4D IS s!o)(7 6e memo$y-6ase7= i.e. t!ey s!o)(7 3ee1 a $easona6(e t$ac3 o*
in*o$mation t!at !as 6een )se7 in t!e 1ast an7 6e a6(e to $et$ie'e some
o* it *o$ t!e 1)$1ose o* inne$ $e*inement= o$ *o$ ans2e$ing an e8te$na(
>)e$y.
CKD IS s!o)(7 im1$o'e contin)o)s(y C1ossi6(y in a (i*e;(ong mo7eD t!$o)g!
acti'e inte$action 2it! ot!e$ IS an7 2it! t!e en'i$onment t!ey o1e$ate
in.
C0D IS s!o)(7 6e a6(e to ana(yse t!emse('es in te$ms o* 6e!a'io)$= g(o6a(
e$$o$ an7 s)ccessB to e81(ain 2!at !as 6een (ea$ne7B to ma3e 7ecisions
a6o)t its o2n im1$o'ementB to mani*est int$os1ection.
C7D IS s!o)(7 a7e>)ate(y $e1$esent s1ace an7 time in t!ei$ 7i**e$ent sca(esB
s!o)(7 !a'e 1a$amete$s to $e1$esent s)c! conce1ts as s1atia( 7istance=
s!o$t-te$m an7 (ong-te$m memo$y= age= *o$getting= etc. Un(ess t!e
a6o'e se'en iss)es a$e a77$esse7 in t!e c)$$ent an7 t!e *)t)$e IS= it is
)n(i3e(y t!at a signi*icant 1$og$ess is ma7e in a$eas= s)c! as a7a1ti'e
s1eec! $ecognition an7 (ang)age ac>)isition= a7a1ti'e inte((igent
1$e7iction an7 cont$o( systems= inte((igent agent systems= mo6i(e
$o6ots= 'is)a( monito$ing systems= m)(ti-mo7a( in*o$mation
1$ocessing= an7 many mo$e.
+.# T!e ECOS *$ame2o$3
E'o('ing connectionist systems CECOSD a$e systems t!at e'o('e in time
t!$o)g! inte$action 2it! t!e en'i$onment. T!ey !a'e some Cgenetica((yD 1$e-
7e*ine7 1a$amete$s C3no2(e7geD 6)t t!ey a(so (ea$n an7 a7a1t as t!ey
o1e$ate. In cont$ast 2it! t!e e'o()tiona$y systems t!ey 7o not necessa$i(y
c$eate co1ies o* in7i'i7)a(s an7 se(ect t!e 6est ones *o$ t!e *)t)$e. T!ey
eme$ge= e'o('e= 7e'e(o1= )n*o(7 t!$o)g! innateness an7 (ea$ning= an7
t!$o)g! c!anging t!ei$ st$)ct)$e in o$7e$ to 6ette$ $e1$esent 7ata . ECOS
(ea$n in an on-(ine an7 a 3no2(e7ge;6ase7 mo7e%ODEt)$e= 6)t $at!e$ a
mo7)(a$= Ho1enI st$)ct)$e.
T!e main 1a$ts o* ECOS a$e 7esc$i6e7 6e(o2.
C1D Feat)$e se(ection 1a$t. It 1e$*o$ms *i(te$ing o* t!e in1)t in*o$mation=
*eat)$e e8t$action an7 *o$ming t!e in1)t 'ecto$s. T!e n)m6e$ o*
in1)ts C*eat)$esD can 'a$y *$om e8am1(e to e8am1(e *$om t!e in1)t
7ata st$eam *e7 to t!e ECOS.
C#D P$esentation an7 $e1$esentation Cmemo$yD 1a$t= 2!e$e in*o$mation
C1atte$nsD a$e sto$e7. It is a m)(ti-mo7)(a$= e'o('ing st$)ct)$e o* NN%
o$ganise7 in s1atia((y 7ist$i6)te7 g$o)1sB *o$ e8am1(e one mo7)(e can
$e1$esent t!e 1!onemes in a s1o3en (ang)age Cone NN $e1$esenting
one c(ass 1!onemeD.
C+D 9ig!e$-(e'e( 7ecision 1a$t t!at consists o* se'e$a( mo7)(es= eac! ta3ing
7ecision on a 1a$tic)(a$ 1$o6(em Ce.g.= 1!oneme= 2o$7= conce1tD. T!e
mo7)(es $ecei'e *ee76ac3 *$om t!e en'i$onment an7 ma3e 7ecisions
a6o)t t!e *)nctioning an7 t!e a7a1tation o* t!e 2!o(e ECOS.
C4D Action mo7)(es= t!at ta3e t!e o)t1)t *$om t!e 7ecision mo7)(es an7
1ass o)t1)t in*o$mation to t!e en'i$onment.
CKD Se(*-ana(ysis= an7 $)(e e8t$action mo7)(es. T!is 1a$t e8t$acts
com1$esse7 a6st$act in*o$mation *$om t!e $e1$esentation mo7)(es an7
*$om t!e 7ecision mo7)(es in 7i**e$ent *o$ms o* $)(es= a6st$act
associations= etc.
+.#.1 P$inci1(es O* F)nctioning O* Connectionist Systems
T!e ECOS *)nctioning is 6ase7 on t!e *o((o2ing gene$a( 1$inci1(es<
C1D ECOS e'o('e inc$ementa((y in an on-(ine= !y6$i7= a7a1ti'e
s)1e$'ise7L)ns)1e$'ise7 mo7e t!$o)g! accommo7ating mo$e an7
mo$e e8am1(es 2!en t!ey 6ecome 3no2n *$om a contin)o)s in1)t
7ata st$eam. D)$ing t!e o1e$ation o* ECOS t!e !ig!e$-(e'e( 7ecision
mo7)(e may acti'ate an a7a1tation 1$ocess t!$o)g! t!e a7a1tation
mo7)(e.
C#D ECOS a$e memo$y-6ase7 an7 sto$e e8em1(a$s C1$ototy1es= $)(esD t!at
$e1$esent g$o)1s o* 7ata *$om t!e 7ata st$eam. Ne2 in1)t 'ecto$s a$e
sto$e7 in t!e NN%s 6ase7 on t!ei$ simi(a$ity to 1$e'io)s(y sto$e7 7ata
6ot! on t!e in1)t an7 t!e 7esi$e7 o)t1)t in*o$mation. A no7e in an
NN% is c$eate7 an7 7esignate7 to $e1$esent an in7i'i7)a( e8am1(e i* it
is signi*icant(y 7i**e$ent *$om t!e 1$e'io)s(y )se7 e8am1(es C2it! a
(e'e( o* 7i**e$entiation set t!$o)g! 7ynamic 1a$amete$sD. Lea$ning is
6ase7 on (oca((y t)ne7 e(ements *$om t!e ECOS st$)ct)$e t!)s ma3ing
t!e (ea$ning 1$ocess *ast *o$ $ea(-time 1a$a((e( im1(ementation. T!$ee
2ays to im1(ement (oca( (ea$ning in a connectionist st$)ct)$e a$e
1$esente7.
C+D T!e$e a$e t!$ee (e'e(s at 2!ic! ECOS a$e *)nctiona((y an7 st$)ct)$a((y
7e*ine7<
CaD Pa$amete$ CgeneD (e'e(= i.e. a c!$omosome contains genes t!at
$e1$esent ce$tain 1a$amete$s o* t!e 2!o(e systems= s)c! as< ty1e o* t!e
st$)ct)$e CconnectionsD t!at 2i(( 6e e'o('e7B (ea$ning $ateB *o$getting
$ateB siFe o* a NN%B NN% s1ecia(isation= t!$es!o(7s t!at 7e*ine
simi(a$ityB e$$o$ $ate t!at is to(e$ate7= an7 many mo$e. T!e 'a()es o*
t!e genes a$e $e(ati'e(y sta6(e= 6)t can 6e c!ange7 t!$o)g! genetic
o1e$ations= s)c! as m)tation o* a gene= 7e(etion an7 inse$tion o* genes
t!at a$e t$igge$e7 6y t!e se(* ana(ysis mo7)(e as a $es)(t o* t!e o'e$a((
1e$*o$mance o* t!e ECOS.
b! Re1$esentation Csyna1ticD (e'e(= t!at is t!e in*o$mation containe7 in t!e
connections o* t!e NN%. T!is is t!e (ong-te$m memo$y o* t!e system
2!e$e e8em1(a$s o* 7ata a$e sto$e7. T!ey can 6e eit!e$ $et$ie'e7 to
ans2e$ an e8te$na( >)e$y= o$ can 6e )se7 *o$ inte$na( ECOS
$e*inement.
c! Be!a'io)$a( Cne)$ona( acti'ationD (e'e(= t!at is t!e s!o$t-te$m
acti'ation 1atte$ns t$igge$e7 6y in1)t stim)(i. T!is (e'e( 7e*ines !o2
2e(( t!e system is *)nctioning in t!e en7.
C4D ECOS e'o('e t!$o)g! (ea$ning Cg$o2ingD= *o$getting C1$)ningD= an7
agg$egation= t!at a$e 6ot! 7e*ine7 at a genetic (e'e( an7 a7a1te7
7)$ing t!e (ea$ning 1$ocess. ECOS a((o2 *o$< c$eatingLconnecting
ne)$onsB $emo'ing ne)$ons an7 t!ei$ co$$es1on7ing connections t!at
a$e not acti'e(y in'o('e7 in t!e *)nctioning o* t!e system t!)s ma3ing
s1ace *o$ ne2 in1)t 1atte$ns to 6e (ea$ne7B agg$egating no7es into
6igge$-c()ste$ no7es.
CKD T!e$e a$e t2o g(o6a( mo7es o* (ea$ning in ECOS<
CaD Acti'e (ea$ning - (ea$ning is 1e$*o$me7 2!en a stim)()s Cin1)t
1atte$nD is 1$esente7 an7 3e1t acti'e.
C6D Passi'e Cinne$= ECOD (ea$ning mo7e - (ea$ning is 1e$*o$me7 2!en
t!e$e is no in1)t 1atte$n 1$esente7 to t!e ECOS. In t!is case t!e
1$ocess o* *)$t!e$ e(a6o$ation o* t!e connections in ECOS is 7one in a
1assi'e (ea$ning 1!ase= 2!en e8isting connections= t!at sto$e
1$e'io)s(y *e7 in1)t 1atte$ns= a$e )se7 as Hec!oI C!e$e 7enote7 as
ECOD to $eite$ate t!e (ea$ning 1$ocess.
T!e$e a$e t2o ty1es o* ECO t$aining<
casca7e eco-t$aining< a ne2 connectionist st$)ct)$e Ca NND is c$eate7 in
an on-(ine mo7e 2!en conce1t)a((y ne2 7ata Ce.g.= a ne2 c(ass 7ataD is
1$esente7. T!e NN is t$aine7 on t!e 1ositi'e e8am1(es o* t!is c(ass= on
t!e negati'e e8am1(es *$om t!e *o((o2ing incoming 7ata= an7 on t!e
negati'e e8am1(es *$om 1$e'io)s(y sto$e7 1atte$ns in 1$e'io)s(y
c$eate7 mo7)(es.
Js(ee1J eco-t$aining< NNs a$e c$eate7 2it! t!e )se o* on(y 1a$tia(
in*o$mation *$om t!e in1)t st$eam Ce.g.= 1ositi'e c(ass e8am1(es on(yD.
T!en t!e NNs a$e t$aine7 an7 $e*ine7 on t!e sto$e7 1atte$ns
Ce8em1(a$sD in ot!e$ NNs an7 NN%s Ce.g.= as negati'e c(ass e8am1(esD.
C0D ECOS 1$o'i7e e81(anation in*o$mation e8t$acte7 *$om t!e NN%s
t!$o)g! t!e se(*-ana(ysisL $)(e e8t$action mo7)(e. ,ene$a((y s1ea3ing=
ECOS (ea$n an7 sto$e 3no2(e7ge= $)(es= $at!e$ t!an in7i'i7)a(
e8am1(es o$ meaning(ess n)m6e$s.
C7D T!e ECOS 1$inci1(es a6o'e a$e 6ase7 on some 6io(ogica( *acts an7
6io(ogica( 1$inci1(es. Im1(ementing t!e ECOS *$ame2o$3 an7 t!e
NN% *$om it $e>)i$es connectionist mo7e(s t!at com1(y 2it! t!e
ECOS 1$inci1(es. One o* t!em= ca((e7 e'o('ing *)FFy ne)$a( net2o$3
CEF)NND is 1$esente7 in t!e ne8t section.
C9APTER 4
APPLICATIONS OF CONNECTIONIST S.STE%S
4.1 Connectionist-Base7 Decision S)11o$t Systems an7 E81e$t
Systems <
A 7ecision s)11o$t system is an in*o$mation system t!at e'a()ates
7i**e$ent o1tions in o$7e$ to !e(1 !)mans ma3e a 7ecision on a gi'en
1$o6(em= )n7e$ gi'en ci$c)mstances an7 const$aints. A 7ecision ma3ing
system= in cont$ast= is a system t!at ma3es t!e *ina( 7ecision an7 ta3es
actions= e.g. a)tomate7 t$a7ing systems on t!e Inte$net= o$ systems t!at g$ant
(oans t!$o)g! e(ect$onic s)6missions. E81e$t systems a$e in*o$mation
systems t!at contain e81e$t 3no2(e7ge on a 1a$tic)(a$ 1$o6(em an7 1e$*o$m
in*e$ence 2!en ne2 7ata is ente$e7 t!at may 6e 1a$tia( o$ ine8act. T!ey
1$o'i7e a so()tion t!at is e81ecte7 to 6e simi(a$ to t!e so()tion 1$o'i7e7 6y
e81e$ts in t!e *ie(7. An e81e$t system )s)a((y consists o* se'e$a( 1a$ts< C1D a
3no2(e7ge 6ase= 2!e$e t!e e81e$t 3no2(e7ge $esi7esB C#D a 7ata 6ase= 2!e$e
!isto$ica( an7 ne2 7ata is sto$e7B C+D an in*e$ence mac!ine t!at 1$o'i7es
7i**e$ent ty1es o* in*e$encesB C4D an e**icient inte$*ace to )se$sB CKD an
e81(anation mo7)(e t!at 2o)(7 1$o'i7e an e81(anation o* !o2 an7 2!y a
ce$tain 7ecision 2as ecommen7e7 6y t!e systemB C0D a mo7)(e t!at (ea$ns
an7 acc)m)(ates ne2 3no2(e7ge= 6ase7 on t!e systemJs o1e$ation an7 on
ne2 incoming 7ata .
T!e com1(e8ity an7 t!e 7ynamics o* many $ea(-2o$(7 1$o6(ems= s)c!
as 7ecision ma3ing an7 1$e7iction o* economic an7 *inancia( in7e8es=
7ecision ma3ing in me7icine= 3no2(e7ge 7isco'e$y in 6ioin*o$matics=
a7a1ti'e inte((igent cont$o( o* in7)st$ia( 1$ocesses= on-(ine 7ecision ma3ing
6ase7 on (a$ge amo)nt o* contin)o)s an7 7ynamica((y c!anging in*o$mation
on t!e = an7 many mo$e= set ce$tain $e>)i$ements to t!e met!o7s an7
t!e too(s *o$ 6)i(7ing DS. S)c! $e>)i$ements a$e<
Dea( 2it! 7i**e$ent ty1es o* 7ata an7 3no2(e7ge.
A7A)st inc$ementa((y to 7ynamic c!anges in t!e o1e$ating en'i$onment=
accommo7ating ne2 7ata= int$o7)cing ne2 'a$ia6(es an7 *eat)$es i* nee7e7
2it!o)t t!e nee7 to $e-7esign o$ $e-t$ain t!e 2!o(e systemB s)c! a7a1tation
may 6e $e>)i$e7 to 6e ac!ie'e7 in an on-(ine= $ea(-time mo7e.
U17ate t!e systemJs 3no2(e7ge in a 7ynamic 2ay.
E81(ain in an a11$o1$iate 2ay 2!at 3no2(e7ge is containe7 in t!e system=
o$ 2!at 3no2(e7ge !as 6een (ea$ne7 7)$ing t!e systemJs o1e$ation.
4.1.1 A gene$a( *$ame2o$3 o* a connectionist-6ase7 7ecision system
A *$ame2o$3 o* a CBDS is s!o2n in *ig.1. It consists o* t!e *o((o2ing
1a$ts C*ig.1D<
- P$e-1$ocessing 1a$t Ce.g. *i(te$ing o* 7ata= *eat)$e e8t$actionD.
- Ne)$a( net2o$3 1a$t - NN mo7)(es t!at a$e t$aine7 an7 inco$1o$ate
3no2(e7ge.
- 9ig!e$-(e'e( 3no2(e7ge-6ase7 1a$t Ce.g. $)(es *o$ 1$o7)cing *ina(
7ecisionsD.
- A7a1tation 1a$t ; it e'a()ates t!e system 7ecisions an7 ma3es c!anges to
t!e system st$)ct)$e an7 *)nctiona(ity. O)t1)t e$$o$ can 6e )se7 to a7A)st
$e(e'ant NN mo7)(es. PBNN a$e 1$e-st$)ct)$e7 ne)$a( net2o$3s to a((o2 *o$
7ata an7 3no2(e7ge mani1)(ation= inc()7ing (ea$ning *$om 7ata= $)(e
inse$tion= $)(e e8t$action= a7a1tation an7 $easoning CTo2e( an7 S!a'(i3=
1//+B %it$a an7 9ays!i=#555D. PBNN !a'e 6een 7e'e(o1e7 eit!e$ as a
com6ination o* sym6o(ic AI systems an7 NN= o$ as a com6ination o* *)FFy
(ogic systems an7 NN= o$ as ot!e$ !y6$i7 systems.
4.# E'o('ing Connectionist Systems *o$ On-(ine=
Pno2(e7ge-Base7 Lea$ning <
T!e !)man 6$ain 1$o'e7 to 6e t!e 6est com1)tationa( mec!anism *o$
many tas3s= s)c! as s1eec! an7 (ang)age 1$ocessing= image 1$ocessing=
na'igation= cont$o(. One o* t!e most im1o$tant c!a$acte$istics o* t!e 6$ain is
its a6i(ity to (ea$n in an on-(ine mo7e= in a (i*e(ong mo7e= to a7a1t >)ic3(y= to
ma3e a6st$actions an7$e1$esent t!em as 3no2(e7ge= to e'o('e its st$)ct)$e
an7 *)nctions 7)$ing its (i*etime in an inte$acti'e 2ay an7 its innate 2ay.
T!e *o((o2ing a$e some 1$inci1(es o* t!e e'o('ing 6$ains<
1. E'o('ing is ac!ie'e7 t!$o)g! 6ot! genetica((y 7e*ine7 in*o$mation an7
(ea$ning.
#. T!e e'o('e7 ne)$ons !a'e a s1atia(-tem1o$a( $e1$esentation 2!e$e
simi(a$ stim)(i acti'ate c(ose ne)$ons.
+. Re7)n7ancy= i.e. t!e$e a$e many $e7)n7ant ne)$ons a((ocate7 to a
sing(e stim)()s o$ a tas3B e.g.= 2!en a 2o$7 is !ea$7= t!e$e a$e
!)n7$e7s o* t!o)san7s o* ne)$ons t!at get imme7iate(y acti'ate7.
4. %emo$y-6ase7 (ea$ning= i.e. t!e 6$ain sto$es e8em1(a$s o* *acts t!at
can 6e $eca((e7 at a (ate$ stage.
K. E'o('ing t!$o)g! inte$action 2it! t!e en'i$onment an7 2it! ot!e$
6$ains.
0. Inne$ 1$ocesses= 6ase7 on in*o$mation t!eo$y= ta3e 1(aceB t!ese
1$ocesses can 6e 7esc$i6e7 as an Hinstinct *o$ in*o$mationIB t!ey a$e
6ase7 on in*o$mation ent$o1y an7 ca)se t!e 6$ain to ac>)i$e
in*o$mation.
7. T!e e'o('ing 1$ocess is contin)o)s= (i*e(ong.
&. E'o('ing !ig!e$ (e'e( *)nctions= cognition an7 inte((igence= i.e. !ig!e$-
(e'e( conce1ts eme$ge t!at a$e em6o7ie7 in t!e st$)ct)$e an7 can 6e
$e1$esente7 as a (e'e( o* a6st$action at any time o* t!e e'o('ing
1$ocess= e.g. ac>)isition an7 t!e 7e'e(o1ment o* s1eec! an7 (ang)age=
es1ecia((y in m)(ti(ing)a( s)6Aects.
/. E'o('ing Qg(o6a( 6$ainsJ t!$o)g! inte$action o* in7i'i7)a(s= i.e. an
in7i'i7)a( 6$ain acts as an QagentJ in a co((a6o$ati'e en'i$onment o*
ot!e$ agents. A co((ection o* agents can 6e 'ie2e7 as a Qg(o6a( 6$ainJ=
t!at im1$o'es in a contin)o)s= en7(ess 2ay= 2it! t!e eme$gence o* ne2
in7i'i7)a(s.
CONCLUSION
T!ese t!$ee a11$oac!es a$e $e1$esent attem1ts to 7ea( 2it! t!e s!o$t-
comings o* t!e cogniti'e mo7e(s. T!e connectionist *$ame2o$3 is 1$ima$i(y
conce$ne7 2it! 6io(ogica( im1(ementation= an7 2it! 1$o6(ems in (ea$ning
an7 $e1$esentation. Can an in7)cti'e (ea$ning 1$oce7)$e 7isco'e$ a6st$act
gene$a(iFations= )sing on(y e8am1(es $at!e$ t!an e81(icit(y *o$m)(ate7
inst$)ction" An7 !o2 7o t!e $es)(ting 3no2(e7ge st$)ct)$es ca1t)$e t!e
gene$a(iFations" A$e t!e$e im1o$tant 7i**e$ences 6et2een t!e 2ays in 2!ic!
net2o$3s $e1$esent gene$a(iFations an7 t$a7itiona( $)(e systems"
T!e *oc)s o* A(i*e is 7i**e$ent. A(i*e $eAects t!e 'ie2 t!at cognition
consists on(y o* !ig!(y 7e'e(o1e7 menta( acti'ities s)c! as c!ess= an7
em1!asiFes t!e inte((igence 2!ic! is $e>)i$e7 sim1(y to s)$'i'e. T!e $o(e o*
a7a1tation an7 t!e $o(e o* e'o()tion as ac!ie'ing a7a1tation a$e 'a()e7B an7
a t!eme common to m)c! o* t!e 2o$3 in A(i*e is t!e eme$gence o* st$)ct)$e
an7 6e!a'io$s 2!ic! a$e not 7esigne7= 6)t $at!e$ t!e o)tg$o2t! o* com1(e8
inte$actions.
T!e 7ynamica( systems a11$oac! a(so is conce$ne7 2it! inte$action
an7 eme$gentismB mo$e gene$a((y= it can 6e 'ie2e7 as a mat!ematica(
*$ame2o$3 *o$ )n7e$stan7ing t!e so$t o* eme$gentism an7 t!e !ig!-o$7e$
inte$actions 2!ic! a$e *o)n7 in 6ot! connectionist an7 a$ti*icia( (i*e mo7e(s.
Dynamica( systems a(so $e*(ects a 7ee1e$ commitment to t!e im1o$tance o*
inco$1o$ating time into o)$ mo7e(s. T!e t!$ee a11$oac!es s!a$e m)c! in
common. T!ey a(( $e*(ect an inc$ease7 inte$est in t!e 2ays in 2!ic! 1aying
c(ose$ attention to nat)$a( systems Cne$'o)s systemsB e'o()tionB 1!ysicsD
mig!t e()ci7ate cognition.