12 views

Uploaded by Carlos Cruz Treviño

save

You are on page 1of 8

Credit work to subject “Information, uncertainty and their processing” Department of Computer Science at Palacký University Olomouc Roman Nepšinský (INF98), nepsinsr@inf.upol.cz 04.05.2003 (2nd revision)

Abstract: This article focuses on bi-polar Hopfield networks and how to generalize known adaptation and activation rules for bi-polar networks to achieve similar results for experimental tri-polar networks. The cardinal question to discuss is, if the tri-polar network stops in finite time for every given input as the bi-polar network does (when using asynchronous network activation).

Contents:

Bi-polar Hopfield network example 2 Bi-polar network adaptation and activation rules generalization 4 Tri-polar network based on generalized rules 6 Tri-polar network finality testing 8

1} arranged in grid 3× 3 connected each other..+ -.1 = 0 w08 = w80 = 1.1 + -..= -2 w46 = w64 = 1.1 = 2 w38 = w83 = -..1 = 2 w18 = w81 = -..+ 1.1 = -2 w05 = w50 = 1. Network: x0 x1 x2 x3 x4 x5 x6 x7 x8 Patterns: 1 ..1 + -.1 = 0 w48 = w84 = 1..1 = 0 w47 = w74 = 1.+ -...1 = -2 w26 = w62 = 1..+ -.Bi-polar and (experimental) tri-polar Hopfield networks Bi-polar Hopfield network example This example shows bi-polar associative network consisting of 9 bi-polar neurons xi ∈{-1.1 + 1..= 2 w34 = w43 = -.1 = 2 w23 = w32 = 1.1 + 1.= 2 w03 = w30 = 1...1 = 0 w35 = w53 = -.1 + -.1 + 1.1 + -.1 + 1.1 = -2 w36 = w63 = -.1 + 1.= -2 w58 = w85 = -..+ 1..+ 1.= -2 w13 = w31 = -.1 + -...1 + -..+ -.1 + 1.= 2 w28 = w82 = 1...1 = -2 w27 = w72 = 1.= 2 w w04 = w40 = 1.+ 1.1 + 1.1 + 1.= -2 w17 = w71 = -..1 = 0 w16 = w61 = -.+ 1.= -2 w67 = w76 = 1.1 + 1.1 + 1.1 + 1.1 + -.1 = 0 w06 = w60 = 1.= 0 w57 = w75 = -.1 1 1 1 1 1 1 1 1 - (symbol “–“ representing value of -1 used for simplicity of notation) Adaptation (weights): Based on: wij = ∑xi(p)xj(p) w00 = w11 = w22 = w33 = w44 = w55 = w66 = w77 = w88 = 0 w01 = w10 = 1.= -2 0 1 2 3 4 5 6 7 8 0 -2 2 -2 0 0 2 -2 2 -2 0 -2 2 0 2 -2 2 -2 2 -2 0 -2 0 -2 2 -2 2 -2 2 -2 0 0 2 -2 2 -2 0 0 0 0 0 0 0 0 0 0 2 -2 2 0 0 -2 2 -2 2 -2 2 -2 0 -2 0 -2 2 -2 2 -2 2 0 2 -2 0 -2 2 -2 2 -2 0 -2 2 -2 0 0 1 2 3 4 5 6 7 8 2/8 ..= 2 w07 = w70 = 1.+ 1...+ -..1 = -2 w12 = w21 = -.= 2 w15 = w51 = -...+ 1.1 = 2 w78 = w87 = -...+ 1.1 = 2 w14 = w41 = -.+ -.1 = -2 w68 = w86 = 1...= 0 w56 = w65 = -.= -2 w24 = w42 = 1.1 = 0 w25 = w52 = 1..+ 1..1 = 2 w37 = w73 = -.1 = -2 w02 = w20 = 1.= -2 w45 = w54 = 1. Every connection between two neurons i and j is weighed using wij ∈ Z.1 + -.1 + 1.+ -.

pattern 1 was recognized.Bi-polar and (experimental) tri-polar Hopfield networks Activation (asynchronous): Based on: xj(t+1) = -1 / xj(t) / 1 iff ∑wijxi(t) < / = / > θ (=0) Example 1: 1 1 1 1 1 1 8 1 1 1 1 1 1 1 -6 1 1 1 1 1 1 1 6 1 1 1 1 1 1 1 -6 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 -8 1 1 1 1 1 1 1 0 1 1 1 1 1 1 -14 Round #1 1 1 1 1 1 14 1 1 1 1 1 12 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 -14 1 1 1 1 1 0 1 1 1 1 1 -12 1 1 1 1 1 14 Round #2 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 12 Round #3 Network is now stable. Example 2: 1 1 1 1 -8 1 1 1 6 1 1 1 -6 1 1 1 6 1 1 1 0 1 1 1 8 1 1 1 1 1 0 1 1 1 1 14 Round #1 1 1 1 1 1 -14 1 1 1 1 1 -12 1 1 1 1 1 14 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 0 1 1 1 1 1 12 1 1 1 1 1 -14 Round #2 1 1 1 1 1 14 1 1 1 1 1 14 1 1 1 1 1 -12 Round #3 Network is now stable. 3/8 . pattern 2 was recognized.

w04 = w40 = 0 etc. -1&1. Thus all 4 possible situations (1&1. Instead of computing ∑wijxi(t) we will make sum of elements in row vectors of Wij (appropriate row will be chosen according to neuron i value). 0 -1 1 and so on. B&C.e. C&C). w02 = w20 = 2. respectively 3× 3 (for tri-polar networks). It is however obvious that one whole number can not be enough for coding information in tri-polar networks (with total 9 possible situations A&A A&B. -1&-1) between two neurons in each pattern can be described using one whole number. After adding all elements for every j we get two-element vector in which we locate the maximum and set neuron i value to 1 if the first element was maximal. xi(p) = xj(p)). otherwise is decreased by 1 (this is consequence of using bi-polar values 1 and -1 for neuron states). Now we have to simulate adaptation rule (xj(t+1) = -1 / xj(t) / 1 iff ∑wijxi(t) < / = / > θ) to work with matrix Wij instead of weight wij but with the same behavior. C&B. Wij 1 1 -1 Wij A A B C -1 B C For the example bi-polar network from previous chapter (see page 2) we have: W01 1 1 0 -1 1 W10 1 1 0 -1 1 -1 1 0 -1 1 0 W02 1 1 1 -1 0 W20 1 1 1 -1 0 -1 0 1 -1 0 1 W03 1 1 0 -1 1 W30 1 1 0 -1 1 -1 1 0 -1 1 0 W04 1 1 1 -1 1 W40 1 1 1 -1 0 -1 0 and so on. -1 otherwise (when both of the elements have the same value. neuron’s value remains unchanged). C&A.. 1&-1.Bi-polar and (experimental) tri-polar Hopfield networks Bi-polar network adaptation and activation rules generalization Basic adaptation rule wij = ∑xi(p)xj(p) does nothing but coding of information about every pair of neuron values in each pattern. 4/8 . weight wij is increased by 1. 0 This corresponds to weights w01 = w10 = -2. If two neurons i and j have the same value in pattern p (i. B&B.. B&A. But we can safely store the same information about values of every pair of neurons in a matrix Wij of size 2× 2 (for bipolar networks)... w03 = w30 = -2. A&C.

Bi-polar and (experimental) tri-polar Hopfield networks Activation: Example 1: 1 1 1 1 1 1 3 6 1 1 1 1 1 1 1 6 3 1 1 1 1 1 1 1 3 6 1 1 1 1 1 1 1 6 3 1 1 1 1 1 1 1 0 8 1 1 1 1 1 1 1 7 2 1 1 1 1 1 1 2 7 1 1 1 1 1 1 8 1 Round #1 1 1 1 1 1 1 8 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 0 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 Round #2 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 1 8 Round #3 Network is now stable. pattern 1 was recognized. Example 2: 1 1 1 1 6 3 1 1 1 3 6 1 1 1 6 3 1 1 1 3 6 1 1 1 0 8 1 1 1 2 7 1 1 1 1 7 2 1 1 1 1 1 8 Round #1 1 1 1 1 1 8 1 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 0 8 1 1 1 1 1 1 8 1 1 1 1 1 8 1 Round #2 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 8 1 Round #3 Network is now stable. 5/8 . pattern 2 was recognized.

C}. activation rule chooses maximum in 3element vector (when there is more than one maximal element (e. two elements having the same maximal value). In this network every neuron xi is allowed to have one of three values {A. neuron’s value remains unchanged as long as it belongs to one of those elements). B. Weight matrix has size of 3× 3. Patterns: B B B B C B B B B B A B A B A B A B C C C A C A A C A A C C A A A C C A Weights: 01 A B C 12 A B C 23 A B C 34 A B C 45 A B C 56 A B C 67 A B C 78 A B C A 0 1 0 A 0 0 0 A 0 1 2 A 1 0 0 A 1 1 1 A 1 0 0 A 0 1 0 A 0 0 2 B 0 1 0 B 1 1 0 B 0 1 0 B 1 0 0 B 0 0 1 B 1 1 0 B 0 1 0 B 1 1 0 C 1 0 1 C 0 0 2 C 0 0 0 C 1 1 0 C 0 0 0 C 1 0 0 C 1 0 1 C 0 0 0 02 A B C 13 A B C 24 A B C 35 A B C 46 A B C 57 A B C 68 A B C A 0 0 0 A 1 0 2 A 0 0 1 A 3 0 0 A 0 0 1 A 1 0 0 A 1 0 1 B 0 2 0 B 0 1 0 B 0 1 0 B 0 1 0 B 0 1 1 B 0 1 0 B 0 2 0 C 1 0 1 C 0 0 0 C 0 1 1 C 0 0 0 C 1 0 0 C 2 0 0 C 0 0 0 03 A B C 14 A B C 25 A B C 36 A B C 47 A B C 58 A B C A 1 1 1 A 0 0 1 A 0 1 2 A 1 0 0 A 0 1 0 A 2 0 0 B 0 1 0 B 1 0 0 B 0 1 0 B 1 1 0 B 0 0 1 B 1 1 0 C 0 0 0 C 0 1 1 C 0 0 0 C 1 0 0 C 1 0 1 C 0 0 0 04 A B C 15 A B C 26 A B C 37 A B C 48 A B C A 1 0 0 A 1 0 2 A 0 0 1 A 1 0 0 A 1 0 1 B 0 1 0 B 0 1 0 B 0 2 0 B 0 1 0 B 0 1 1 C 0 1 1 C 0 0 0 C 0 0 1 C 2 0 0 C 0 0 0 05 A B C 16 A B C 27 A B C 38 A B C A 1 1 1 A 0 0 1 A 0 1 0 A 2 0 0 B 0 1 0 B 1 1 0 B 0 1 0 B 1 1 0 C 06 A 0 A 0 0 B 0 0 C 1 C 17 A 0 A 1 0 B 0 1 C 0 C 28 A 0 A 0 0 B 0 2 C 2 C 0 0 0 B 0 2 0 B 0 1 0 B 0 2 0 C 07 A 1 A 0 0 B 1 0 C 0 C 18 A 0 A 0 0 B 0 2 C 2 C 0 0 0 B 0 1 0 B 1 1 0 C 08 A 1 A 1 0 B 0 1 C 1 C 0 0 0 B 0 2 0 C 0 0 0 6/8 . In our example we use associative network with 9 tri-polar neurons and 4 patterns.g.Bi-polar and (experimental) tri-polar Hopfield networks Tri-polar network based on generalized rules Now we can use generalized weight adaptation (matrix Wij) and activation rule (locating maximum in 2-element vector) for tri-polar network.

pattern 3 was recognized.Bi-polar and (experimental) tri-polar Hopfield networks Activation: Example: A A A 3 B A A 5 B C C 0 B C C 13 B C C 5 B C C 13 B C C 6 B C C 5 A C A 8 A C A 4 A C A 9 A C A 4 A C A 5 A C A 4 A C A 7 A C A 4 B C B 4 B C B 7 B C B 9 B C B 0 B C B 8 B C B 0 B C B 5 B C B 9 Round #1 B C C 11 B C C 6 B C C 3 B C C 0 B C C 14 B C C 6 B C C 14 B C C 8 A C A 7 A C A 5 A C A 2 A C A 5 A C A 2 A C A 3 A C A 2 A C A 3 B C B 0 B C A 7 B C A 12 B C A 12 B C A 0 B C A 8 B C A 0 B C A 6 Round #2 B C C 2 B C C 13 B C C 6 C C C 2 C C C 0 C C C 14 C C C 6 C C C 14 A C A 1 A C A 3 A C A 3 A C A 1 A C A 3 A C A 1 A C A 2 A C A 1 A C A 13 A C A 0 A C A 8 A C A 13 A C A 13 A C A 0 A C A 8 A C A 0 Round #3 C C C 8 C C C 2 C C C 13 C C C 6 A C A 3 A C A 1 A C A 3 A C A 3 A C A 6 A C A 13 A C A 0 A C A 8 Round #4 Network is now stable. 7/8 .

The question is.36× 10 (* Note: net1 is example bi-polar network from page 2. net2 is example tri-polar network from page 6) (** Note: pattern is recognized only for approximately every 0. It uses generalized adaptation and activation rules that were introduced in previous chapters and offers testing capability. 8/8 . for bi-polar networks it is 2n.Bi-polar and (experimental) tri-polar Hopfield networks Tri-polar network finality testing We have shown that our generalized adaptation and activation rules produce behavior similar to basic rule for bi-polar associative networks. for tri-polar networks 3n.00162 input thus the results were rounded to 0%) Conclusion: I was not successful finding such a network that will end up in a cycling state. Every tested input is being simulated in network until stable state is achieved or maximal number of iterations has exceeded. Here is a summary of test results over several bi-polar and tri-polar networks: Network (2/3) Search space Total tests Average iterations Recognized Patterns Cycling net1 (2)* 29 = 512 512 16 100% 2 (50:50%) 0 * net2 (3) 39 = 19683 19683 21 95% 4 (14:10:65:6%) 0 3colors (3) 39 = 19683 19683 17 100% 3 (24:33:43%) 0 3states (2) 216 = 65536 65536 30 100% 3 (3:6:91%) 0 46 (2) 215 = 32768 32768 29 100% 2 (38:62%) 0 cd (2) 260 = 1152921504606846976 420000 118 100% 2 (50:50%) 0 geom (3) 358000 191 100% 2 (53:47%) 0 396 > 6. it seems it might be true. Although this is not a proof of the hypothesis about tri-polar networks.36× 1045 96 45 flags (3) 640000 205 0% 3 (0:0:0%)** 0 3 > 6. If the last mentioned situation happens it indicates that there is some cyclicity in the network and thus our hypothesis (that tri-polar network will stop in finite time for any given input) is wrong. i. Application Hopfield3 (Hopfield network simulator) serves as a simple modeling and simulating tool for both bi-polar and tri-polar networks. if our tri-polar network stops in finite time for any given input as the bi-polar network does (when using asynchronous network activation and basic activation rule). This feature allows testing within whole search space of the network (search space covers any possible input. where n is the number of neurons in network).e.

- Poly CompUploaded byJaka Haris Mustafa
- b2gcourse4bUploaded byNicholas Berryman
- Graphs of Straight LinesUploaded byoskidol
- 1341Uploaded byhunghtd_snu
- docxUploaded byhanytheking
- Tech Report Guanfeng NLNC1Uploaded bycopsumit
- DOE 2015Uploaded byRigel
- Neuro fuzzyUploaded byAdam Anugrah
- Calc Exam 3 Cheat Sheet Part 2Uploaded byKrypto_Bipto
- Prolog Home AssignmentsUploaded bymikebites
- M-9 C-5,6,7,8Uploaded byshaniarain
- Sample_Test_I_4Uploaded bysiva kumar
- 1.3(6,10,16)Uploaded by1000ankit
- mit6 003f11 lec05Uploaded byapi-246008426
- reviewUploaded byapi-235922849
- Introduction.docxUploaded byZalila Nadirah Zahari
- Basic DynamicsUploaded byRogerio
- MATLABUploaded byThiyagarajan Munivel
- Social Recommendation System for Real World Online ApplicationUploaded byGRD Journals
- [IJCST-V6I5P20]:Kadam Pallavi R, Dr. Satish Kumar VarmaUploaded byEighthSenseGroup
- isight_intro-summaryUploaded bygaryofcourse
- Book WipUploaded bykailashrawat
- Rr410210 Optimization TechniquesUploaded bySrinivasa Rao G
- Adaptive ControlUploaded byHari Thota
- Linked List pptUploaded byKhali Pankaj
- Decision Tree.10.11Uploaded byamit_iitd
- 1. VLookupUploaded byHasham Zahid
- lec-17,18Uploaded byEngr Tariq
- ITS Undergraduate 19595 2507100138 BibliographyUploaded byLucky Roza Nugroho
- elctronica de potencia circuito dcUploaded byjaronjer

- Poly CompUploaded byJaka Haris Mustafa
- b2gcourse4bUploaded byNicholas Berryman
- Graphs of Straight LinesUploaded byoskidol
- 1341Uploaded byhunghtd_snu
- docxUploaded byhanytheking
- Tech Report Guanfeng NLNC1Uploaded bycopsumit
- DOE 2015Uploaded byRigel
- Neuro fuzzyUploaded byAdam Anugrah
- Calc Exam 3 Cheat Sheet Part 2Uploaded byKrypto_Bipto
- Prolog Home AssignmentsUploaded bymikebites
- M-9 C-5,6,7,8Uploaded byshaniarain
- Sample_Test_I_4Uploaded bysiva kumar
- 1.3(6,10,16)Uploaded by1000ankit
- mit6 003f11 lec05Uploaded byapi-246008426
- reviewUploaded byapi-235922849
- Introduction.docxUploaded byZalila Nadirah Zahari
- Basic DynamicsUploaded byRogerio
- MATLABUploaded byThiyagarajan Munivel
- Social Recommendation System for Real World Online ApplicationUploaded byGRD Journals
- [IJCST-V6I5P20]:Kadam Pallavi R, Dr. Satish Kumar VarmaUploaded byEighthSenseGroup
- isight_intro-summaryUploaded bygaryofcourse
- Book WipUploaded bykailashrawat
- Rr410210 Optimization TechniquesUploaded bySrinivasa Rao G
- Adaptive ControlUploaded byHari Thota
- Linked List pptUploaded byKhali Pankaj
- Decision Tree.10.11Uploaded byamit_iitd
- 1. VLookupUploaded byHasham Zahid
- lec-17,18Uploaded byEngr Tariq
- ITS Undergraduate 19595 2507100138 BibliographyUploaded byLucky Roza Nugroho
- elctronica de potencia circuito dcUploaded byjaronjer
- The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital RevolutionWalter Isaacson
- Elon Musk: Tesla, SpaceX, and the Quest for a Fantastic FutureAshlee Vance
- Dispatches from Pluto: Lost and Found in the Mississippi DeltaRichard Grant
- Yes PleaseAmy Poehler
- The Unwinding: An Inner History of the New AmericaGeorge Packer
- Sapiens: A Brief History of HumankindYuval Noah Harari
- The Prize: The Epic Quest for Oil, Money & PowerDaniel Yergin
- The Emperor of All Maladies: A Biography of CancerSiddhartha Mukherjee
- This Changes Everything: Capitalism vs. The ClimateNaomi Klein
- Grand Pursuit: The Story of Economic GeniusSylvia Nasar
- A Heartbreaking Work Of Staggering Genius: A Memoir Based on a True StoryDave Eggers
- Devil in the Grove: Thurgood Marshall, the Groveland Boys, and the Dawn of a New AmericaGilbert King
- John AdamsDavid McCullough
- The Hard Thing About Hard Things: Building a Business When There Are No Easy AnswersBen Horowitz
- Smart People Should Build Things: How to Restore Our Culture of Achievement, Build a Path for Entrepreneurs, and Create New Jobs in AmericaAndrew Yang
- Rise of ISIS: A Threat We Can't IgnoreJay Sekulow
- The World Is Flat 3.0: A Brief History of the Twenty-first CenturyThomas L. Friedman
- The New Confessions of an Economic Hit ManJohn Perkins
- Team of Rivals: The Political Genius of Abraham LincolnDoris Kearns Goodwin
- Bad Feminist: EssaysRoxane Gay
- Steve JobsWalter Isaacson
- Angela's Ashes: A MemoirFrank McCourt
- How To Win Friends and Influence PeopleDale Carnegie
- The Incarnations: A NovelSusan Barker
- You Too Can Have a Body Like Mine: A NovelAlexandra Kleeman
- The Sympathizer: A Novel (Pulitzer Prize for Fiction)Viet Thanh Nguyen
- Extremely Loud and Incredibly Close: A NovelJonathan Safran Foer
- Leaving Berlin: A NovelJoseph Kanon
- The Silver Linings Playbook: A NovelMatthew Quick
- The Light Between Oceans: A NovelM.L. Stedman
- The Flamethrowers: A NovelRachel Kushner
- Brooklyn: A NovelColm Toibin
- We Are Not Ourselves: A NovelMatthew Thomas
- The First Bad Man: A NovelMiranda July
- The Rosie Project: A NovelGraeme Simsion
- The Blazing World: A NovelSiri Hustvedt
- The MasterColm Toibin
- Bel CantoAnn Patchett
- A Man Called Ove: A NovelFredrik Backman
- The Love Affairs of Nathaniel P.: A NovelAdelle Waldman
- Life of PiYann Martel
- The Cider House RulesJohn Irving
- A Prayer for Owen Meany: A NovelJohn Irving
- The Bonfire of the Vanities: A NovelTom Wolfe
- Lovers at the Chameleon Club, Paris 1932: A NovelFrancine Prose
- The Perks of Being a WallflowerStephen Chbosky
- The Kitchen House: A NovelKathleen Grissom
- Beautiful Ruins: A NovelJess Walter
- Interpreter of MaladiesJhumpa Lahiri
- The WallcreeperNell Zink
- Wolf Hall: A NovelHilary Mantel
- The Art of Racing in the Rain: A NovelGarth Stein
- A Tree Grows in BrooklynBetty Smith