You are on page 1of 8

Bi-polar and (experimental) tri-polar Hopfield networks

Credit work to subject “Information, uncertainty and their processing” Department of Computer Science at Palacký University Olomouc Roman Nepšinský (INF98), nepsinsr@inf.upol.cz 04.05.2003 (2nd revision)

Abstract: This article focuses on bi-polar Hopfield networks and how to generalize known adaptation and activation rules for bi-polar networks to achieve similar results for experimental tri-polar networks. The cardinal question to discuss is, if the tri-polar network stops in finite time for every given input as the bi-polar network does (when using asynchronous network activation).

Contents:
Bi-polar Hopfield network example 2 Bi-polar network adaptation and activation rules generalization 4 Tri-polar network based on generalized rules 6 Tri-polar network finality testing 8

1} arranged in grid 3× 3 connected each other..+ -.1 = 0 w08 = w80 = 1.1 + -..= -2 w46 = w64 = 1.1 = 2 w38 = w83 = -..1 = 2 w18 = w81 = -..+ 1.1 = -2 w05 = w50 = 1. Network: x0 x1 x2 x3 x4 x5 x6 x7 x8 Patterns: 1 ..1 + -.1 = 0 w48 = w84 = 1..1 = 0 w47 = w74 = 1.+ -...1 = -2 w26 = w62 = 1..+ -.Bi-polar and (experimental) tri-polar Hopfield networks Bi-polar Hopfield network example This example shows bi-polar associative network consisting of 9 bi-polar neurons xi ∈{-1.1 + 1..= 2 w34 = w43 = -.1 = 2 w23 = w32 = 1.1 + 1.= 2 w03 = w30 = 1...1 = 0 w35 = w53 = -.1 + -.1 + 1.1 + -.1 + 1.1 = -2 w36 = w63 = -.1 + 1.= -2 w58 = w85 = -..+ 1..+ 1.= -2 w13 = w31 = -.1 + -...1 + -..+ -.1 + 1.= 2 w28 = w82 = 1...1 = -2 w27 = w72 = 1.= 2 w w04 = w40 = 1.+ 1.1 + 1.1 + 1.= -2 w17 = w71 = -..1 = 0 w16 = w61 = -.+ 1.= -2 w67 = w76 = 1.1 + 1.1 + 1.1 + 1.1 + -.1 = 0 w06 = w60 = 1.= 0 w57 = w75 = -.1 1 1 1 1 1 1 1 1 - (symbol “–“ representing value of -1 used for simplicity of notation) Adaptation (weights): Based on: wij = ∑xi(p)xj(p) w00 = w11 = w22 = w33 = w44 = w55 = w66 = w77 = w88 = 0 w01 = w10 = 1.= -2 0 1 2 3 4 5 6 7 8 0 -2 2 -2 0 0 2 -2 2 -2 0 -2 2 0 2 -2 2 -2 2 -2 0 -2 0 -2 2 -2 2 -2 2 -2 0 0 2 -2 2 -2 0 0 0 0 0 0 0 0 0 0 2 -2 2 0 0 -2 2 -2 2 -2 2 -2 0 -2 0 -2 2 -2 2 -2 2 0 2 -2 0 -2 2 -2 2 -2 0 -2 2 -2 0 0 1 2 3 4 5 6 7 8 2/8 ..= 2 w07 = w70 = 1.+ 1...+ -..1 = -2 w12 = w21 = -.= 2 w15 = w51 = -...+ 1.1 = 2 w78 = w87 = -...+ 1.1 = 2 w14 = w41 = -.+ -.1 = -2 w68 = w86 = 1...= 0 w56 = w65 = -.= -2 w24 = w42 = 1.1 = 0 w25 = w52 = 1..+ 1..1 = 2 w37 = w73 = -.1 = -2 w02 = w20 = 1.= -2 w45 = w54 = 1. Every connection between two neurons i and j is weighed using wij ∈ Z.1 + -.1 + 1.+ -.

pattern 1 was recognized.Bi-polar and (experimental) tri-polar Hopfield networks Activation (asynchronous): Based on: xj(t+1) = -1 / xj(t) / 1 iff ∑wijxi(t) < / = / > θ (=0) Example 1: 1 1 1 1 1 1 8 1 1 1 1 1 1 1 -6 1 1 1 1 1 1 1 6 1 1 1 1 1 1 1 -6 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 -8 1 1 1 1 1 1 1 0 1 1 1 1 1 1 -14 Round #1 1 1 1 1 1 14 1 1 1 1 1 12 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 -14 1 1 1 1 1 0 1 1 1 1 1 -12 1 1 1 1 1 14 Round #2 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 12 Round #3 Network is now stable. Example 2: 1 1 1 1 -8 1 1 1 6 1 1 1 -6 1 1 1 6 1 1 1 0 1 1 1 8 1 1 1 1 1 0 1 1 1 1 14 Round #1 1 1 1 1 1 -14 1 1 1 1 1 -12 1 1 1 1 1 14 1 1 1 1 1 -14 1 1 1 1 1 14 1 1 1 1 1 0 1 1 1 1 1 12 1 1 1 1 1 -14 Round #2 1 1 1 1 1 14 1 1 1 1 1 14 1 1 1 1 1 -12 Round #3 Network is now stable. 3/8 . pattern 2 was recognized.

w04 = w40 = 0 etc. -1&1. Thus all 4 possible situations (1&1. Instead of computing ∑wijxi(t) we will make sum of elements in row vectors of Wij (appropriate row will be chosen according to neuron i value). 0 -1 1 and so on. B&C.e. C&C). w02 = w20 = 2. respectively 3× 3 (for tri-polar networks). It is however obvious that one whole number can not be enough for coding information in tri-polar networks (with total 9 possible situations A&A A&B. -1&-1) between two neurons in each pattern can be described using one whole number. After adding all elements for every j we get two-element vector in which we locate the maximum and set neuron i value to 1 if the first element was maximal. xi(p) = xj(p)). otherwise is decreased by 1 (this is consequence of using bi-polar values 1 and -1 for neuron states). Now we have to simulate adaptation rule (xj(t+1) = -1 / xj(t) / 1 iff ∑wijxi(t) < / = / > θ) to work with matrix Wij instead of weight wij but with the same behavior. C&B. Wij 1 1 -1 Wij A A B C -1 B C For the example bi-polar network from previous chapter (see page 2) we have: W01 1 1 0 -1 1 W10 1 1 0 -1 1 -1 1 0 -1 1 0 W02 1 1 1 -1 0 W20 1 1 1 -1 0 -1 0 1 -1 0 1 W03 1 1 0 -1 1 W30 1 1 0 -1 1 -1 1 0 -1 1 0 W04 1 1 1 -1 1 W40 1 1 1 -1 0 -1 0 and so on. -1 otherwise (when both of the elements have the same value. neuron’s value remains unchanged). C&A.. 1&-1.Bi-polar and (experimental) tri-polar Hopfield networks Bi-polar network adaptation and activation rules generalization Basic adaptation rule wij = ∑xi(p)xj(p) does nothing but coding of information about every pair of neuron values in each pattern. 4/8 . weight wij is increased by 1. 0 This corresponds to weights w01 = w10 = -2. If two neurons i and j have the same value in pattern p (i. B&B.. B&A. But we can safely store the same information about values of every pair of neurons in a matrix Wij of size 2× 2 (for bipolar networks)... w03 = w30 = -2. A&C.

Bi-polar and (experimental) tri-polar Hopfield networks Activation: Example 1: 1 1 1 1 1 1 3 6 1 1 1 1 1 1 1 6 3 1 1 1 1 1 1 1 3 6 1 1 1 1 1 1 1 6 3 1 1 1 1 1 1 1 0 8 1 1 1 1 1 1 1 7 2 1 1 1 1 1 1 2 7 1 1 1 1 1 1 8 1 Round #1 1 1 1 1 1 1 8 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 0 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 Round #2 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 1 8 Round #3 Network is now stable. pattern 1 was recognized. Example 2: 1 1 1 1 6 3 1 1 1 3 6 1 1 1 6 3 1 1 1 3 6 1 1 1 0 8 1 1 1 2 7 1 1 1 1 7 2 1 1 1 1 1 8 Round #1 1 1 1 1 1 8 1 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 1 8 1 1 1 1 1 0 8 1 1 1 1 1 1 8 1 1 1 1 1 8 1 Round #2 1 1 1 1 1 1 8 1 1 1 1 1 8 1 1 1 1 1 1 8 1 Round #3 Network is now stable. 5/8 . pattern 2 was recognized.

C}. activation rule chooses maximum in 3element vector (when there is more than one maximal element (e. two elements having the same maximal value). In this network every neuron xi is allowed to have one of three values {A. neuron’s value remains unchanged as long as it belongs to one of those elements). B. Weight matrix has size of 3× 3. Patterns: B B B B C B B B B B A B A B A B A B C C C A C A A C A A C C A A A C C A Weights: 01 A B C 12 A B C 23 A B C 34 A B C 45 A B C 56 A B C 67 A B C 78 A B C A 0 1 0 A 0 0 0 A 0 1 2 A 1 0 0 A 1 1 1 A 1 0 0 A 0 1 0 A 0 0 2 B 0 1 0 B 1 1 0 B 0 1 0 B 1 0 0 B 0 0 1 B 1 1 0 B 0 1 0 B 1 1 0 C 1 0 1 C 0 0 2 C 0 0 0 C 1 1 0 C 0 0 0 C 1 0 0 C 1 0 1 C 0 0 0 02 A B C 13 A B C 24 A B C 35 A B C 46 A B C 57 A B C 68 A B C A 0 0 0 A 1 0 2 A 0 0 1 A 3 0 0 A 0 0 1 A 1 0 0 A 1 0 1 B 0 2 0 B 0 1 0 B 0 1 0 B 0 1 0 B 0 1 1 B 0 1 0 B 0 2 0 C 1 0 1 C 0 0 0 C 0 1 1 C 0 0 0 C 1 0 0 C 2 0 0 C 0 0 0 03 A B C 14 A B C 25 A B C 36 A B C 47 A B C 58 A B C A 1 1 1 A 0 0 1 A 0 1 2 A 1 0 0 A 0 1 0 A 2 0 0 B 0 1 0 B 1 0 0 B 0 1 0 B 1 1 0 B 0 0 1 B 1 1 0 C 0 0 0 C 0 1 1 C 0 0 0 C 1 0 0 C 1 0 1 C 0 0 0 04 A B C 15 A B C 26 A B C 37 A B C 48 A B C A 1 0 0 A 1 0 2 A 0 0 1 A 1 0 0 A 1 0 1 B 0 1 0 B 0 1 0 B 0 2 0 B 0 1 0 B 0 1 1 C 0 1 1 C 0 0 0 C 0 0 1 C 2 0 0 C 0 0 0 05 A B C 16 A B C 27 A B C 38 A B C A 1 1 1 A 0 0 1 A 0 1 0 A 2 0 0 B 0 1 0 B 1 1 0 B 0 1 0 B 1 1 0 C 06 A 0 A 0 0 B 0 0 C 1 C 17 A 0 A 1 0 B 0 1 C 0 C 28 A 0 A 0 0 B 0 2 C 2 C 0 0 0 B 0 2 0 B 0 1 0 B 0 2 0 C 07 A 1 A 0 0 B 1 0 C 0 C 18 A 0 A 0 0 B 0 2 C 2 C 0 0 0 B 0 1 0 B 1 1 0 C 08 A 1 A 1 0 B 0 1 C 1 C 0 0 0 B 0 2 0 C 0 0 0 6/8 . In our example we use associative network with 9 tri-polar neurons and 4 patterns.g.Bi-polar and (experimental) tri-polar Hopfield networks Tri-polar network based on generalized rules Now we can use generalized weight adaptation (matrix Wij) and activation rule (locating maximum in 2-element vector) for tri-polar network.

pattern 3 was recognized.Bi-polar and (experimental) tri-polar Hopfield networks Activation: Example: A A A 3 B A A 5 B C C 0 B C C 13 B C C 5 B C C 13 B C C 6 B C C 5 A C A 8 A C A 4 A C A 9 A C A 4 A C A 5 A C A 4 A C A 7 A C A 4 B C B 4 B C B 7 B C B 9 B C B 0 B C B 8 B C B 0 B C B 5 B C B 9 Round #1 B C C 11 B C C 6 B C C 3 B C C 0 B C C 14 B C C 6 B C C 14 B C C 8 A C A 7 A C A 5 A C A 2 A C A 5 A C A 2 A C A 3 A C A 2 A C A 3 B C B 0 B C A 7 B C A 12 B C A 12 B C A 0 B C A 8 B C A 0 B C A 6 Round #2 B C C 2 B C C 13 B C C 6 C C C 2 C C C 0 C C C 14 C C C 6 C C C 14 A C A 1 A C A 3 A C A 3 A C A 1 A C A 3 A C A 1 A C A 2 A C A 1 A C A 13 A C A 0 A C A 8 A C A 13 A C A 13 A C A 0 A C A 8 A C A 0 Round #3 C C C 8 C C C 2 C C C 13 C C C 6 A C A 3 A C A 1 A C A 3 A C A 3 A C A 6 A C A 13 A C A 0 A C A 8 Round #4 Network is now stable. 7/8 .

The question is.36× 10 (* Note: net1 is example bi-polar network from page 2. net2 is example tri-polar network from page 6) (** Note: pattern is recognized only for approximately every 0. It uses generalized adaptation and activation rules that were introduced in previous chapters and offers testing capability. 8/8 . for bi-polar networks it is 2n.Bi-polar and (experimental) tri-polar Hopfield networks Tri-polar network finality testing We have shown that our generalized adaptation and activation rules produce behavior similar to basic rule for bi-polar associative networks. for tri-polar networks 3n.00162 input thus the results were rounded to 0%) Conclusion: I was not successful finding such a network that will end up in a cycling state. Every tested input is being simulated in network until stable state is achieved or maximal number of iterations has exceeded. Here is a summary of test results over several bi-polar and tri-polar networks: Network (2/3) Search space Total tests Average iterations Recognized Patterns Cycling net1 (2)* 29 = 512 512 16 100% 2 (50:50%) 0 * net2 (3) 39 = 19683 19683 21 95% 4 (14:10:65:6%) 0 3colors (3) 39 = 19683 19683 17 100% 3 (24:33:43%) 0 3states (2) 216 = 65536 65536 30 100% 3 (3:6:91%) 0 46 (2) 215 = 32768 32768 29 100% 2 (38:62%) 0 cd (2) 260 = 1152921504606846976 420000 118 100% 2 (50:50%) 0 geom (3) 358000 191 100% 2 (53:47%) 0 396 > 6. it seems it might be true. Although this is not a proof of the hypothesis about tri-polar networks.36× 1045 96 45 flags (3) 640000 205 0% 3 (0:0:0%)** 0 3 > 6. If the last mentioned situation happens it indicates that there is some cyclicity in the network and thus our hypothesis (that tri-polar network will stop in finite time for any given input) is wrong. i. Application Hopfield3 (Hopfield network simulator) serves as a simple modeling and simulating tool for both bi-polar and tri-polar networks. if our tri-polar network stops in finite time for any given input as the bi-polar network does (when using asynchronous network activation and basic activation rule). This feature allows testing within whole search space of the network (search space covers any possible input. where n is the number of neurons in network).e.