2 Single-Input Neuron

Inputs General Neuron

p w AA
AAA
Σ A
b
n
f
a

1
a = f (wp + b)

2
2 Transfer Functions
a a

AA
+1 +1

n p
0 -b/w 0
-1 -1
a = hardlim (n) a = hardlim (wp + b)
Hard Limit Transfer Function Single-Input hardlim Neuron
a a

A
+1 +b
-b/w
n p
0 0
-1
a = purelin (n) a = purelin (wp + b)
Linear Transfer Function Single-Input purelin Neuron 3
2 Transfer Functions

a a

AA
+1 +1

AA
n p
0 -b/w 0

-1 -1
a = logsig (n) a = logsig (wp + b)
Log-Sigmoid Transfer Function Single-Input logsig Neuron

4
 2 Multiple-Input Neuron

Inputs Multiple-Input Neuron Input Multiple-Input Neuron

p1
p2
AA AA
w1, 1
AA
AA
p
W
AA
AA
a

AAAA AA
Rx1 1x1

Σ
p3 n a 1xR n
f f

AA
1x1
w1, R b 1 b
pR
1x1
1 R 1
a = f (Wp + b) a = f (Wp + b)

Abreviated Notation

5
2 Layer of Neurons

AA A
Inputs Layer of S Neurons

AAA
n1 a1
w1,1
Σ f

AA A
p1
b1

AA A
p2 1
n2 a2
Σ f

AA A
p3
b2

AAA
1
pR nS aS
wS, R Σ f
bS
1

a = f(Wp + b)
6
2 Abbreviated Notation

Input Layer of S Neurons w 1, 1 w1, 2 … w 1, R

A AA
w 2, 1 w2, 2 … w 2, R
W =
p a

A AA
Rx1 W Sx1
w S, 1 wS, 2 … w S, R
SxR n
f

A AA
Sx1

1 b
Sx1
p1 b1 a1
R S
p2 b2 a2
p = b = a =
a = f(Wp + b) pR bS aS

7
2 Multilayer Network

AA AA AA AA AA A
Inputs First Layer Second Layer Third Layer

p1 AAAA
w 11,1
Σ
b11
n11
f1
a11

AAAA
w 21,1
Σ
b21
n21
f2
a21 w 31,1

AAA Σ
b31
n31
f3
a31

AAAA AAAA AAA

1 1 1
p2 n12 a12 n22 a22 n32 a32
Σ Σ Σ
AA
f1 f2 f3
p3 b12 b22 b32

AA AA AA AA AA A
1 1 1
pR n1S 1 a1S 1 n2S 2 a2S 2 n3S 3 a3S 3
Σ Σ Σ
AAAA AAAA AAA
w 1S 1, R f1 f2 f3
w 2 2
S ,S
1 w 3 3
S ,S
2

b1S 1 b2S 2 b3S 3

1 1 1

a1 = f 1 (W1p + b1) a2 = f 2 (W2a1 + b2) a3 = f 3 (W3a2 + b3)

a3 = f 3 (W3f 2 (W2f 1 (W1p + b1) + b2) + b3)

8
2 Abreviated Notation

Hidden Layers Output Layer

Input First Layer Second Layer Third Layer

AA
AA AA
AA AA AA AA AA
AA AA AA AA
p a1 a2 a3
Rx1
W1 S1 x 1
W2 S2 x 1
W3 S3 x 1
n 1 n 2 n3

AA AAAA AAAA AA
S1 x R S2 x S1 S3 x S2
S1 x 1
f1 S2 x 1
f2 S3 x 1
f3
1 b1 1 b2 1 b3
S1 x 1 S2 x 1 S3 x 1
R S1 S2 S3

a1 = f 1 (W1p + b1) a2 = f 2 (W2a1 + b2) a3 = f 3 (W3a2 + b3)

a3 = f 3 (W3 f 2 (W2f 1 (W1p + b1) + b2) + b3)

9
2 Delays and Integrators

Integrator

AA
Delay

AA
u(t) a(t)
u(t) a(t)
D
a(0)
a(0)
t

a(t) = u(t - 1) a(t) = u(τ) dτ + a(0)

0

10
2 Recurrent Network
Initial
Condition Sym. Sat. Linear Layer

p
Rx1 AA
W
SxR
n(t + 1)AA AA
AAAA
a(t + 1)
D
a(t)

A AA
Sx1 Sx1 Sx1

1 b
Sx1
S S

a(0) = p a(t + 1) = satlin (Wa(t) + b)

a ( 1 ) = satlins ( Wa ( 0 ) + b ) = satlins ( Wp + b )
a ( 2 ) = satlins ( Wa ( 1 ) + b )

11