Professional Documents
Culture Documents
Lecture 2: Single Layer Perceptrons: Kevin Swingler
Lecture 2: Single Layer Perceptrons: Kevin Swingler
Kevin Swingler
kms@cs.stir.ac.uk
Wj1
I2
Aj
Yj
I3
In
1.
2.
3.
Wjn
I1i
W1i
I2i
Yi
wij
I3i
Neuron k
Iki
Iij
i
Synapse ij
Neuron i
Neuron j
I ki = Yk wki
Yi = sgn(
I ki i )
I ij = Yi wij
k =1
The Perceptron
We can connect any number of McCulloch-Pitts neurons together in
any way we like
An arrangement of one input layer of McCulloch-Pitts neurons feeding
forward to one output layer of McCulloch-Pitts neurons is known as
a Perceptron.
i
:
:
N
wij
:
:
M
Y j = sgn( Yi wij j )
i =1
NOT
in
out
OR
AND
in1
in2
out
in1
in2
out
?
?
Decision Surfaces
Decision surface is the surface at which the output of the unit is
precisely equal to the threshold, i.e. wiIi=
In 1-D the surface is just a point:
I1=/w1
Y=0
I1
Y=1
I1 w1 + I 2 w2 = 0
which we can re-write as
w
I2 =
1 I1
w2 w2
AND
w1=1, w2=1, =1.5
OR
AND
I1
I2
out
I1
I1
I2
out
I1
(1, 0)
(1, 1)
(1, 1)
(1, 0)
(0, 0)
(0, 0)
(0, 1)
(0, 1)
I2
I2
I2
out
I1
I2
I2
Solution: either change the transfer function so that it has more than one
decision boundary, or use a more complex network that is able to generate
more complex decision boundaries.
9
ANN Architectures
Mathematically, ANNs can be represented as weighted directed graphs. The
most common ANN architectures are:
Single-Layer Feed-Forward NNs: One input layer and one output layer of
processing units. No feedback connections (e.g. a Perceptron)
Multi-Layer Feed-Forward NNs: One input layer, one output layer, and one or
more hidden layers of processing units. No feedback connections (e.g. a
Multi-Layer Perceptron)
Recurrent NNs: Any network with at least one feedback connection. It may, or
may not, have hidden units
Further interesting variations include: sparse connections, time-delayed
connections, moving windows,
10
Multi-Layer
Feed-Forward
Recurrent
Network
11
1
f ( x) =
0
if x 0
if x < 0
x
Piecewise-Linear Function
f ( x) = x + 0.5
if x 0.5
f(x)
if 0.5 x 0.5
if x 0.5
Sigmoid Function
f ( x) =
f(x)
1 + e x
x
12
Ii wij j = I1 w1 j + I2 w2 j + K+ In wnj j
i =1
i =1
i =0
i =1
i =0
13
Length
Class
10.0
Lorry
20.0
Lorry
5.0
Van
2.0
Van
2.0
Van
3.0
Lorry
10.0
Lorry
15.0
Lorry
5.0
Lorry
How do we construct a neural network that can classify any Lorry and Van?
14
w0
w1
Mass
Length
16
Supervised Training
1.
2.
3.
4.
5.
18
- Compute output y
- Compute error, =(ytarget y)
- Use the error to update weights as follows:
w = w wold=**x
or
wnew = wold + **x
where is called the learning rate or step size and it determines how
smoothly the learning process is taking place.
3.