Neural networks

Feedforward neural network - artificial neuron

September
Abstract6, 2012

hugo.larochelle@usherbrooke.caSeptember
September
6, 2012 6
iversite de Sherbrooke
September
6,
2012
Septe
ARTIFICIAL
NEURON
Math for my slides Feedforward
neural network.
Abst
arochelle@usherbrooke.ca
September
6, 2012
Abstract
2

Abstract
P
Math for my slides
Feedforward
neu
Topics: connection weights, bias,>activation function
Abstract
x) = bMath
+ for
w
x
=
b
+
w
x
my
slides
Feedforward
neural
network.
i
i
Abstract Abstrac
i
Math
for
my
slides
Feedforward
neural
network.
Neuron
September
6,
2012
P
pre-activation (or input activation):
>
Math
for my
slides
Feedforward
neural
network.
P
Abstract
P
Math
for
my
slides
Feedforward
neural
network.

a(x)
=
b
+
w
x
=
b
+
w
x
i
i
P
i Feedforward neural n
>
Math
for
my
slides
(x)
=
g(a(x))
=
g(b
+
w
x
)
>
a(x)
=
b+
+ iwwi ixxi i==b bi++wiwix x P
Math for my slides Feedforwar
a(x)
=
b
P
i
P
Math for my slides
Feedforward neural network.
>>
P
a(x)
= b=
+ b+h(x)
=
b
+
w
x
P
P a(x)
wi=
x
=
b
+
w
x
g(a(x))
=
g(b + > i wi xi )
i wiix
i i
P
P
>
h(x)
=
g(a(x))
=g(b
g(b++ i>w
wxiix)a(x)
x
b
w
w
=
b
+
w
x
=
b
+
w
x
i)
h(x)
=
g(a(x))
=
1
d
1
d
i
i
P
i
i

a(x)
=
b
+
w
x
=
b
+
w
P
i
i
i

a(x)
=
b
+
w
x
=
b
+
w
x
i
Abstract
i i
Neuron (output)i activation
h(x)
= g(a(x))
=
g(b
+
w
x
)
h(x)
= g(a(x))
=
g(b
+
w
x
i
i
i
i )P
i
i

x
x
b
w
w
1
d
1
d
P
P
xxh(x)
b w
w11 =wwdg(b
x=
11 x
d g(a(x))
d +

h(x)
=
g(a(x))
=
g(b
+
w
x
)

h(x)
=
g(a(x))
=
g(b
+
w
x
)
i
i
i
i
1
i
dforward neural network. i
i wi

x1 xx1d xbd wb 1 ww
1 d wd

w
w
w
x1 xd b w 1 w d

x
x

x
x
1
d
1
d

w
>

w
+ w arex the connection weights

{
{{w

P
{
) b is the neuron bias

g()
b
w
x
)
i
i
{ isbbcalled the activation function
g()
ig()

g()
b
{
g() b {

(x) = g(a(x))

...

 h(x) = g(a(x)) = g(bi i wixi)

Abstract

ARTIFICIAL NEURON
-1

Abstract

Abs

0
Math for my slides Feedforward neural network. -1
Math for my slides
Feedforward
neural
network
Math
for
my
slides
Feedforward
ne
0
P
-1

>

a(x) = b +

1
w
x
=
b
+
w
x
i
i
P
i weights, bias, activationP
x1 >
Topics: connection
function
>
Pa(x) = bi i wia(x)
xi ==bbi + wi wxzikxi = b + w x
h(x) = g(a(x)) = g(b + i wi xi )
P
P

h(x)
=
g(a(x))
=
g(b
+
w
x
)
i
i
i
h(x) = g(a(x)) = g(bi i wi xi )
w
sortie k
w

w
y
1
x2
{
1
-.4
-1
w

{
range determined
kj
y
y
1 .7
2
0
by g() b
1
bias b only
g()
-1
-1.5

w
{

-1

-1
1

x1

(from Pascal Vincents slides)

biais

changes the
position
of
.5
the riff

cach

1 1

wji

entre