Professional Documents
Culture Documents
Dr Shabnam Kadir
Neural Networks and Deep Learning Tutorial 1
(eh −e−h )
6. Show that for g(h) = (eh +e−h )
, g 0 (h) = 1 − g(h)2 .
∂J
7. For the following loss functions J(y) compute ∂y
(i) J = 12 (y − y ∗ )2
(ii) J = y ∗ log(y) + (1 − y ∗ ) log(1 − y)
with
σ := tanh(z).
Let the training set by (xµ , tµ ) for µ = 1, . . . , M , where x is the input vector and tµ is the desired target
output. Let the error function be:
M M
1 X1 1 X µ
E := (y(xµ ) − tµ )2 ≡ E .
M µ=1 2 M µ=1
Derive an incremental learning rule from E using the gradient descent method applied separately to each
µ
training example. (Hint: ∆wiµ := η ∂E µ 1 µ µ 2
∂wi , where E = 2 (y(x ) − t ) and use the expression in Question
5).
Page 2