Professional Documents
Culture Documents
Roger Grosse
z = wx + b L=1
y = σ(z) y =y −t
1 z = y σ 0 (z)
L = (y − t)2
2 w =zx
b=z
t1 = wx
Original program:
z = t1 + b
z = wx + b t3 = −z
1 t4 = exp(t3 )
y=
1 + exp(−z)
t5 = 1 + t4
1
L = (y − t)2 y = 1/t5
2
t6 = y − t
t7 = t62
L = t7 /2
Autograd’s fake NumPy module provides primitive ops which look and
feel like NumPy functions, but secretly build the computation graph.
They wrap around NumPy functions:
Example:
Examples
Matrix-vector product
z = Wx J=W x = W> z
Elementwise operations
exp(z1 ) 0
y = exp(z) J=
..
z = exp(z) ◦ y
.
0 exp(zD )
https://github.com/HIPS/autograd#end-to-end-examples