Professional Documents
Culture Documents
Learning From Data Solutions To Selected Exercises: N N M J J N 2
Learning From Data Solutions To Selected Exercises: N N M J J N 2
2
M
N
X
X
1
t n
wj xjn
E(w) =
2 n=1
j=0
To obtain the normal equations we take the partial derivatives with
respect to the weights and equate to zero. Application of the chain rule
gives
N
M
X
X
E
t n
=
wj xjn xin
wi
n=1
j=0
Equating this expression to zero gives
N
N
M X
N
M
X
X
X
X
t n
wj xjn xin =
tn xin +
xjn xin wj = 0
n=1
n=1
j=0
j=0 n=1
So we get
M X
N
X
(xn )i+j wj =
j=0 n=1
N
X
tn (xn )i
i = 0, . . . , M
n=1
N
M
X 2
1X
w
(y(xn , w) tn )2 +
2 n=1
2 j=0 j
N
X
y(xn , w)
y(xn , w)
y(xn , w)
+wi =
tn
wi
wi
n=1
n=1
Plugging in y(xn , w) =
M X
N
X
PM
j=0
i = 0, . . . , M
wj xjn we get
(xn )i+j wj + wi =
j=0 n=1
N
X
tn (xn )i
i = 0, . . . , M
n=1
p(g|o) =
1.5
var[f (x)] = E[(f (x) E[f (x)])2 ]
2
(1.38;expand)
2
(1.39)
and
1.6
cov[x, y] = E[xy] E[x]E[y]
(1.41)
XX
x
xy p(x)p(y)
x p(x)
y p(y) = E[x]E[y]
XX
x
x p(x, z) +
XX
z
z p(x, z)
X X
X X
p(x, z)
z
p(x, z) +
x
x p(x) +
In exercise 1.5 we have shown that var[x + z] = E[(x + z)2 ] E[x + z]2 .
Since E[x + z] = E[x] + E[z] as we have shown above, we get
var[x + z] = E[(x + z)2 ] E[x]2 2E[x]E[z] E[z]2
Left to determine E[(x + z)2 ]:
XX
(x + z)2 p(x, z)
E[(x + z)2 ]
x
XX
XX
x
(independence/expand)
x2 p(x)p(z) +
x p(x)
XX
x
p(z) + 2
2xz p(x)p(z) +
X
x
x p(x)
XX
z
X
z
z p(z) +
z 2 p(x)p(z)
z 2 p(z)
X
x
p(x)
1.13
#
N
N
1 X
1 X
2
E
(xn ) =
E[(xn )2 ]
N n=1
N n=1
"
N
1 X
1
var(xn ) = N 2 = 2
N n=1
N