You are on page 1of 2

CS519-F14 Homework 1: Part 2 and 3

Guilherme Palma Peixoto
Stevens Institute of Technology
September 18, 2014
Part 2 R1 (x) = R(α1 |x) = 0.6 · 105 · √12π
Given two samples C1 and C2 of the same random variable, we want to
demonstrate how the mean and variance of union of the samples are related
to the mean and variance of each sample individually.
P|X|

Mean The mean of any given set X can be written as meanx = n1 i=1 xi .
Let C1 = {x1 , x2 , ..., xN1 } and C2 = {y1 , y2 , ..., yN2 }, where N1 , N2 stands for
the cardinality of each set respectively. Then, the mean of C1 ∪ C2 can be
written as:
b
c
a
= ab b+c
= ac b+c
then:
Lemma Given that b+c
1
(x1 + ... + xN1 + y1 + ... + yN2 )
N1 + N2
N1
N2
X
X
1
( xi +
=
yj )
N1 + N2 i=1
j=1

mean(C1 ∪C2 ) =

=

N1
N2
X
X
1
1
xi +
yj
N1 + N2 i=1
N1 + N2 j=1

N1
N2
N1
1 X
N2
1 X
=
xi +
yj
N1 + N2 N1 i=1
N1 + N2 N2 j=1

N1
N2
mean1 +
mean2
N1 + N2
N1 + N2
N1 mean1 + N2 mean2
=
N1 + N2

=
mean(C1 ∪C2 )

1

for each element e of the test set. Let vari be the variance of the training set i and µi be the mean of the training set i. Variance The demonstration of the relationship is eerily similar to the proof above. Then the decision rule was elaborated as below: P (Ci |x) = p(x|Ci )p(Ci ) Then. it was possible to estimate the likelihood p(x|Ci ) utilizing the mean and variance of each set. e belongs to class 2 (where xe is the value of the feature x present in the test set for the current element e). The prior probabilities were estimated as: p(Ci ) = |Ci | |C1 ∪C2 | Assuming the Gaussian distributions.Which is basically the weighted average of the means. 2 . else. then decide that e belongs to class 1. Part 3: Decision Rule Assuming that the both training sets follow Gaussian distributions. Let also Ci be the set of elements that are in the class of the training set i. the decision rule was formulated. if P (C1 |xe ) > P (C2 |xe ). The proof (in which a few steps are omitted) follows: var(C1 ∪C2 ) = N1 N2 X X 1 ( (xi − mean1 )2 + (yj − mean2 )2 ) N1 + N2 i=1 j=1 N1 N2 X X 1 1 2 = (xi − mean1 ) + (yj − mean2 )2 N1 + N2 i=1 N1 + N2 j=1 = N1 N2 N1 1 X 1 X N2 (xi − mean1 )2 + (yj − mean2 )2 N1 + N2 N1 i=1 N1 + N2 N2 j=1 N1 N2 var1 + var2 N1 + N2 N1 + N2 N1 var1 + N2 var2 = N1 + N2 = var(C1 ∪C2 ) Which is basically the weighted average of the variances.