Homework-3

GNR-652
February 2019

1. [Linear model] Given a set of data points {xn }, we can define the convex
hull as the set of all points given by

x = Σαn xn (1)
n

with non-negative α and Σαn = 1. Similarly, consider a second set of points

n
{ym } together with their respective convex hull. Prove that in order to have
{xn } and {yn } linearly separable, the respective convex hulls should not inter-
sect.

2. [Decision tree] Can the function φ(x) = max(x, 1 − x), 0 ≤ x ≤ 1 be

considered as the impurity function in DT construction? Justify.

3. [Maximum likelihood] Consider we have a sample of data points from

the real line {x1 , x2 , ·, xn }. Each sampled from the p.d.f p(x) having the follow-
ing form:

p(x) = α exp(−αx), x ≥ 0 (2)

and 0 otherwise.
Find the maximum likelihood estimate of α.

4. [VC diension] Prove that a rectangular classifier has a VC dimension of ≥ 4.

5. [KNN classifier] Deduce the time complexity of KNN in terms of O.