assignment 3 of comp 531

© All Rights Reserved

25 views

assignment 3 of comp 531

© All Rights Reserved

- 7 2 using functions
- NMTC final stage Sub Junior 2014
- independent practice video 10
- CAT Quant Test 2
- engr2056_dec07
- Ch 4 Know It Notebook
- August 2009 Bachelor of Computer Application (BCA)
- 12.5 Markov Chains 2 (or Models)
- Sequences of pyramidal numbers
- 2da Practica de Control
- Lecture 18
- Pstricks Add Doc
- Ujian Bulanan 1 Matematik Tambahan Tingkatan 5
- Garry D. Divinagracia_Evaluation and Operations of Functions
- img 0001
- Spm f4 Ch4 PDF Final
- Basic Calculus 3rd Tr Quiz#1
- lesson 57 jan 21 p5
- Flexibility Method (Double Doi)
- Excel Formulas

You are on page 1of 3

Student: Johannes Brustle, ID: 260580221

Due march 7th 2016

Assignment 3

Question 1.

Let x0 the origin and xi = ei , the i-th standard vector. We will show that

the set S = ni=0 xi can be shattered by linear threshold functions. Given

U S let U the characteristic function of U mapping to {1, 1}. Now set

w

~ = (U (x1 ), ..., U (xn )) and b = U (x0 )/2. For xi , 1 i n, wx

~ i=1b

if xi U and w

~ xi = 1 < b if xi

/ U . Also, w

~ x0 = 0 < b = 1/2 if x0

/U

and w

~ x0 = 0 b = 1/2 if x0 U . That is, S can be shattered by the

class of linear threshold functions.

Question 2.

1. Let x0 < x1 < < x2k be 2k + 1 distinct points on the real line, call this

set X. Consider subset S ={xi : i is even}. Then |S| = k + 1. Then every

x2l S for 0 l n, but no two elements of S can be in the same closed

interval since x2l1 and x2l+1 are not. That is, the VC dimension stated in

the question is < ((2k) + 1).

Now let Y = X x2k , then |Y | = 2k. In this case, let U Y .

Then consider closed intervals:

L = {[xi , xj ] : ((xs U ), i s j)(i = 0xi1

/ U )(j = 2k1xj+1

/

U )}. Elements of L are closed disjoint intervals containing no point in Y U

and all points of U . Suppose |L| > k. But by definition of L, there must

be at least one point in Y U between each interval of L, hence at least k.

Also every interval of L contains at least one point in Y . But this implies

|Y | > k + k, contradiction. Hence |L| k so the VC dimension is 2k.

2. Let X = {e1 , , en , e1 , . . . , en } in Rn . Let I and J be two arbitrary

subsets of [n]. Consider X S = {ei : i I} {ej : j J}.

Also, hyper-rectangle R = [a1 , b1 ] [an , bn ] where aj = 1 if j J, else

aj = 0. Similarly, bi = 1 if i I, else bi = 0. Then obviously R contains all

points in S and no points in X S. Hence the VC dimension is 2n.

Now suppose we are given Y Rn , Y = {y1 , , y2n+1 }.

Let amin

= min{ai , lower bound of i-th coordinate of yj , 1 j 2n + 1},

i

and bmax

= max{bi , upper bound of i-th coordinate of yj , 1 j 2n + 1}.

i

Since there are 2n + 1 points, yl such that amin

and bmax

are the same for

i

i

Y and Y yl , for all i. Then it is easy to see that @R, hyper-rectangle, such

that R Y = Y yl . This proves VC dimension is < 2n + 1

Question 3.

Note that since the only update rule in Perceptron alg. is w~t+1 w

~ t + y t xt

and yt and xt are integer t in our case, w

~ will be integer. Hence suppose

w

~ = (w1 , , wn ) an integer valued linear seperator with respect to S, meaning the Perceptron algorithm makes no mistakes.

This means w

~ x~1 0 w1 0.

Next, w

~ x~2 < 0 w

~ x~2 1 w1 1 + w2 (1) 1 w1 + 1 w2

w2 1.

By induction wt 0 t. Base case was just covered. P

Suppose the claim holds

for {1, . . . , t 1}. If t is odd, then

w

~ xt 0 = 1 t1

k=1 wk + wt wt 0.

Pt1

If t is even, then w

~ xt 1 = k=1 wk wt wt 1.

More P

precisely, since we now know w

~ positive, from above we get that in fact

t1

wt k=1 wk t.

Claim: wt 2t3 for t 2 Do this by induction on t. Base case: w2 1

223 = 1/2.

Pt1

Suppose

claim

holds

for

{1,

.

.

.

,

t

1}.

Then

w

t

k=1 wk wt

Pt1 k3 Pt3 s1 Pt4 s

t3

2

=

2

.

2

2

=

s=0

s=0

k=2

Hence wn 2n3 and recall that we start with w

~ = 0 in the Perceptron

algorithm and the update rule is w~t+1 w

~ t + yt xt so wn changes by 1, only

if a mistake was made. Conclude that a lower bound for total number of

mistakes made is 2n3 .

Question 4.

1. Let C the target concept, |C| = r. If ~x a positive example, at least one

element of C ouputs 1, else if ~x a negative example, all elements of C output

0. Let WC = {wc1 , , wcr } the weights corresponding to C.

Hence our algorithm, on a negative example, leaves WC unchanged, and on

a positive example for which our algorithm predicts negative, i, 1 i r

such that wci wci 2. Moreover, assume for some j, wcj n. If, on a

positive example, the corresponding element of C outputs 1, w

~ ~x n and

no updates are done. That is, no weights in WC that are n are updated.

Also, weights in WC are never decreased as this would mean a corresponding

element of the target concept output 1 on a negative example.

~ starts out at (1, , 1) and 2k n k < logn

we get that any weight in WC is updated at most (logn) + 1 times. Conclude

that, since every time a mistake occurs on a positive example, at least one

element of WC is updated, after r((logn) + 1) such mistakes, all elements of

C have weight n and so no mistakes can be made anymore.

P

2. Now consider the total sum of all weights at time t: St = ni=1 wi .

Let the number of mistakes coming from positive examples since time t = 0

be Pt . Similarly, the number of mistakes coming from negative examples

since time t = 0 be Nt .

If Pj increases by one at some time j w

~ ~x < n Sj increases by at most

by n. If Ni increases by one at some time i w

~ ~x n Si decreases by

at least n/2.

Note that S0 = n, and St > 0 t. Thus 0 < St n + Pt (n) Nt (n/2)

Nt < 2 + 2Pt .

3. Total number of mistakes is Mt = Nt + Pt . From 1. we know Pt

r((logn) + 1) t, and 2. implies Nt < 2 + 2r((logn) + 1) t. Hence conclude

that Mt < 2 + 3r((logn) + 1) t.

- 7 2 using functionsUploaded byapi-332361871
- NMTC final stage Sub Junior 2014Uploaded byyatishgoyal
- independent practice video 10Uploaded byapi-331338526
- CAT Quant Test 2Uploaded byapi-19859643
- engr2056_dec07Uploaded byMohamed Alqaisi
- Ch 4 Know It NotebookUploaded byspeterlee6253
- August 2009 Bachelor of Computer Application (BCA)Uploaded bybablukumar13
- 12.5 Markov Chains 2 (or Models)Uploaded byVijaya Shree
- Sequences of pyramidal numbersUploaded byAnonymous 0U9j6BLllB
- 2da Practica de ControlUploaded byClinton Estacio Gomez
- Lecture 18Uploaded byHisham Kolaly
- Pstricks Add DocUploaded byRafaelOlivoD
- Ujian Bulanan 1 Matematik Tambahan Tingkatan 5Uploaded byhafaba
- Garry D. Divinagracia_Evaluation and Operations of FunctionsUploaded byGarry D. Divinagracia
- img 0001Uploaded byapi-232572892
- Spm f4 Ch4 PDF FinalUploaded byAina
- Basic Calculus 3rd Tr Quiz#1Uploaded byVlad Vizconde
- lesson 57 jan 21 p5Uploaded byapi-276774049
- Flexibility Method (Double Doi)Uploaded byAmierul Mukmien
- Excel FormulasUploaded byWilliam Ampomah
- Chhattisgarh Board Class 12 Mathematics Sample Paper 3.pdfUploaded byprem sahu
- A FEW OLD PROBLEMS.pdfUploaded bymoka0687
- IJEART01007Uploaded byerpublication
- CXC Revision Test 2Uploaded byBlack Ink Tutorials & Services
- func_eqns.pdfUploaded byYahoo
- func_eqns.pdfUploaded byAbhishek Yadav
- _MCR3UPracticeTest2LevelsbUploaded byFrankie Pulcino
- big-m.nbUploaded byFahima Taj
- MATH 1150 - Assign #6Uploaded byAnthony Basanta
- Equation System SolverUploaded byalvaro

- OpenAir MaOpenairnualUploaded byJosh Burke
- 2_SimpleOnePassCompilerUploaded byRen Zo
- Caltech basics.. sampleUploaded byCristhianares Lonjawon OboLight
- Nastran Advanced DynamicsUploaded byanon_173559594
- HW No 2. February 26, 2017.Docx SolutionUploaded bykhalid
- A Program to Check if a Binary Tree is BST or Not _ GeeksforGeeksUploaded bydeveshmanish
- Pavcnik Et Al 2004Uploaded byPaksi Danurdara
- edma 262 reportUploaded byapi-357538071
- India Bix MCQ'sUploaded byarslanjameelmalik
- UKCAT tips.docxUploaded byAnonymous 3fKxW0CD0
- rhino 3dUploaded bykinglion004
- Module 1 - CirclesUploaded byHenry Languisan
- Lecture 5 Pert-cpm (2)Uploaded byMuhammad Noor
- Fast Estimation of ReboilerUploaded byMarco A. Castillo Ludeña
- Raymond Tallis on ParmenidesUploaded byPedro Durão
- Transient Conduction Heat TransferUploaded byAbubaker Faisal
- Jurnal proses manufakturUploaded byNur Rokhim
- UmlUploaded byshijinbgopal
- Your Introduction to Rapid LearningUploaded bydsfbdfbdsgn
- Advanced Excel ProgrammeUploaded byMufaddal Hussain
- Thermocouple LabReportUploaded byCasey Thompson
- 2000 Distillation Columns. I. the Ponchon-Savarit MethodUploaded bySalim Chohan
- ArticleUploaded byMazda Cai
- 72498_05Uploaded byHai Hai
- A R. Daniels-Introduction to Electrical Machines.pdfUploaded bySalih Ince
- Examen Matemáticas Radicales y Ecuaciones 4º EsoUploaded byJosé Francisco Navarro Martín
- Kelantan Stpm 2012-Mathst Paper 1(Q&A)Uploaded bySK
- Ultimate Tech Analysis HandbookUploaded byDavid Chalker
- 323 Symmetry IntroUploaded byFadjar Mulya
- Gear BasicsUploaded byHarshit Mishra