© All Rights Reserved

0 views

© All Rights Reserved

- quiz-eecs2602.pdf
- Ages Solved Examples and Exercise
- grade 12 calculus syllabus
- C4 C MS
- Model Paper 4 Em
- solf
- TN-305 Components of Torsion FP
- Assignment 4, Valentin Perlov
- Lectures Chapter 2C
- Conq Part 1
- Markov Model of Forecasting
- Probability
- THE FIRST CONSTANT OF SMARANDACHE
- iit books
- Babuska81
- Mark Scheme Add Maths P2 PPT 2012
- Trabajo
- hw13soln
- Ramin Shamshiri ABE6981 HW_05
- Calculus

You are on page 1of 2

distributions by ρ0 ( x ) = p( x |Y = 0), ρ1 ( x ) = p( x |Y = 1). The Bayes optimal classifier is

(

1, ρ1 ( x ) > ρ0 ( x )

f ∗ (x) =

0, otherwise.

A plug-in classifier is (

1, ρe1 ( x ) > ρe0 ( x )

f (x) =

0, otherwise.

We will use the following upper bound for the classification error. For a proof see next page.

Lemma 1.

1 1

Z Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ |ρ0 ( x ) − ρe0 ( x )| + |ρ1 ( x ) − ρe1 ( x )|.

2 2

Now, our classifier first estimates θ, and then approximates the class conditional distributions using

θ̂. Therefore, ρe0 ( x ) is the density of N (−θ̂, σ2 I) and ρe1 ( x ) is the density of N (θ̂, σ2 I). Using Pinsker’s

inequality we have

s

1 Dkl N (θ, σ2 I)kN (θ̂, σ2 I) kθ − θ̂ k2

Z

|ρ0 ( x ) − ρe0 ( x )|dx ≤ =

2 2 2σ

Similarly,

1 kθ − θ̂ k2

Z

|ρ1 ( x ) − ρe1 ( x )|dx ≤

2 2σ

Therefore, we obtain

kθ − θ̂ (( X1 , Y1 ), . . . , ( Xn , Yn ))k2

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ E(X1 ,Y1 ),...,(X1 ,Yn ) . (1)

2σ

σ2 I

Now we have E[θ̂ ] = θ and Cov[θ̂ ] = n .

dσ2

= tr(E[((θ̂ − θ )(θ̂ − θ )T )]) = tr(σ2 I) =

n

By Jensen’s inequality we get r

d

q

E[kθ̂ − θ k2 ] ≤ E[kθ̂ − θ k22 ] =σ (2)

n

Combining (1) and (2) we obtain

r

∗ 1 d

E[1{ f ( X ) 6= Y }] − E[1{ f ( X ) 6= Y }] ≤

2 n

and finally using the Bayes error that we computed in class we have

r

σ2 1 d

E[1{ f ( X ) 6= Y }] ≤ +

kθ k22 2 n

1

Lemma 2.

1 1

Z Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] ≤ |ρ0 ( x ) − ρe0 ( x )| + |ρ1 ( x ) − ρe1 ( x )|.

2 2

Proof. For any classifier f we have

1 1

Z Z

E[1{ f ( X ) 6= Y }] = 1{ f ( x ) = 1}ρ0 ( x )dx + 1{ f ( x ) = 0}ρ1 ( x )dx

2 2

1 1

Z Z

= 1{ f ( x ) = 1}ρ0 ( x )dx + (1 − 1{ f ( x ) = 1})ρ1 ( x )dx

2 2

1 1 1

Z Z

= + 1{ f ( x ) = 1}ρ0 ( x )dx − 1{ f ( x ) = 1})ρ1 ( x )dx

2 2 2

1 1

Z

= + 1{ f ( x ) = 1}(ρ0 ( x ) − ρ1 ( x ))dx

2 2

Therefore, for the classifiers f , f ∗ we have that

1

Z

E[1{ f ( X ) 6= Y }] − E[1{ f ∗ ( X ) 6= Y }] = (1{ f ( x ) = 1} − 1{ f ∗ ( x ) = 1})(ρ0 ( x ) − ρ1 ( x ))dx

2

Now we prove that

and ρe0 ( x ) < ρe1 ( x ). Therefore, the LHS of inequality (3) is

- quiz-eecs2602.pdfUploaded bysiarwafa
- Ages Solved Examples and ExerciseUploaded byNaveenRathi
- grade 12 calculus syllabusUploaded byapi-263788371
- C4 C MSUploaded byshah143
- Model Paper 4 EmUploaded byReddy Gmd
- solfUploaded byMuhammad Suleman
- TN-305 Components of Torsion FPUploaded byPartha Pratim Roy
- Assignment 4, Valentin PerlovUploaded byKaterina Pearl
- Lectures Chapter 2CUploaded byShivneet Kumar
- Conq Part 1Uploaded byBelinda Jade
- Markov Model of ForecastingUploaded bysheziboy
- ProbabilityUploaded byRushina Singhi
- THE FIRST CONSTANT OF SMARANDACHEUploaded byRyanElias
- iit booksUploaded byshubham
- Babuska81Uploaded byDuong Pham
- Mark Scheme Add Maths P2 PPT 2012Uploaded byleekoonheng
- TrabajoUploaded byNelly Liliana
- hw13solnUploaded bypja752
- Ramin Shamshiri ABE6981 HW_05Uploaded byRaminShamshiri
- CalculusUploaded byMarjenneil
- The Existence of a Periodic Solution of a Boundary Value Problem for a Linear SystemUploaded byIOSRjournal
- Java PrintingUploaded byDavidThomas
- Time and WorkUploaded byVicky Vinay
- Pahang Add Math P1 Ans 2008.pdfUploaded bygigi_pan
- Closed profile spline filter libraryUploaded byRohith M. Athreya
- EE1991.pdfUploaded byArpit Patel
- june-2016-cape-pure-mathematics-u1-p2.pdfUploaded byMackenzie Kangoo
- back to basic 1Uploaded byazri
- A Level Paper B3 mark scheme.docxUploaded byJamie Heredge
- Unit 2 2016 Paper 2 SolutionsUploaded byRenee Roberts

- Survival Communications FAQUploaded byBurt Gummer
- Eric Johnson the Art of Guitar PDFUploaded byYesenia
- 15 P's of MarketingUploaded byrainavr1
- Test 1_Trav C1Uploaded byMariaPopova
- hard of hearing notesUploaded byapi-288159071
- UWCABook.pdfUploaded byDeonisis Versola
- salesforce_creating_on_demand_apps.pdfUploaded byCecilio Gutierrez
- question-paper-paper-2h-june-2014Uploaded byapi-316936840
- 6 Introduction to Petrophysics August 2015Uploaded byCassielAlexzylalas Liamba
- 0199912100k.pdfUploaded byOscar Frizzi
- Lab1 - StudentUploaded byEmily Owen
- Group and Panel DisscusionUploaded bySovan Mishra
- Biology Study ListUploaded byJocelyn Botero
- Caterpillar DozerUploaded byAgung Ardhana
- caleb stokols resumeUploaded byapi-239144738
- Alabama-Power-Co-Rate-BTA---Business-Time-Advantage-(Optional)Uploaded byGenability
- 160522 Hw MetaformersUploaded byDrunegar
- Sanctuary magazine issue 14 - Reborn from fire - Callignee, VIC green home profileUploaded bySanctuary Magazine
- Cycloidal trajectorysimulinkUploaded byUNsha bee kom
- Roman Infrastructure[1]Uploaded byNamita Gupta
- Environmental PollutionUploaded byJAIN2013
- Vulcraft 2013 5 Steel Joist and Joist GirdersUploaded bymurdicks
- DSPACE RESSTOR UBANTUUploaded byAshok
- Module Configuration Template - General ConfigUploaded byPavan Hurukadli
- Food and Beverage Service PowerPoint Presentation on cigar and cigaretteUploaded byyash agarwal
- computational financeUploaded byg1256985
- Handout 1 for Chapter 7_ Recognition and Valuation of Accounts ReceivablesUploaded byJamie
- ts_101724v080900p (1)Uploaded byscropio_indian
- N.Bar Price MenuUploaded byCassandra Pallagud
- Endotracheal Intubation Without Muscle Relaxants in Children Using Remifentanil and Propofol_ Comparative StudyUploaded byYanna Rizkia

## Much more than documents.

Discover everything Scribd has to offer, including books and audiobooks from major publishers.

Cancel anytime.