Model Selection

Structural risk MinimizationPrinciple

Intro limits of uniform

IJ convergence
In previous lectures ve son chat
or of
classes
of function of finite cardinality
Rademacher complexity had the
decreasing and thus are
Uniform convergence Property ocp

PAC learnable with ERM dare precisely

For any distribution P

for any SE 30,1C for S P ffff
I RCF Ès f I S ECN 8 F w.p.AT

Gif IFIL ECNS.FI IKE

otherwise E NS F ZPiadsll.FI 4czhzF

In case Fis it Redemander

finie on
Cohen Nsx have
average tends too we

UCP and PAC learnability

There exists n E s such that
for distribution P
any if N m ES the
for S P
Such Rep Fdj IRI Rgt LE w.p.tt
8
PAS IRCfs RCH ICE up 1
where
fscanggiIRssyl ft aggE218
Unfortunately many classes such
as

linear classifies with

decision trees on

no restriction on their no n
do not
Rademacher averages No UCP Nop
have finite
Let us show this
Rd ne R
Lit amor oe

DT decision trees of any depthwhoseportig

Rads F
I E
pl je Érik D
j'Eunifk
Rad E
info kaEoiotxi
Liv
I
En.nl aElEED

Riad DT

Illustration with IS 3in R2

x
dû
À nuit
nôtre sa
 Y IN ÏÏÏÏÏ

Y 1N
yô à Raids DT 1
Does not decrease with N
Let us show that for bounded norm linear
classifies and bounded non exoyls Rôds
is finite

Cet arson Holleck

Ling
Assume Kei Maille E XL

Radskinat's Er nl feEoiotxi
Er at ft Eoiril
EEf oYIeoTErin
tini

Es Ell Erin p
FETEE
JETÉ Jensen inequality

TÊT