High Quality
Open the downloaded document, and select print from the file menu (PDF reader required).
Fast Cross-Validation via Sequential Analysis
Tammo KruegerDanny PankninMikio Braun
Machine Learning GroupTechnische Universitaet Berlin
16.12.2011 Big Learning Workshop
Motivation
Cross-validation is an indispensable tool for applied MLbut unfortunately very time consumingExample
fitted Q iteration
:
Cross-Validation
10
×
10
parameter
×
10
fold
×
50
max. iter.
×
10
reps.
= 500
,
000 reg. problemsDirectly optimizing the error landscape to avoidcalculations difficult due to noiseOur approach: use increasing subsets of the training data
1
smaller subsets
→
less training time
2
more training data
→
better error estimate
3
relative behavior of parameter configurations converges
Motivation – Main Idea (Average over 500 Reps.)
log
(
σ
)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
−4 −3 −2 −1 0 1 2
Size
q
501002505007501000
Add a Comment