"

()046195
'

:4
4.1
4.2
4.3
4.4
4.5
4.6

: ,

()

( HTF: 2.7-2.9, 7.2, 7.10 : ).

4.1

( :)1

, D {xk , yk }k 1
n

, xk

. yk

D , .
xk yk , ( :) . xk -
,

( . ) Y , X :

( ,) f
) y f ( x

. x

) f ( " " "".

, y :

[a, b]

x , , .
x :
:

. x xk

) , xk ( xk1 ,, xkd . xkj

: ,:

{xk , yk }kn 1 , {x (i ) , y (i ) }iN1 , {xi , yi }iN1 , {xi , ti }iN1

x j

, xk

) . x ( x1 ,, xd ,

( k / i .) j

 (")

4.2

: 4 -

- : ,
.
f .
D {xk , yk }k 1 . ) , f ( x
n

(:)Empirical Risk

1 n
)) L( yk , f ( xk
n k 1

ER( f )

L ( )Loss y . y f ( x ) -

2
)) . L( y, f ( x )) 12 ( y f ( x :

1 n
ESE ( f )
( yk f ( xk ))2

2n k 1
, , (:)Sum of Square Errors
n

SE ( f ) ( yk f ( xk )) 2
k 1

(:)Empirical Risk Minimization

. D {xk , yk }k 1
n

}) { f ( x

) min ER( f
f

f ,
" x" . .

: { f ( x), } :
:

f ( x) 0 1x1 d xd

. (1,, d ) ( .) x
T

 : 4 -

(")

)f ( x) 11 ( x) M M ( x
, (1 ,, M ) ) 1 ( x ),, M ( x -

, ( ) . , ,
M

. ( .) x
, ( -
) , , ( ) . ""
.

, .

(:)Empirical Risk Minimization

. D {xk , yk }k 1
n

} { f , :

) min ER( f

, .
f ,
" x" . .

4.3

, ( ) ,
. (
) D ,
, . . ,
.
:

Y f0 ( X )

 : 4 -

(")

f 0 ( ) , ,
("") .0

, ) N (0, 2

.:

) (Y | X x, ) ~ N ( f0 ( x), 2
( x -) .
, , f 0 ( )MMSE y x:

)y MMSE ( x) E (Y | x, ) f0 ( x
. f 0

f 0 }

{ f ( x ), .

- , .
( )MLE . D {xk , yk }k 1
n

, Yk f ( xk ) k ::

1
) exp( ( yk f ( xk ))2 / 2 2
2

p(Yk yk | xk , )

) p( yk | xk ,

} { k , :

) L( ) p({ yk }1n |{xk }1n , ) k 1 p( yk | xk ,

n

) exp( ( yk f ( xk ))2 / 2 2
k 1

) L( ,
n

arg max L( ) arg min ( yk f ( xk ))2

k 1

MLE

,
) . SE (

 (")

4.4

: 4 -

- , .
- (.)Kernel methods
) f ( x0 x0
.
(
:)Nadaraya-Watson

K ( x0 , xk ) yk
f ( x0 ) k n1
) k 1 K ( x0 , xk
n

) K ( x0 , x x0 x . x0 -
:

|| x x0 ||2
K ( x0 , x ) exp

.
( - ).
} D {xi , yi ,
. . x0
.Lazy Learning
* :
. ) f 0 ( x0

) , f ( x, ) f ( x0 ) f ( x0 ,

min over

E ( ) K ( x0 , xi )( yk f ( xk , ))2
k 1

) f ( x, . x - :
f ( x, ) (.)Nadaraya-Watson

, f ( x, ) 0 x ,
T

-.Locally Linear Regression ,

 (")

4.4

: 4 -

,
. ,
.
.
( , , -)
- .
, , ,
.
.
( ) , ,
. ( )
:

( ) ""
. : .

( ) (
) .:
() 3 .
700 .

(:)Bias-Variance Tradeoff

 : 4 -

(")

( )bias
( ,
) . .

) (variance

. .

, ,
. ( 2.11
,)HTF:2001 .

()

.
, , ,
( ) . ,
.
:
. , ("") ,
.
) (tradeoff .

 : 4 -

(")

.
.
, " "
( : ) . ,
.
()
. :

( )
""

( )
""

 (")

4.5

: 4 -

: ;
; . ,
.
, , .
, ,
.
:
- ,
. :
. ) (Training set ("
).
.

( (Validation set ,
, ,'.

) (Test set .

.
, .
.
,
. ( 60% :)( 20% ,)( 20% ,).
,
.
. , 40%
. .
) :(Cross Validation K-

() . ( , K

) i 1, :

.1 , . i -
.2 . i -
K .
.
K 5 .10 ( K n
) , (". ("Leave-one-out cross-validation