On-Line Q-Learning Using Connectionist Systems

! "#%$"
&'
(!)+*,).-0/+1!12436587:9;)=<?>@3ACB DACB

EF=GH G?
IF=JLK=MNM
OP24QPR241,S+243JT4UCUWV
X0AC1,S+36>ZY+[C2,\IBP>Z]C243_^>ZR`5aBP[b>cBP2 243_>cBP[de24Q+Ab36Rf1,24BbR
g 3h/+1!QP>@BP[CRijBOPRf362 2 R
X0AC1,S+36>ZY+[C2X0kHlT mn
aBP[Co@ACBPY
a1!Ap>qosrt[jAC3u24BP[N)wv4AC1)xApvC)x/+y.zPBP>@3ACB DACB+u24BP[.)wv4AC1)xAWvC)x/+y

"!#%$ &'#'()$
!#(*&#+',"$
&-$/.0
1234'!56$ 7+'#'8!#%$
'#'()7+''#9:4';
<=
12>*%?@4'7+A
&+'B12*&C8*D&+',6$ !(*+',=+$
,FE-G'>*!#.0GD1#&+D( $ &GD&
G'#,&IH0'#&7J3,&7$
&5K8$
C
>:L$
.'E'!#B,%?:1+'#7+",:&,&&##>F"$
:MN%$
!OJ312*&!#G
:>:#&*':,%;@P+'&Q?0R#,G',7$
E'!#S&T 'G8+',I+':G',R&+'#(+BG'#U,&#$ !
*:&#:4'4',S,V$
7J3,&.$ ,%?R1+'#7+A&9:4'#&,N+'4',
R4''&#-$ .'.'&
T:#6$ &#"6(*'WJ
$ !#,2+'X#:*&"$
&#*Y!#%$ &:RE M5&+U,&M:,&Z;\[3+'#,&.0%?:+'X4',X ]E$ 7CJ3.'.$ ( $
#
'4'V$ !'312*&C:,Y^_4'B!#+$
%?<X#:&*$
'Gà!#!#L$
,bcde*f@,g',#G'&G6#+'#,:T:%;
`DSU,&#G'S$5 4UE G'Oh]:Y$ !(*+',gE$
,GZ$
4'UGZijJWkl%$
'#'(8^L`-$ &C:#',
bcdc*f=E'#'GZ1#&+A&+UP\.07$
!m=Oh]'"$
!#(*&#&+Unô4'&)b%c*dd*f?p#'!#4'GUU(q$
'1r$ !#(#&+'s^_K":G'OHG-tRU'*'#,&NijJ&kp%$ &'#'(:f?\$
'GDi5û0f^_v@'(Z$ 'GD`)#!#!L$ ,
bcc
w:f;S[3D$
G'GU#x?01RN.',& $
!#(*&#+',R*Y$ .'.'!#M:U(&+U,&4'.0G$ &,I*:J3!#'FG'4'&#'(
&L$
!#,%?U4''!##CSE$ VC:1$
G6.'!L$M"4',&GZE:M6kl#y^b%c*czfR+$
=&9:4'#,S1{$ #'(N4U #!x&+UI'G
|%$ V+"&L$
!pE07*&I4'.0G$ &#'(5%$
":4';2}=:J3!#UY4U.G$ &#'(5#,R4'UG&E0IB&E'4U,&
N+'SV+U#S
\&V$ #'#'(N.0$
V$ &,R&+0$
"E$
7C:1$
&GB&.U!~$M:?'$
UG6$
!#,N0$
E'!#,+'I$
!#( J
+',R&E0S4',&G8#6:&#:4'4U,&!#M*.7$
U(,&M:,&,1+'&I''G8
\&~$ !pUG'#&#',
:4U%;
`D5B.$
N+'.3*&6$ '5
&+',&6$ !#(#&+'B,D$%$
!##,&#NE0I$>:#( $ &#
.'E'!#A?:1+'$,&#4'!L$ &GBEU!#=&E0=#,YV$ UG8&B(4'#G'F,&!O26$( $ !x.0,#
#A&+'.',&'"
2*E',&7$
!#,;P+U&E0S4',X!#MZ*Z!###&G6,',&MA_G'E$ 7C8
#&,S,4'&4''GUU(,%?\$
UG86$ CIG'#,&#',F&+$ Y$
AE(*'7$
!##,&GA&8$
E'#&7$
&M!~$M*4',S

*E',&7$
!#,;
P+',,4U!~$ &#U,,&+'
1&+$ I:J3!##'N!#%$ &'#'("$
!#(*&#&+U,Y$ &!#,,I,&U,&#&#>"&+'
7+'*I
&7$
#'#'(.$ V$ &,=&+$ 8E$
7C:1$
&G&.'!L$M:?$ 'G"&+$ X&+U$ !&0$
&#>*Y4U.G$ &
4'!#,
K8tijJWkD$ 'G8iN^_uf=$
YBX&*E'4',&+$ 6,V$ 'G$
GZijJ!#%$ &'#'(54'.0G$
,%;
T

K
!#"$# %'&()+*
l+)@T ,.-0%/ 24AC3BP>@BP[ 12121312121412121312141212131212131212141212131212141213121213121214121 V
l+)xl g 241Q+ij3Apo de>65"243624BPv 27/%24AC3BP>@BP[8141212131212131212141212131212141213121213121214121 V
l+):9 a+; QPoZij3AWR>ZijB<12121312121412121312141212131212131212141212131212141213121213121214121 V
3=+> @?A B&(C+* D
+9 )@T XHij1,SP>@BP>@BP[E,.-0%/ 24Ab3BP>@BP[ACBPY g 241Q+ij3Apo"de>65 243_24BPv 23%/ 24AC3BP>@BP[F1213121213121214121 G
+9 )xl 9ijY+>6.H 2 Y X ijB+BP2 v R>ZijBP>Z^R2,.-0%/ 24AC3BP>@BP[I131212131212141212131212141213121213121214121 J
+9 ):9 ,K!LMN1412121312121312121412121312141212131212131212141212131212141213121213121214121 O
P Q%SRT+ B&(C* U
D 7V+ +W= YX3W=RZ#" [
+G )@T g\ 2-HijS iCRIatBC]P>@36ijB+1,24BbRN121312141212131212131212141212131212141213121213121214121 U
+G )xl a+; Q+2436>@1,24BbRfAWo de2 RfAW>ZoZÊ1412121312141212131212131212141212131212141213121213121214121 T]
+G ):9 -H2 ^f/PoZR^^12121312121312121412121312141212131212131212141212131212141213121213121214121 TCT
+G )wV kt2 ^RIXHijBbRf36ibo m iCoZ>Zv 5$121412121312141212131212131212141212131212141213121213121214121 TG
M EYT?B??_# `>?aCR ? K+M
+J )@T 0b 24/+3_>q^hR>Zvm AC3Ab1,2 R2436^c1412121312141212131212131212141212131212141213121213121214121 T J
+J )xl ed B(- oZ>cBP2,]kHAWv yf0e AC3_Y8-H24QPo@A 5g12141212131212131212141212131212141213121213121214121 T O
+J ):9 XHij1!Q+Ab36>ZîjBih \Q+Y.ApR2-0/PoZ2 ^ 141212131212131212141212131212141213121213121214121 TO
U 3fRC?ZiC? Kj
3&R@BCR& *
^RT*@Wk@R@Z Al m&>? K[
l
=
S2
9/Pv \ ih R \ 22e iP3yJY+ijBP2e>@BR \ 2e362 >@B(h iP36v 241,24BbR0oZ24AC3BP>@BP[ oZ>ZR243ApRf/+362/P^2 ^0Y+>Z^v43_2 R2e^RfAWRh2 - ^fQ+AWv 2 ^4z
e0>ZR \ oZie Y+>c124BP^>ZijB+AWo>@B+Q+/PR - ^ Q+AWv 2 ^4) g.\ >q^J>Z^S+2 v4Ab/P^2!v4/+33624BbR362 >@B(hsij36v 241,24BbRoZ24AC3BP>@BP[ AWoZ[Ci-
36>ZR \ 1,ê362 =/P>@362J2Aj; Rh24BP^>Z]C2!3624Q+24AWRh2 Y^24AC3_v \ 2 îh R \ 2^R AWR2 - ^fQ+AWv 2e>@Bij36Y+243 Ri Q+36ijQ+Ap[jAWR2e>@B(hsij3!-
1!AWR>ZijB;ACS ij/PR!R \ 2Q+A 5Cia5 ^,A ] AW>Zo@ACSPoZ2CzIACBPY;^hi ^f1!AWoZoZ243^RfAWR2 - ^ Q+AWv 2 ^,v4ACB;S+22A=; AC1>cBP2 Y 1,ij362
24AW^>ZoZ5C);*IoZîNz=h 3_ij1 AR \ 2 ij362 Rh>qv4Apo Q iC>@BCRih ]P>Z2Ae ztR \ 2 iPBPoq5 Q+3_ijish ^,iIh v ijBb]C2436[b24BPv 2A4]WAW>Zo@ACSPoZ2
h ij3?362 >@B(h ij3_v 241,24BCReoZ24AC3hBP>cBP[Apoq[bij36>ZR \ 1,êAC362S+AW^h2 Y iPB>@B(h iP31!AWR>ZijBS+2 >@BP[^RiP362 Y2A+; QPoZ>qv >ZRoZ5 AWR
24AWv \ ^R AWR2C)20b iHe 2 ]C243 zN>ZRe>ZêY+2 ^>@3ACSPoZ2 Rhi 2AP; R24BPY 362 >@B(h iP36v 241,24BbRoZ24AC3BP>@BP[ Apoq[bij36>ZR \ 1,^RiYHe ij3y
2 v >Z24BCRoZ5>@Bo@AC36[C2,^R AWR2 - ^fQ+AWv 2 ^ z+e \ >Zv \ 362 +/P>@362 ^ "!Ri8S 2!/P^2 Y Ri^fQ+3624AWY>@B(sh ij3!-
1!AWR>ZijBS 2 R0He 2 24B^>@1,>Zo@AC30^hRfAWR2 ^4z+ACBPY \ 24BPv 23_2 Y./Pv 2R \ 2eAC1,ij/+BbRHih >@B(sh ij31!AWRh>qiPBR \ AWR01,/P^R0S+2
v iCoZoq2 v Rh2 Y") g\ >Z^0>ZÎ2 ^fQ 2 v >@AWoZoq5>@1!Q ij36RfACBbRH>@BR \ 2Jv4AW^2ei$h #%!& "'!'( )*,+-#%. zBe \ 243_2R \ 24362
AC362H2 "5 2 v R>Z]C2 oZ5 ACBJ>@B(H BP>qRh2Bj/+1,S+243 ih ^hRfAWR2 ^4zWACBPY!R \ 2IQ+36ijS+ACSP>ZoZ>ZR65!ih 2 ]b243 362 Rf/+3BP>@BP[Rhie2A=; AWv Rhoq5
R \ 2^fAb1,2^R AWR2e2 ]C243AW[jAW>@B>Z^BP2 [CoZ>Z[C>@SPoZ2C) / BR \ 2 ^2v4Ap^2 ^4zN[C24BP243AWoZ>Z^fAWR>ZijBih 2A+; Q+243_>q24BPv 2>qêBPiCR
ijBPoZ5Y+2 ^>@3ACSPoZ2Cz S+/PReA,BP2 v 2 ^^>ZR65C)
/ B R \ >Z^!3624Q+iP36R4zH e 2!2A+; AC1,>@BP2R \ 2/P^2i1h 02# 3)+4!+- 5 "!67'8&97 :;!23eRi^Rij3_2 R \ 2
>cB(sh ij31AWR>ZijBoZ24AC3BbRISb5!R \ 23,.- oZ24AC3BP>@BP[AWoZ[Cij36>ZR \ 1 =K < AWRfyb>@BP^T4U>CU( M )k%/ >@B 6K T4UCUf9(M 2A+; AC1,>@BP2 YR \ 2
/P^2ih BP24/+3hAWo BP2 R He ij3yb^!>@B R \ >Z^v ijBbR2Aj; R z0ACBPY Q+36iW]P>ZY+2 Y Apoq[bij36>ZR \ 1,^`h iP3,Rf3AW>@BP>@BP[ S+AW^2 Y ijB
R \ 2 >ZY+24Ai?h 02# 3@:;AB+4CWz=e \ >Zv \ >@BC]biCoZ]C2 ^,^Rij36>@BP[R \ 2!^hRfAWR2 `- AWv Rh>qiPB Q+AW>@36^,]j>Z^>ZR2 Y Sb5 R \ 2
^5j^hR241 /+BbR>Zo R \ 2J24BPY iah A!Rf3_>cApoke \ 24BR \ 2!362A0e AC3_Y+ÎAC362!yPBPi e B%z ACBPYR \ 24B3624QPo@A 5j>@BP[R \ 241 >@B
362 ]C2436^2ij36Y+243JRiRf3AW>@B R \ 2BP2 R He ij3yN)D/ B R \ >Z^ 3_24Q+ij36R z \ iHe 2 ]C243 z=He 2 \ A4]b2 2A+; AC1,>@BP2 Y;R \ 2/P^2
ih Apoq[bij36>ZR \ 1,^!R \ AWR,v4ACB;S+2ACQ+QPoZ>q2 YE!F)8"4z >cB R \ 2^24BP^2R \ AWR!R \ 2/+Q Y.AWR2 ^!Ri R \ 2BP24/+3AWo
BP2 R0He ij3hyCÂC3_2 1!ApY+2IAWR24Apv \ R>@1,2^R24Q,Y./+36>@BP[!R \ 2IRf3_>cApo zB0e >ZR \ iP/PRR \ 2eBP2 2 Y Rhi^RiP362IACBb5,^RfAWR2 -
AWv R>ZijBe>@B(sh ij31!ApR>ZijB%)G< 2^ \ ie R \ AWR >@Bv iP1!Q+AC36>ZîjB0e >ZR \ S+AWv yf0e AC36Y3_24QPocA 5CzWijB(- oZ>@BP2I/+Q Y.AWR2 ÂC362
S+2 RR243?AWR0v ijBb]C2436[C>@BP[!Rhi!ijQPR>@1!AWo"Q iCoZ>qv >Z2 ^Ji ]b243 A 0e >ZY+243I3AbBP[C2eih Rf3AW>@BP>@BP[ Q+AC3Ab1,2 R243H]WAWo@/P2 ^4)
< 2,AWoZî 2A+; AC1,>@BP2,iCR \ 243 h ij3h1,Îiah /+Q Y.AWR2!3/PoZ2,R \ ACBR \ 2^RfAbBPY.AC36Yc,.- oZ24AC3BP>@BP[ Apoq[bij36>ZR \ 1z
>cBbRf36iPY./Pv >@BP[ ABP2Ae Apoq[bij36>ZR \ 1 e \ >qv \ e 2v4AWoZo9ijY+>6.H 2 Y;XHijB+BP2 v R>ZijBP>Z^R,.-0%/ 24Ab3BP>@BP[.zACBPY 2A;(-
R24BPY+>@BP[c,K!LM3sK m 24BP[ ACBPYH< >qoZoZ>@AC1,^!T4UbUWV'M%h iP3I/P^240e >ZR \ v iPBCR>@BP/Pij/P^.h /+BPv R>ZijBACQ+Q+36iP; >c1AWRij36^ )
g\ 2 AWoZ[CiP36>ZR \ 1,^ AC362 241!QP>@36>Zv4AWoZoZ5,R2 ^R2 YJijBA01ijSP>Zoq2036iPS+iCR Q+3_ijSPoZ241ze \ 24362HA0^>@1,/Po@AWR2 Y,1,ijSP>ZoZ2
36ijS+ibR0>Z^0Rf3AW>@BP2 YRi![j/P>ZY+2>ZR^h2 o:h RhiA[CijAWo"Q+ib^>ZR>ZijB>cBA,lWd 24BC]P>@36ijB+1,24BbR4)
g\ 2 3_241!AW>@BPY+243Jih R \ >Z^,3624Q+ij3_R>Z^,ij36[jACBP>Z^2 Y AW^^h iboqoZi0e ^4rJ^2 v R>ZijB l>@BCR 36ijY./Pv 2 ^,R \ 2S+AW^>Zv ^
ih 362 >@B(sh ij36v 241,24BbRoZ24AC3hBP>cBP[ ACBPY ,.- oq24Ab3BP>@BP[.z^h2 v R>ZijB 9 Y+>Z^v4/P^^2 ^v ijB+BP2 v Rh>qiPBP>q^hR ,.- oZ24AC3hBP>cBP[
ACBPY>cBbRf36iPY./Pv 2 ^R \ 2 AWoZR243B+AWRh>q]b2/+Q Y.AWR2 3h/Poq2 ^ z ACBPY^2 v R>ZijBV2A=; QPo@AW>@BP^R \ 2!ijB(- oZ>@BP28/+Q Y.AWR>@BP[
AWoZ[Cij36>ZR \ 1)tOP2 v R>ZijBP^ ! G ACBPYc,J Q+362 ^h24BCR0R \ 2362 ^f/PoZRÎACBPYY+>Z^v4/P^h^>ZijBPêih R \ 2e2A+; Q+243_>c124BCRîjBR \ 2
36ijS+ibRIQ+36ijSPoZ241z.AbBPYYH B+AWoZoZ5^2 v R>ZijB ,O [C>Z]C2 ^0R \ 2ev iPBPv oc/P^h>qiPBP^4)
I J K
M L$ NP O mE QYK RPk
S
/ B!362 >@B(sh ij36v 241,24BbRoZ24AC3hBP>cBP[!Q+36iPSPoq241^4z+A^v4AWo@AC3 ] Apoc/P2v4AWoZoZ2 YAT+UC@!V >Z^ 362 v 2 >Z]C2 YSC5JR \ 20v ijBbRf36ibo
^5j^hR241 h iP3 Rf3ACBP^>ZR>ZijBP^=h 36ij1 iPBP2 ^hRfAWR2HRieACBPiCR \ 243 ) g.\ 2IAW>@1 ih R \ 2 ^h5j^Rh241 >q^tRi^H BPY Av ijBbRf36ibo
Q+iCoZ>Zv 5Ye \ >Zv \ 1Aj; >@1,>Z^2 ^?R \ 2I2A+; Q+2 v Rh2 YYh /PRf/+362Y+>q^hv ij/+BCRh2 Y^f/+1'ih Q+A 5Ci"5 ^ 3_2 v 2 >q]b2 Y"z.yjBPiIe BAW^
R \ 2 '%N)
g\ 2W'5YX '57# "!!>ZêA!Q+362 Y+>Zv R>ZijBih R \ 2,362 Rf/+3BA ]WAW>ZocAbSPoq24h 36ij1'24AWv \ ^RfAWRh2Cz
Z "K []\ Y
M ^`_ba6d c e i \kj e5l 6K TM
e%f7gh
e \ 24362 i \ >Z^0R \ 2Q+A 5Ci5 362 v 2 >Z]C2 YYsh ij3tR \ 2eRf3hACBP^>ZR>ZijB`h 36ij1'^RfAWRh2I]C2 v Rhij3Y[N\ Rhi?[]\kj]m ACBPY >ZÎR \ 2
A #!'577Xn# ! Kp ] o o T M )<0ibR2R \ ApR Z "K [N\)0M R \ 243_2 h ij3_23624Q+362 ^h24BCR^R \ 2JY+>q^hv ij/+BCRh2 Y h ^f/+1 ih
h
9
Q+A45bi 5"^ 362 v 2 >Z]C2 Y h 36iP1 R>@1,2e^Rh24Q(!:;A@ z+ACBPYR \ AWRtR \ >Z^0^f/+1 e?>qoZo"Y+24Q 24BPYijBR \ 2^2 +/P24BPv 2
ihApv R>ZijBP^!RfACyp24B K R \ 2Q+iboq>Zv 5M ) g\ 2v ijBCR 36iCoH^5P^R241 >Z^8362 +/P>@362 Y Ri H BPY R \ 2Q+iCoZ>Zv 5 e \ >qv \
1!A ;P>c1>q^h2 ^ Z K"[N\)MH>@B24Apv \ ^R AWR2C)

/ B )%"K"< ApRfyC>@BP^T U>CUfM R \ 2e>ZY+24A!>Z^Ri,oq24Ab3B ) X '57# "!.z e \ >Zv \ >ZêA!Q+362 Y+>Zv R>ZijBiah R \ 2
362 Rf/+3BAW^^hijv >@AWR2 Y e0>ZR \ 24AWv \ AWv Rh>qiPB >@B24AWv \ ^RfAWRh2C) g\ >Z^Q+362 Y+>Zv R>ZijB v4ACBS+2 /+Q Y.AWR2 Y
e0>ZR \ 362 ^ Q+2 v R0RiR \ 2Q+362 Y+>Zv R2 Y 362 Rf/+3Biah R \ 2BP2A;PR0^R AWR2e]P>q^h>qRh2 Y"z
EK=[N\"!# \)MY^ i \$ Z K"[N\kj]mSM Ksl(M
h
*I^HR \ 2eiW]C243Apoqo"AW>@1'i h R \ 2e^5j^hR241'>q^?Ri!1!A ;P>c1>q^h2eR \ 2 Q+A 5Cai 5^H362 v 2 >Z]C2 Y"zNR \ 2Iv4/+3h3624BCR?2 ^R>@1!AWR2
ih Z K"[ \ Mti hA,^RfApR2I>Z^[C>Z]C24BSb51!A ;&%('*)+EK"[ \ !#' M zPî, 2 +/+AWR>ZijBl!S+2 v ij12 ^4z
YK"[N\,!-& \ MP^ i \.$ 1%/'0A ) ; YK"[N\kj]m1!-B M K)9(M
h
g\ >Z^>Z^JR \ 2D!7 ), +N,.- oZ24AC3BP>@BP[ /+Q+Y.ApR2!2 +/+AWR>ZijB%z=e \ >Zv \ \ AW^,S+2 24B ^ \ iIe BRiv ijBb]C2436[b23hsij3
H BP>qRh2 - ^RfAWR29 AC3hyWiW]j>@ACB Q+3_ijSPoZ241,ê \ 24B;AoZijiPyj/+Q RfAbSPoq2>Z^ /P^h2 Y;Ri^Rij362R \ 2] AWo@/P2 îhIR \ 2
,.-)h /+BPv R>ZijB K"< AWRfyb>@BP^T4U>bUfM )deBPv 2,R \ 2 ,.-)h /+BPv R>ZijB \ AW^v ijBb]C2436[C2 Y"zNR \ 2iPQPR>@1!AWo Q+iCoZ>Zv 5>q^Ri
RfACyW2JR \ 2AWv Rh>qiPB>@B24AWv \ ^R AWR23e?>qR \ R \ 2 \ >Z[ \ 2 ^RQ+362 Y+>Zv R2 Y 362 Rf/+3B%) g\ >Zê>qêv4ApoqoZ2 Y R \ 2 5%A&C
Q+iCoZ>Zv 5C)
2 346587:9;<>=@?ACB4<ED.F
46><EG
a;+/+AWR>ZijBI9 >Z^ ACB /+Q+Y.ApR2 S+AW^2 Y ijB R \ 26 IH$+-! A V # K O+/PRRijB T4U>>fMS+2 R eH2 24B R \ 2
v4/+33624BbR ACBPY BP2A;PR Q+362 Y+>Zv R>ZijB ih 362 R /+3B%) b?i eH2 ]C243 z R \ 2BP2A;PR Q+362 Y+>Zv R>ZijB_e?>qoZov \ AbBP[C2Ye \ 24B
\kj]m>ZÎ/+Q Y.AWR2 Y"zBe \ >Zv \ >@1!QPoZ>Z2 ÎR \ ApRE \ ^ \ ij/PoZYS+2eAWYWD/P^R2 YRiRfAbyW20R \ >Z^H>@BCRi,AWv v ij/+BbR4) *I^0A
[C24BP243AWo.3/PoZ2CzbR \ 24362 hsij362CzW24AWv \ BPA2 e R241!Q ij3AWoPY+6> 5"243624BPv 2 e?>qoZo+2 5"2 v RBPiCR%D/P^R R \ 2Ho@AW^RQ+362 Y+>Zv R>ZijB%z
S+/Pp R " Q+3_2 ]j>Zij/P^,Q+362 Y+>Zv R>ZijBP^4) .g \ >q^,362 ^f/PoZR^J>cBR \ 2 g 241!Q+ij3hAWo de6> 5243_24BPv 2!oZ24AC3BP>@BP[ AWoZ[Cij36>ZR \ 1
e \ >Zv \ v4ACBS+2,ACQ+QPoZ>q2 YRi^fQ+2 2 Y/+Qv ijBb]C2436[C24BPv 2eai h ^ 2 +/P24BbR>@AWo Q+362 Y+>Zv R>ZijBRfAW^ yC^4)
/
B h AWv R4zWO+/PRRijB>@BCRf3_ijY./Pv 2 ÂCBJ24BbR>@362 h AC1,>ZoZ5,i h.AWoZ[Cij36>ZR \ 1,^ v4ApoqoZ2 Y g Qd K!LM Az e \ 24362 ] oIL o T
>q^!A e 2 >Z[ \ R>@BP[iPBR \ 2362 oZ2 ] ACBPv 2i h0362 v 24BbRRh241!Q+ij3hAWo Y+6> 5243_24BPv 2 ^Rhi24AC36oZ>Z243,Q+362 Y+>Zv R>ZijBP^4c ) b?2
^f/P[C[C2 ^Rh^0R \ AWR0/P^>@BP[oqi eH243H]WAWo@/P2 Îi hmL83_2 Y./Pv 2 ÎR \ 2e]WAC36>@ACBPv 2Ji h R \ 2Q+362 Y+>Zv R>ZijBPê^2 24BSb524AWv \
^RfAWR2CG) /sR \ AW^S+2 24B,^ \ i eB!SC5JdA 5jACQ B K_T4UCUCfl M R \ AWR g Q d K!LM%v ijBb]C2436[C2 =^ hsiji3 H BP>ZR2 - ^RfApR209 AC3yWiW]P>@ACB
Q+36ijSPoZ241,^4z%S+/PRIBPi Q+36iPi h A2 ;j>Z^Rhm^ h iP3Hv ijBbR>@Bj/Pij/P^?^5j^hR241,^4)
KJ +L ME7E=9N6OPQ9;
g\ 23624ApîjB`hsij3HRfACyb>@BP[ 2A;+QPoZij3AWRhij365AWv R>ZijBP^>q^R \ AWR?>qR>Z^BPiCRIQ iC^^>@SPoZ24h ij3?R \ 23,.-)h /+BPv R>ZijBPêRi
oq24Ab3B 6> hIR \ 2 AWv R>ZijB e0>ZR \ R \ 2 \ >Z[ \ 2 ^RREK=[N\"!#\)Me>Z^,v \ iC^24B AWRAWoZoHR>@1,2 ^4z AW^4z Y./+36>@BP[ R 3AW>@BP>@BP[.z
iCR \ 243!AWv Rh>qiPBP ^ e0>ZR \ oZi e 243?+4%A"# AQ+A 5Ci5"^!1!A 5 >@B h AWv R!S+2 S 2 RR243 ) g\ 21,2 R \ iPY /P^2 Y;Ri
^2 oZ2 v R Apv R>ZijBP^ \ AW^ AY+>c3_2 v R 2 5"2 v RtijB!R \ 2I3AWR2AWR e \ >Zv \ R \ 2I362 >@B(h iP36v 241,24BbR oZ24AC3hBP>cBP[AWoZ[Cij36>ZR \ 1
e0>Zoqotv ijBb]C2436[C2!RhiACBijQPR>@1!ApoQ+iCoZ>Zv 5C) /sY+24AWoZoZ5Cz R \ 2!^h5j^Rh241 ^ \ ij/PoZY iPBPoq5 v \ iPiC^2,RiQ 243!h iP31 A
BPij(B - [j362 2 Y+5 Apv R>ZijB,6> h">qRto@AWv yb^Hv ij(B H.Y+24BPv 2>cBR \ 20v4/+3h3624BCRt[j362 2 Y+5!Q+362 Y+>Zv R>ZijB%)t*IoZR \ iP/P[ \ ] Ab36>Zij/P^
1,2 R \ iPY+^ \ A4]b2S+2 24B^f/P[C[C2 ^Rh2 c Y hsij3/P^2,>@BY+>Z^v43_2 R2!^RfApR2 - ^fQ+AWv 2J^5j^hR241,Q ^ K g \ 3h/+B T4UCUbfl M zNR \ 2 5
AC362BPiCR0[b24BP243AWoZoZ5ACQ+QPoZ>qv4AbSPoq2Ri,^5j^hR241,Î/P^>@BP[v ijBbR>@Bj/Pij/P ^ h /+BPv R>ZijBACQ+Q+36i ;P>c1AWRij36^ )
V
I273K=
S2
S
#QYKRPk
S
/%24AC3BP>@BP[R \ 27,.-)h /+BPv R>ZijB362 +/P>@362 êîj121,2 R \ iPYiah ^Rhij36>@BP[ R \ 2v4/+33624BbReQ+362 Y+>Zv R>ZijBPÂWR24AWv \
^RfAWR2ish ij3%24AWv \ AWv R>ZijB%zWACBPY/+Q Y.AWR>@BP[R \ 241 AW^ BP2Ae >cB(sh ij31AWR>ZijB>Z^ [PAWR \ 24362 Y") OPRij36>@BP[IAH^h24Q+AC3AWR2
] AWo@/P2.h ij3 24AWv \ ^RfAWRh2 `- AWv R>ZijB!Q+AW>@3 =/P>Zv yCoZ5S+2 v ij1,2 ^H>@1!Q+3Apv R>Zv4AWohsij3Q+3_ijSPoZ241,^me0>ZR \ o@AC36[C2?^RfAWR2 -
^fQ+AWv 2 ^4z0ACBPY h iP3,v ijBbR>@Bj/Pij/P^^RfApR2 - ^fQ+AWv 2 ^!>Z^^>@1!QPoZ5 BPiCRQ+iC^^h>cSPoZ2C) g\ 24362 sh ij362CzHR \ 24362>Z^R \ 2
BP2 2 Y Ri/P^h2!îj1,2 h ij3h1 ih h /+BPv R>ZijB ACQ+Q+36ij; >@1!ApR>ZijB e \ >Zv \ [C24BP243AWoZ>Z^2 ^!Q+362 Y+>Zv R>ZijBP^!S+2 R eH2 24B
^RfAWR2 ^ z=AbBPYR \ /P^Q+36iW]j>ZY+2 êQ+362 Y+>Zv R>ZijBPê2 ]C24B>@B^>ZRf/+AWR>ZijBP^0R \ AWR \ A ]C2BP2 ]C2430S 2 24B2A+; Q+2436>Z24BPv 2 Y
S+2 sh ij362C)I<?24/+3AWo BP2 R0He ij3hyC^4z ij3 ' )NC@
#+4 !F z+Q+36iW]P>qY+2,^f/Pv \ Av ijBbR>@Bj/PiP/P^h /+BPv R>ZijB
ACQ+Q+36ij; >@1!ApR>ZijB!Rh2 v \ BP>+/P2Cz.ACBPY \ A4]b2IR \ 2eAWY+]WACBCR AW[C20ih ^v4AWoZ>@BP[He 2 oqo%RiJo@AC36[C2>@B+Q+/PRIACBPY^RfAWR2 -
^fQ+AWv 2 ^ sK /+BPoZ>@yW2Cz#sh ij3t>@BP^RfACBPv 2Cz%XH9*IXH^3K *o@S+/P^T4U&>+TtM ij3H-0AWY+>@AWo kHAW^>Z^%/+BPv R>ZijBP^ M )
/ BJij3_Y+243"Ri3624Q+362 ^24BbR R \ 2 ,.-)h /+BPv R>ZijB!/P^h>cBP[BP24/+3hAWo=BP2 R He ij3yb^4z 2 >ZR \ 243 A?^>@BP[CoZ2IBP2 R He ij3y.0e >ZR \
ij/PRfQ+/PR^J>Z^ 362 +/P>c3_2 Y"z ij3 ^24Q+AC3AWRh2 BP2 R e iP3yC^ z"24AWv \ ?e >qR \ A^>@BP[CoZ2ij/PRfQ+/PR ).iCoZoZi0e >@BP[
%/ >@B$6K T UCUClf M zHe 2Hv \ iC^2Ri3624Q+362 ^24BbRR \ 2 ,.-)h /+BPv Rh>qiPB/P^>@BP[ijBP2IBP24/+3hAWo BP2 R He ij3y h ij3 24AWv \ AWv R>ZijB%z
e \ >Zv \ A ]CiC>ZY+êR \ 2 \ >ZY+Y+24B;BPiPY+2 ^362 v 2 >Z]P>cBP[v ijB.>Zv R>@BP[ 243h36ij3^>Z[jB+AWoZ^h 36iP1 Y+>:"5 243624BbRiP/PRfQ+/PR^4)
0b iHe 2 ]C243 z R \ 2ijB(- oZ>@BP20AWoZ[Cij36>ZR \ 1,^ Q+362 ^24BbR2 Y>@Be^h2 v R>ZijBJV0v4ACBeS+2HACQ+QPoZ>Z2 Y!Ri02 >ZR \ 243 AC36v \ >ZR2 v m Rf/+362C)
%/ >@B 6K T UCUClf0M /P^h2 Y R \ 2iPBP2 - ^R24Q ,.- oZ24AC3BP>@BP[2 +/+AWR>ZijB 9 Riv4AWoZv4/Po@AWR2!R \ 2,2433_ij30>@B \ ACBPY
/+Q+Y.AWR2JR \ 2BP24/+3AWo BP2 R0He ij3hy7He 2 >q[ \ R^\AWv v ij36Y+>@BP[!Rhi.z
\ i \.$ 1!%/'0A) ; \kj]mA\N \ K VBM
h
e \ 24362 >q^JR \ 2 %"H#%!Fn & ACBPY!;\ >Z^,A]C2 v Rij3iHh R \ 2,ij/PRfQ+/PRe[j3hAWY+>Z24BCR^"6 \$#%"'& \
v4AWoZv4/PocApR2 Y SC5S+AWv y`- Q+36iPQ+AW[jAWR>ZijB%)
J ( 9;5*)
,+ 346587:9;=:?F2B46D.A
46EQ
/ B ij36Y+243eRhi^fQ 2 2 Y /+Q Rf3AW>@BP>@BP[.zG< AWR yC>@BP^cK6T4U>bUfM0^ /P[C[C2 ^R^Jv ij1,SP>@BP>cBP[N,.- oZ24AC3BP>@BP[Ne?>qR \ g d^-
oq24Ab3BP>@BP[ K ^2 v R>ZijBl+)xlfM )Y/ BR \ >q^.hsij31,/Po@AWR>ZijB%z+R \ 2Jv4/+33624BbRI/+Q+Y.ApR2,24336ij3t>Z^/P^2 YRhi AWYWD/P^RIBPiCR
ijBPoZ5 R \ 2!v4/+33624BbRe2 ^R>@1!AWRh2!ih \`z S+/PRAWoZîR \ AWReih Q+3_2 ]j>Zij/P^J^RfAWR2 ^ z"SC5 yW2 24QP>@BP[ A eH2 >Z[ \ R2 Y
^f/+1 i h 24AC36oZ>Z2430243h36ij3H[j3hAWY+>Z24BCR^ z
A\ d !K L M \$- e ;
\
\ i \$ 1!%/'0A) ; \kj]m e )K G(M
h ef7g h
e \ 24362 L eH2 >Z[ \ R^,R \ 2362 oZ2 ] ACBPv 2ihIR \ 2 v4/+33_24BCR,24336iP3iPB 24AC36oZ>Z243,.- ] AWo@/P2Q+362 Y+>Zv R>ZijBP^4) g\ 2
ijBP2 - ^R24Q ,.- oq24Ab3BP>@BP[ 2 +/+AWR>ZijB>q^R \ 24362 hsij362A^fQ+2 v >@AWo >@BP^RfACBPv 2Jih R \ >Z^2 =/+AWRh>qiPBYe \ 243627L. +] )
/%>@B K6T4UbUf9fMe3624Q ij36R^,R \ AWR,/P^>@BP[ R \ >Z^4h iP31 ihI/+Q+Y.AWR2 hsij3v ijB+BP2 v R>ZijBP>Z^RY,.- oq24Ab3BP>@BP[Ne0>ZR \
L0/ ]362 ^f/PoZR^>cB h AW^R243?v ijBb]C2436[C24BPv 2JR \ ACBR \ 2,ijBP2 - ^R24Q ,.- oq24Ab3BP>@BP[ AWoZ[Cij3_>qR \ 1) b?ie 2 ]b243 z \ 2
ACQ+QPoZ>q2 ê2 +/+AWR>ZijBYG/P^>@BP[ R \ 2h iboqoZie0>@BP[ AWoZ[CiP36>ZR \ 1z21
\34K 65\ F \ 7M 8>\ K)J(M
e \ 24362Cz
5\ i \.$ _K T L.M 1!%/'0A) ; \kj]m$ .L 5\kj]m K)O(M
36ij1 2 +/+AWR>ZijB70O >ZR v4ACB,S 2 ^h2 24BR \ AWR h 24Apv \ 5\ Y+24Q+24BPY+^H362 v4/+36^>Z]C2 oZ5,ijBh /PR /+362E 5\ ]WAWo@/P2 ^4zfe \ >qv \
1,24ACBP^R \ AWRe/+Q+Y.AWR>@BP[v4ACBijBPoZ5iPv v4/+3IApRIR \ 2J24BPYih 24AWv \ Rf36>@AWos)\IBbR>Zo R \ 24B%z%AWoZo ^R AWR2 `- AWv R>ZijB
;9 :=<?>A@;BDC@FE$>HG;I2JA@$KMLNEPOQCFCSRUTVJUEPK2>7<QWSXZYF[\K2W'TVW$CSB
JUWJ,<V>UE$J]EPK2>A<VJAI^WPOQ>7@;EPOQJ7<QBL>7@_Y,àb[cedD[Pfg
hSi'OjK2WK2WkJ7I2W$>lEPOQCJAI2m]>A@$K2EPOVnoTQW$CpBqDNi'OD@$Tj<KM<
OjK2Wsr'>A@$t>7<
GS>7<j<VCSGSEPK2>7<jK2W$EukvI2CSJA@$<DKM<QmCUg mjgì'Oj@$TD<wpxUxUyzfg
G
Q+AW>@36^01/P^hR S+2^RiP362 Y"zjACBPYR \ 24BQ+362 ^24BbR2 Y>cBAJR241!Q ij3AWoZoZ5 02# 3@:;AIiP36Y+243 Ri,Q+36ijQ+AW[PAWR20R \ 2
Q+362 Y+>Zv R>ZijB24336ij36^Hv ij3h362 v RoZ5C) g.\ >q^?>q^v4AWoZoZ2 Y 0 #.3 :;A +4CW)
/ B^2 v R>ZijBJ^
V e 2Q+3_2 ^24BCRApoq[bij36>ZR \ 1,^ Ri0>@1!QPoZ241,24BbR 2 =/+ApR>ZijB7G0Y+>@362 v RoZ5h ij3 !-) " /+Q Y.AWR2 ^4z
ACBPY î 36241,iW]C2R \ 2 3_ 2 =/P>@36241,24BbRRi e0AW>ZR8/+BbR>ZoIR \ 224BPY ih24AWv \ Rf36>@AWos) g\ >Z^36241i ]b2 ^ R \ 2
BP2 v 2 ^^>ZR65 Ri^Rhij362^RfAWR2 -Àpv R>ZijB;Q+AW>@36^4z iP32 ]b24B Ri \ A4]b2ACB 24BPY;iah Rf3_>cApo v iPBPY+>qRh>qiPB%z AW^24AWv \
R241!Q+iP3AWo"Y+6> 5"243624BPv 2,24336ij3 >qê/P^2 Y>c11,2 Y+>@AWR2 oZ5C)0*ÎR \ 2A2 ;+Q+2436>@1,24BbRê^ \ iezPR \ >ZÂWoZî 3_2 ^f/PoZR^
>cBoZ2 ^Î^h24BP^>ZR>Z]j>ZR65Ri!R \ 2eoZ24AC3hBP>cBP[Q+Ab3AC1,2 R243_^ hsij3tR \ 2J^5j^hR241 Ri,v ijBb]C2436[C2C)
*eB>@1!Q+ij3_RfACBbRQ+ib>cBbRAbS+ij/PRJ 2 =/+AWRh>qiPB G>Z^,R \ AWRe>ZR,>Z^BPiCR,ARf3/P2 g ^d - oZ24AC3BP>@BP[ AWoZ[Cij36>ZR \ 1
/+BPoq2 ^h^ R \ 2 5%A&C Q+iboq>Zv 5;>Z4^ h iboqoZi eH2 Y"z>s)w2C) R \ 2R241!Q+iP3AWoY+6> 5"243624BPv 224336ij367^ e0>ZoZoIBPiCR!AWY+Y;/+Q
v ij3362 v RoZ5Cz
dc e i \"j $ 1%/'0A ) ; \kj e j]m\kj e d c e i \"j e \ K >(M
e
ef7g h h ef7g h
/+BPoq2 ^h^ R \ 2 AWv Rh>qiPB v ij3362 ^ Q+ijBPY+>@BP[ Ri 1A ;&%('*)+EK"[ \ !- \ M>Z^ Q 24!3 h iP31,2 Y AWR2 ]C24365 R>@1,2 ^R24Q%)
< AWRfyb>@BP^3_2 v iC[jBP>Z^2 Y R \ >q^ zACBPY;^f/P[C[b2 ^R2 Y ^2 RR>@BP[ L ] e \ 24BP2 ]C243BPij(B - [j362 2 Y+5;AWv R>ZijBP^!AC362
Q+24!3 hsij31,2 Y KsAW^J>Z^BP2 v 2 ^h^fAC365 hsij3A2 ;=QPoZij3ApR>ZijB ^2 2^2 v R>ZijB;l+6) 9fM ) .g \ >q^>q^,îP1,2 R>@1,2 ^>Z[jBPiP362 Y"z
ACBPY Â H;P2 Y L /P^2 Y"z+S+/PRA2 ;+Q+2436>@1,24BbR^ \ A ]C20^ \ i eIB,R \ AWR R \ >Z^Hoq24ApY+^ RhiA1,ij3_2I/+B+362 oZ>@ACSPoZ2eRf3AW>@(B -
>cBP[ 1,2 R \ iPY;R \ ACB R \ iC^2^Rf/PY+>Z2 Y \ 243_2CB ) /sR!>Z
^ eHij36R \ BPiCR>@BP[.z \ i eH2 ]C243 z R \ AWR />cB ^ S+AWv fy e0AC36Y
3624QPo@A45 2 =/+ApR>ZijBPQ^ K)J+z OfM.e0>ZR \ LYH;P2 Y AC362J>@B h Apv R ACB>@1!QPoZ241,24BbRfAWR>ZijBi h ,K!LM K ^2 v R>ZijB 9+:) 9fM z
ACBPYBPiCR0i h 2 +/+AWR>Zijc B G4e0>ZR \ H;j2 Y L")
J2 9F+E + ( ;9 E6D OPQ9;E IO /
EQ
g\ 2p+/P2 ^R>ZijB >Z^7e \ 2 R \ 243,1!A; %('0) EK"[E!#'Me3_24AWoZoq5 Q+36iW]j>ZY+2 ^,R \ 2S+2 ^R,2 ^R>@1!ApR2!ih0R \ 2362 Rf/+3B
ihR \ 2^hRfAWR2 [) / BR \ 2J24AC36oZ5^RfAp[C2 ^0ih oZ24AC3BP>@BP[.z%R \ 27,.-)h /+BPv R>ZijB]WAWo@/P2 êih AWv R>ZijBP^R \ AWR \ A ]C2
BPiCR S+2 24B 2A;+QPoZij362 Y >ZôZ>cyp2 oq5 Ri S 2 v iP1!QPoZ2 R2 oZ5 eI36iPBP[.zACBPY 2 ]C24B;>@B;R \ 2o@AWRR243^RfAW[C2 ^ zR \ 2
1!A;P>c1,/+1 ] Apoc/P2>Z^!1,ij3_2 oZ>@yW2 oZ5;Ri S 28AbB iW]C24!3 - 2 ^hR>@1!AWR>ZijB ih0R \ 2Rf3/P2362 Rf/+3B A ] AW>Zo@ACSPoZ2 KsAW^
AC36[j/P2 Y>@B g\ 3/+BACBPYOPv \ e0AC36R7
K6T4UCfU 9(MSM ') /+3_R \ 243 zPR \ 2e/+Q+Y.AWR23h/Poq2.hsij3m,.- oq24Ab3BP>@BP[ v ij1,SP>@BP2 Y
e0>ZR \ Rh241!Q+ij3hAWo.Y+6> 5"243624BPv 21,2 R \ iPY+^036 2 +/P>c3_2 2^ L!RiS 2
2436iP2 Y ijB2 ]C24365^R24QR \ AWR AJBPij(B - [j362 2 Y+5
AWv R>ZijB>q^?RfACyW24B%) *^ h 36ij1'R \ 2eACS i ]b2 AC3_[j/+1,24BbR^R \ 2[j362 2 Y+5AWv R>ZijBv ij/PoZY>@` B h Apv R S+2e>@BPv ij3h362 v R
K 2 ^fQ+2 v >@AWoZoZ5 >@B R \ 224AC36oZ5 ^RfAW[C2 ^,i heoZ24AC3BP>@BP'[ M z
2436iC>@BP[ R \ 22 5"2 v R,i h ^ /+SP^ 2 =/P24BbR Q+362 Y+>Zv R>ZijBP^
ijB;R \ ib^2Q+36>Zij3Ri A BPij(B - [P362 2 Y+5 AWv R>ZijB >qôZ>@yW2 oZ5 Ri S 2 1,iP362i hA \ >@BPY.3ACBPv 2R \ AbB;A \ 2 o@Q
>cB v ijBb]C2436[b>cBP[ ijB;R \ 236 2 +/P>c3_2 Y Q+3_2 Y+>qv Rh>qiPBP^4 ) %/+36R \ 2431,iP362Cz Ap^!R \ 2^h5j^Rh241 v ijBb]C2436[b2 ^!Ri A
îCo@/PR>ZijB%z"[P362 2 Y+5 AWv Rh>qiPBP2^ e0>ZoqoHS+2 /P^h2 Y1ij362,RiA2 ;=QPoZiC>ZRR \ 2 Q+iCoZ>Zv 5oZ24AC3BbRSC5R \ 2,^5j^hR241z%î
R \ 2 [P362 2 Y+5 362 Rf/+3hBP^ e?>qoZo0S+2^h2 24B ACBbB5 e A 5C) .g \ 24362 h iP362C=z e 2>cBbRf36iPY./Pv 2ACB AWoZR243B+AWRh>q]b28/+Q Y.AWR2
AWoZ[Cij36>ZR \ 1'S+AW^2 Y1,ij3620^hRf36ijBP[CoZ5,ijB g ^d - oq24Ab3BP>@BP[.zPv4AWoZoq2 Y9iPY+6> H.2 YXHijB+BP2 v R>ZijBP>Z^R ,.-0/%24AC3hBP>cBP[
K 9 X ,.-0/M )
de/+30Q+36iPQ+iC^2 Y/+Q+Y.AWRh23/PoZ2>Z^4z
FA\ d K LM \$- e
\
\ i \$ \kj]m e KsU(M
h e%f7g h
g \ >Z^ Y+6> 5"2436`^ h 36iP1 BPij31!Apo ,.- oZ24AC3BP>@BP[;>@B;R \ 2/P^2i heR \ 2 \kj]m AW^^hijv >@AWR2 Y e0>ZR \ R \ 2AWv R>ZijB
^2 oZ2 v R2 Y"z 3ApR \ 243R \ ACBR \ 2![j362 2 Y+51!A; %('0) \"jNme/P^2 Y >@BN,.- oZ24AC3BP>@BP[.) g\ >Z^24BP^ /+362 êR \ AWReR \ 2
R241!Q+iP3AWo"Y+>65"243624BPv 2!2433_ij36ê0>ZoZoAWY+Y/+Qv iP3362 v RoZ5Cz%362 [jAC36Y+oZ2 ^îh e \ 2 R \ 2430[j3_2 2 Y+5 AWv R>ZijBPêAC362
RfACyW24Bij3BPiCR4z e0>ZR \ ij/PRR \ 2BP2 2 YR i
243_c
i L") /@h[j362 2 Y+5Apv R>ZijBP?^ &8RfAbyW24B%z \ ieH2 ]C243 z+R \ 24BR \ >Z^
i'OQ>7TQm7O3 K2G$OUTVE$E$>A<
WPTVmUm]CSW$E$W X]JUW^NU>AT<QCpCSBE$>,t <V>7r $E J]E$Cv"! GSEPK2>7<jv"CSr'J7@;B v# E$JUE$Cv"!sGSEPK2>A<6q%CL>7@;C
$ C@$L>A@;nKM<QmlJA<
T $ BDJUE$CUg
J
2 =/+AWRh>qiPBe>Z^ 2A;=Apv RoZ5e2 =/P>Z] Apoq24BbRRhi0^RfACBPY.Ab36Y3,.- oZ24AC3BP>@BP[.z+ACBPY^hi.z4>@BR \ 2oZ>@1,>ZR e \ 24B2A;=QPoZij3hAWR>ZijB
\ AW^v 24AW^h2 Y ACBPY R \ 2![j362 2 Y+5 Q iCoZ>qv 5 >Z^!S+2 >@BP[ h iCoZoZieH2 Y"z R \ 2 /+Q+Y.AWRh2 ^ e?>qoZo0S+2R \ 2!^ AC1,2 AW^^hsij3
^RfACBPY.AC3_c Y ,.- oq24Ab3BP>@BP[ K 2 =/+ApR>ZijBYGfM )
9 X ,.-0/ R \ 24362 hsij362^fAC1!QPoZ2 ^4h 36ij1#R \ 2 Y+>Z^Rf3_>cS+/PRh>qiPB;ih Q iC^^>@SPoZ2 h /PRf/+362362 Rf/+3BP^,[C>Z]C24B R \ 2
v4/+33624BbRA2 ;+QPoqiP3AWR>ZijB Q+iCoZ>Zv 5CzH3AWR \ 243R \ ACBD/P^R,R \ 2![j3_2 2 Y+5 Q+iCoZ>Zv 5;AW2^ hsij3BPiP31!AWo ,.- oZ24AC3BP>@BP[.)
g\ 24362 hsij362Cz+R \ 32 ,.-)h /+BPv R>ZijB e0>ZoZo"v ijBb]C2436[C2eRhi.z
YK"[ \ !- \ MY^`_ a i \ $ %/d '0) K2 [ \kj]m M,EK=[ \kj]m !#' M l K6T ](M
h
e \ >Zv \ >Z^,R \ 2A2 ;=Q 2 v R2 Y 362 Rf/+3B [C>Z]C24B R \ 2Q+36ijS+ACSP>ZoZ>ZR>Z2 ^4z K2 [N\)M z i h AWv R>ZijBP^!S 2 >cBP[ ^2 oZ2 v R2 Y")
XHijBP^ 2 +/P24BCRhoq5Cz AWRACBb5 Q iC>@BCRJY./+36>@BP[Rf3Ap>cBP>@BP[.z R \
2 ,.-)h /+BPv Rh>qiPB ^ \ iP/PoqY [C>Z]C2!ACB2 ^R>@1!ApR>ZijBi h
R \ 2eA2 ;=Q 2 v R2 Y 3_2 Rf/+3BP^0R \ ApR Ab362A ] AW>Zo@ACSPoZ22 hsij3tR \ 2ev4/+33624BbR0A2 ;+QPoZij3AWR>ZijBQ+iCoZ>Zv 5C)t*I^0>ZR0>ZêBPij31!Apo
Ri 362 Y./Pv 2R \ 2AC1,ij/+BbR,i h0A2 ;=QPoZij3ApR>ZijB;AW^,Rf3AW>@BP>@BP[ Q+36iPv 2 2 Y+^4zt2 ]C24BbRf/+AWoZoZ5 R \ 2[j362 2 Y+5 AWv R>ZijB
e0>Zoqo S+2JRfACyW24BApR 24Apv \ ^Rh24Q%z.ACBPY^hiR \ 2 ,.-)h /+BPv R>ZijYB e0>ZoZo v ijBC]b2436[C2eRi,R \ 2JijQPR>@1!AWo%] Apoc/P2 ^ )
JKJ
m 24BP[ ACBPY < >qoZoZ>@AC1,^$K_T4UCUWV'MeQ+3_2 ^24BCRh2 Y ACBPibR \ 2431,2 R \ ijY iahIv ij1,SP>@BP>cBP[N,.- oZ24AC3BP>@BP[ AbBPY g d^-
oq24Ab3BP>@BP[.z"v4AWoZoZ2 Y ,K!LM ) g\ >Zê>Z^S+AW^2 YijBQ+243!hsij31>cBP[ABPij31AWo iPBP2 - ^R24Q ,.- oZ24AC3BP>@BP[ /+Q Y.AWR2
Ri>c1Q+36iW]C2 R \ 2 v4/+33_24BCR!Q+362 Y+>Zv R>ZijB \z ACBPY;R \ 24B /P^>@BP[ R \ 2R241Q+ij3ApoY+>65"243624BPv 2 ^ S+2 R eH2 24B
^f/Pv v 2 ^^>Z]C2[j362 2 Y+5;Q+362 Y+>Zv R>ZijBP^Ri /+Q+Y.AWR2>ZR7h 3_ij1 R \ 24362iPB%z0362 [jAC36Y+oZ2 ^îh e \ 2 R \ 243![P362 2 Y+5
AWv R>ZijBP^0AC362eQ+24!3 hsij312 Yij3BPiCR ) g\ >Z^01,24ACBP^0R \ ApRL Y+iP2 ^0BPiCR0BP2 2 YRi!S+2
2436iP2 Y"z.S+/PR0362 +/P>@362 ^
R \ AWRR eHieY+6> 5"243624BCRH2433_ij3 R243h1,^Ri,S+2v4AWoZv4/Po@AWR2 YAWRt24AWv \ ^R24Q%) m 24BP[!Q+362 ^24BbR2 YR \ 2eAWoZ[Cij36>ZR \ 1
h ij3tY+>Z^v4362 Rh2I^R AWR2 - ^fQ+AWv 2^5P^R241,^ 'z e \ >qoZ^R \ 24362 eH2A2 ;jRh24BPY>qR hsij3H/P^^2 e0>ZR \ v iPBCR>@BP/Pij/Pm^ h /+BPv R>ZijB
ACQ+Q+36i ;j>@1!ApRij36^4)
*0R 24AWv \ R>@1,20^R24Q%z+ACB/+Q+Y.AWR2>Z^01!AWY+2IAWv v iP36Y+>@BP[RhiR \ 20iPBP2 - ^R24Q ,.- oq24Ab3BP>@BP[! 2 =/+ApR>ZijB,V.z
ACBPYR \ 24BA^2 v ijBPY /+Q Y.AWR2e>Zê1!AWY+2/P^>@BP[.z
\$
-m
\ i \ $ 1!%('0Aa) ; \"jNm 1%/'0A) ; \ d K LM \$- e e K6TCTM
h e%f7g h
0< iCRh2eR \ 2^f/+11!AWR>ZijB>qîjBPoZ5 /+QRi!^Rh24Q F TC)t*^ eH2AC362JY+24AWoZ>@BP[Ye0>ZR \ A,v iPBCR>@BP/Pij/P^^RfAWR2 -
^fQ+AWv 2^5P^R241ztS+iCR \ /+Q+Y.ApR2 ^8A 52 v RJR \ 2^fAC12QeH2 >Z[ \ R^ ACBPY î 362 ^f/PoZR!>@B AbB iW]C243Apoqo0/+Q Y.AWR2
ih_z
8\ i \ $ 1!%('*A ) ; \kj]m_F \ ; \$ i \.$ 1!%('0Aa) ; \"jNm 1!%('*A) ; \ \
K6T4l(M
g \ >ZÎ362 ^f/PoZR^H>@BAeôZ>Z[ \ Rh oZ5AWoZR24362 YACBPYoq2 ^h^02 v >Z24BCRHij(B - oZ>@BPh 2/+Q Y.AWR2AWoZ[CiP36>ZR \ 1'R \ ACB h ij3 2 >ZR \ 243
^RfACBPY.AC3_c Y ,.- oq24Ab3BP>@BP[ij3t9.X ,.-0/K ^h2 v R>ZijB'V M )

3 QYYR =M
MS
/ BR \ >Z^ ^h2 v R>ZijB7eH2IQ+362 ^24BbRHR \ 20R241Q+ij3Apo+Y+>:5"243624BPv 2/+Q Y.AWR2IApoq[bij36>ZR \ 1,^0362 +/P>@362 Y RhiACQ+QPoZ5!R \ 2
v ijB+BP2 v R>ZijBP>Z^R0oZ24AC3BP>@BP[ 12 R \ ijY+^0AWRt24AWv \ R>@1,2^R24Q%z+ACBPY \ 24BPv 2eoZiC^2R \ 2e362 =/P>@3624124BCRHRiJ^Rij362
AWoZo"^RfAWRh2 `- AWv R>ZijBQ+AW>@36Î/+BbR>Zo R \ 2e24BPYih 24AWv \ Rf36>@AWos)
g\ 2eS+AW^>ZvAWoZ[Cij3_>qR \ 1 sh ij3tACQ+QPoZ5j>@BP[ g d^- oZ24AC3hBP>cBP[R2 v \ BP>+/P2 ^0Ri,Rf3Ap>cBBP24/+3AWo"BP2 R He ij3yb^v4ACB
S+22hsij/+BPY>@BO+/PRhRijBN6K T4U&>CUf M ) aApv \ He 2 >Z[ \ R0>@BR \ 2BP2 R He ij3y!1AW>@BCRfAp>cBPêACB 5 0 "k C #4z
\ d K!L M \$- e ;\kj e 4; \$ L \$-m
\
6K T9(M
ef7g h h
O
Tb) - 2 ^2 R Apoqo 2 oq>Z[C>@SP>ZoZ>@R>Z2 ^4z g ]
l ) F ]
9 ) OP2 oZ2 v ReAWv R>ZijB%z&\
VN) / h H/ ]+z
\3 \$-m$ K i \ -4m;$ \'A\$-m M \$-m
G ) X?AWoZv4/PocApR2 \ e)x3 )wR4)"h ^h2 oq2 v Rh2 YApv R>ZijB \ iPBPoq5C)
J ) \ \.$ L \ -4m
O ) m 2403 h ij3h1 Apv R>ZijhB \zNACBPY362 v 2 >Z]C2!Q+A 5Ci 5 i \
> ) / h R 36>@AWo \ AWÎBPibR024BPY+2 Y"zY^ $ TACBPY[bi!Ri,^R24Q +9 )
>q[P/+362 TCrdeB(- oZ>@BP2 v iPB+BP2 v R>ZijBP>Z^R8/+Q Y.AWR2AWoZ[CiP36>ZR \ 1) g\ 2/+Q Y.AWR2>@B ^RfAW[b2QG>q^^ \ ieIB sh ij3
9 iPY+>6H.2 Y XHiPB+BP2 v R>ZijBP>Z^R3,.-0/%24AC3BP>@BP[.)
CT ) H- 2 ^2 RAWoZo"2 oZ>q[b>cSP>Zo@>ZR> 2 ^4z g ]+)

l+) F ]+)
+9 ) PO 2 oZ2 v R Apv R>ZijB%z&\)
.V ) /@hH/ ]+z
i \ \ -4m $ 2K i \$-m $ 1!Aa;&%/'0) \ F \$-m V \$-m $
\$-m$ 1!A ; %('0) \ 1h A; %('0) \$-m \$-m M
G+) \3 Lh \ -4m$ 8; \$-m
J+) X0AWoZv4/Ph o@AWR2 ;\e)x3 )wR )"^2 oZ2 v R2 Y AWv R>ZijB &\ ijBPoZ5C)
O+) m 24!3 hsij31 AWv R>ZijB & \z.ACBPY3_2 v 2 >q]b2Q+A 5Ci5 i \6)
>+) /@h Rf36>@AWo \ AWÎBPiCR024BPY+2 Y"z ^ >$ T AbBPY[Ci,Ri!^hR24QY9+)
>Z[j/+362l r deB(- oZ>@BP2!v ijB+BP2 v Rh>qiPBP>q^hR /+Q Y.AWR2Apoq[bij36>ZR \ 1 sh ij3,K!LM )

Ri yW2 24Q Rf3AWv y;iahR \ 2 eH2 >Z[ \ Rh2 Y ^f/+1Lih!Q+362 ]j>Zij/P^24336ij3[P3AWY+>Z24BCRh^4) O+/PRhRijB Q+362 ^24BbR2 Y R \ 2
AWoZ[Cij36>ZR \ 1<hsij3,R \ 2[C24BP243AWo0v4Ap^2ce \ 243_2A 1,/PoqRh>cQPoZ2ij/PRfQ+/PRBP2 R0eHij3hy \ AW^Y+>:5"243624BbR!2433_ij36^!>@B
24AWv \ i h,>ZRîj/PRfQ+/PR^ z AbBPY \ 24BPv 2 R \ 24362 >Z^R \ 2 BP2 2 Y Ri 1AW>@BCRfAp>cB ijBP22 oq>Z[C>@SP>ZoZ>@R65 Rf3Apv 2Q 243
BP2 0R eHij3hyij/PRfQ+/PR!AWR,24AWv \ eH2 >q[ \ R4)_b?ie 2 ]b243 z >cB R \ 2c,.- oZ24AC3BP>@BP[ h 3Ab1,2AeHij3yR \ 24362>ZîjBPoZ5 A
^>@BP[CoZ2eR241!Q+iP3AWo.Y+6> 5"243624BPv 2e2433_ij3AWRt24AWv \ R>@1,2^R24Q%z(e \ >Zv \ >Z^0/P^2 YRi,/+Q+Y.AWRh2IR \ 22,.- ] Apoc/P2 ^?ih
kNAWv R>ZijBP^>cBJz"ACBPY \ 24BPv 2!362 ^ /PoqRhê>cBR \ 2 h Apv RIR \ ApR0ijBPoZ5 A^>@BP[CoZ2!2 oZ>Z[C>@SP>qoZ>@R65>Z^36 2 +/P>c3_2 c
Y hsij3
24AWv \ eH2 >Z[ \ R K ^2 2AbQ+Q+24BPY+> ;2* M )
g \ 2,>@1!Q+iP36RfACBbRQ+iC>@BbRRiBPibR2,>q^JR \ AW6R N \3K R \ 2!S+AWv y -`Q+3_ijQ+AW[jAWRh2 Y iP/PRfQ+/PRe[j3AWY+>Z24BbR M>Z^
ijBPoZ5!v4AWoZv4/Po@AWR2 Y Y e0>ZR \ 3_2 ^fQ+2 v RHRiJR \ 20ij/PRfQ+/PRHQ+36iPY./Pv >cBP[R \ 2 ,.-)h /+BPv R>ZijB] AWo@/P.2 h iP3 R \ 20^h2 oq2 v Rh2 Y
AWv R>ZijB \ ) b?24BPv 2CzH6> hR \ $ 2 ,.-)h /+BPv R>ZijB >Z^ S+2 >@BP[ 3624Q+362 ^h24BCR2 Y SC5 ^24Q+AC3hAWR2^>@BP[CoZ2ij/PRfQ+/PR
BP2 0R eHij3hyC^4z R \ 24B 8\I>Z ^
2436i h ij3,AWoZmo eH2 >Z[ \ Rh^!>@B;BP2 0R eHij3hyC^,iCR \ 243JR \ AbB R \ 2ijBP2AW^îPv >@AWR2 Y
e0>ZR \ AWv R>ZijBA& \z"ACBPYîR \ 2,2 oZ>Z[C>@SP>qoZ>@Rh>q2 ^ h ij3?R \ 2!R \ 2 ^2!BP2 0R eHij3hyCêAC362D/P^R /+Q Y.AWR2 Y Apv v ij36Y+>@BP[
Ri.z
\ L \$-m K6T BV M
h
>
OBSTACLE GOAL
d goal
15deg Angle to goal
ROBOT
15deg
RANGE SENSORS
>Z[j/+36279+rG< \ AWR0R \ 236iPS+iCRIyPBPie0^0ACS+iP/PR0>ZRÎ^ /+336ij/+BPY+>@BP[C^

g\ 2.h /PoZo%ijB(- oZ>@BP2AWoZ[CiP36>ZR \ 1 >Z^0^ \ ieIB>cB >Z[.)%TC);/ R >@Bb]CiCoZ]C2 ^0S+AWv y-`Q+3_ijQ+AW[jAWRh>cBP[JRi,v ij1!Q+/PR2
R \ 2IiP/PRfQ+/PRH[j3AWY+>Z24BbR^l; \ihsij3 R \ 2eAWv R>ZijBv \ iC^24B%z.ACBPY \ 24BPv 2/+Q+Y.ApR>@BP[!R \ 2ê 2 >Z[ \ RH2 oZ>q[b>cSP>ZoZ> -
R>Z2 ^4z.S+2 hsij362eR \ 2AWv R>ZijB>Z^ Apv Rf/+AWoZoZ5Q+243!hsij31,2 Y") *0RHR \ 2BP2A;jR?R>@1,2e^R24Q%z.Apoqo ih R \ 22eH2 >Z[ \ RÎAC362
/+Q+Y.AWR2 YAWv v ij3_Y+>cBP[,RiJR \ 2R241!Q+iP3AWo+Y+>65"243624BPv 2e24336ij3t1,/PoqRh>cQPoZ>Z2 Y Sb5!R \ 2 >@3t2 oZ>Z[C>@SP>qoZ>@Rh>q2 ^ ) g\ 24362 -
h ij3_2CzWR \ 20ijBPoZ5,^Rij3hAW[C2 3_ 2 =/P>@36241,24BbRÂC3_2 hsiji 3 R \ 2BP24/+3AWo.BP2 R eHij3^y eH2 >Z[ \ RhÂCBPY!2 oZ>Z[C>@SP>qoZ>@Rh>q2 ^ z.ACBPY
R \ 2eo@AW^R ,.-)h /+BPv R>ZijB] AWo@/P2+ \ACBPYQ+A45bi 5 \`)
,K!LM,/+Q+Y.AWRh>cBP[ >Z^ ^hoq>Z[ \ RoZ5;Y+6> 5"243624BCR z0ACBPY;Rf3ACBP^hocApR2 ^!>@BCRhi R \ 2iP(B - oq>@BP2 /+Q+Y.ApR2^ 2 +/P24BPv 2
^ \ i eIBJ>cB >Z[.)Cl+) <?iCR2HR \ 20v \ ACBP[C20>@B,iP36Y+243 i h%^R24QPm^ JeACBPQ Y Oê \ >Zv \ 36 2 +/P>@362 ^R \ AWR R \ 2IBP2 R eHij3y
ij/PRfQ+/PR![j3hAWY+>Z24BCR^ z N \zHAC362^RiP362 Y S+2 R eH2 24B ^R24QP^4) g \ 2 AWoZ[CiP36>ZR \ 1 >Z^ R \ 24362 h iP362ôZ>Z[ \ Rhoq5
1,ij362v ij1!Q+/PRfApR>ZijB+AWoZoZ5 A2 ;+Q+24BP^>Z]C2JR \ ACB^RfACBPY.Ab36E
Y ,.- oZ24AC3hBP>cBP[ACBPY9 X ,.-0/ z+ACBPYAWoZî!36 2 +/P>@362 ^
R \ 2e^Rij3hAW[C2i h R \ 2e[j3AWY+>Z24BbR^ \ AW^ eH2 oZo AW^0R \ 2BP2 R eHij37y eH2 >Z[ \ RÎACBPY2 oZ>Z[C>@SP>Zoq> R>Z2 ^4)
J . 6 O
/ Bij36Y+243 RhiR2 ^RtR \ 2IAWoZ[CiP36>ZR \ 1,^HY+>Z^v4/P^^2 Y"z.AJ3624AWoZ>Z^R>Zve1,ijSP>ZoZ236ijS+ibR Q+36ijSPoZ241 e0AW^v iPBP^>ZY+24362 Y")
g\ 2!36ijS+ibR1/P^hR^H BPY >ZR^40e A45Rhi8A[CiPAWo Q iC^>ZR>ZijB>@B Alpd 24Bb]P>c3_ijB+1,24BbR2e \ >Zoq^hRA ]CiC>ZY+>@BP[ijS(-
^RfAWv oZ2 ^4zS+/PR,ijBPoZ5;362 v 2 >Z]C2 ^ Q+A 5Ci"5 ^,AWRR \ 224BPY;iah 24Apv \ Rf3_>cApo z%e \ 24B R \ 2ij/PRv ij1,2>Z^!yjBPieIB%)
g\ 2eijBPoZ5 >@B(sh ij31!ApR>ZijBA ] AW>Zo@ACSPoZ2,Ri,>qR?Y./+36>@BP[ A,Rf36>@AWo"AC3_2I^24BP^hij303624AWY+>@BP[C^ z.ACBPY>@B(h iP31!AWR>ZijB>ZR
\ AW^Hoq24Ab3BCR h 36iP1 Q+3_2 ]j>Zij/P^0R 36>@AWoZ^4)K/ BiP36Y+243tRi,24BP^f/+362eR \ 2ev ijBbRf36ibo Q+iboq>Zv 5oZ24AC3BbRH>ZÎAW^0[C24BP243hAWoZoq5
ACQ+QPoZ>qv4AbSPoq2AWÎQ+ib^^>@SPoZ2Cz.R \ 2,36ijS iCRH>q^Rf3Ap>cBP2 YijBA,^2 +/P24BPv 2iah 3AbBPY+ij1,oZ5[C24BP243AWRh2 Y24BC]P>@36ijB(-
1,24BCRh^4z0e >ZR \ 24AWv \ /P^2 Y h ij3 ijBPoZ5A,^>@BP[CoZ2Rf3_>cApo )ta="5 2 v R>Z]C2 oZ5Cz.R \ >qêQ+36iW]P>qY+2 Y h iP36v 2 Y2A=; QPoZij3hAWR>ZijB
ih BP2Ae 24BC]P>@36ijB+1,24BbRÂCBPY^>ZRf/+ApR>ZijBP^4z"ACBPYîQ+36iW]j>ZY+2 Y A1,ij3_236ijS+/P^hR36ijS+ibRIv ijBbRf36ibo Q+iCoZ>Zv 5
R \ ACBv ij/PoZY S 2AWv \ >Z2 ]C2 YiPBA4Hj; 2 Y24BC]P>@36ijB+1,24BbR4)
3
9) 9O LRE Q9;E5 NO
g\ 2 36ijS+ibR>q^^h>c1,/Po@AWR2 Y 0e >ZR \ .H ]C2 3ACBP[b2cH BPY+>@BP[ >cB+Q+/PRh^ce \ >Zv \ [C>Z]C2 >ZRAWv v4/+3ApR2Y+>q^hRfACBPv 2
1,24AW^f/+3624124BCR^HRi,ijSP^hRf3/Pv R>ZijBP^4z e \ >Zv \ AC3_2e^fQ+AWv 2 YAWv436ib^^HR \ 236ijS iCR^ h iP3 0e AC36Y AC3_vh 36ij1 9f]
U
Ri $29(] AWRJTG >@BbR2436]WAWoZ^QK ^2 26 >Z[.)+9fM )(/sReAWoZîAWo e0A 5jÎyPBPie0^R \ 2Y+>Z^RfAbBPv 2 ACBPY AbBP[CoZ2Rhi R \ 2
C[ ijAWo"362 o@AWR>Z]C2JRi,>ZRÎv4/+3h3624BCRQ+iC^>ZR>ZijBACBPY h AWv >@BP[.)
g \ m2 eHij36oZYR \ 2 3_ijS+iCR iPv v4/+QP>q2 ^t>ZÂ^ =/+Ab362 36iPij1ze?>qR \ 3hACBPY+ij1,oZ5QPo@AWv 2 Y!v ijBb]C2A;,Q+iCoZ5P[CijB+AWo
ijSP^RfAWv oZ2 ^H>@B>ZR4) g \ 2 3_ijS+iCRH^R AC36RÂWRHA,3ACBPY+ij1'Q+iC^>ZR>ZijB e?>qR \ A,3ACBPY+iP1 ij36>Z24BbRfAWR>ZijB%z+ACBPY \ AW^
Ri 3624AWv \ R \ 2e[CiPAWosz e \ >Zv \ >ZÎAWoZî AWRA!3ACBPY+ij1 Q+iC^>ZR>ZijB%)
g\ 2,36ijS+ibR1,iW]C2 ÎSb5^2 oZ2 v R>@BPY [ h 36ij1 AY+>q^hv4362 R2!^2 Ri h AWv Rh>qiPBPQ
^ K ^2 2,^2 v R>ZijB G+)x(l M ) / RIY+iP2 ^
R \ >Z^ /+BbR>Zo ACBAWv R>ZijB362 ^f/PoZR^>@BA,v iboqoZ>Z^>ZijB e0>ZR \ ACBijSP^RfApv oq2Cz%AC33_>q]P>@BP[8ApR R \ 2[CijApo zPij3?A,R>@1,2 -
ij/PR K R \ 2e36ijS iCR0>ZÎAWoZoZi eH2 YijBPoZ5AoZ>@1,>ZR2 Y BP/+1S 243Hi h ^R24QP^0>@B e \ >Zv \ Ri!3624Apv \ R \ 2e[bijAWZo M ) g \ 2
Rf36>@AWo R \ 24B24BPY+ÂCBPYR \ 2!36ijS iCRI362 v 2 >Z]C2 ÂQ+A 5Ci 5 S+AW^2 YijB>ZR^^ H B+AWo Q+ib^>ZR>ZijBAWÎY+2 ^hv436>@S+2 Y >@B
^2 v R>ZijcB G )xl+) g \ 2o@A 5Cij/PRHi h R \ 2,36ijiP1 >Z^R \ 24B3hACBPY+ij1,>Z^2 Y"z.AbBPYR \ 236iPS+iCR0^hRfAC36R^0A!BPA2 e Rf36>@AWos)
XHijBP^ 2 +/P24BCRhoq5Cz R \ 2!ijBPoZ5>@(B h iP31!AWR>ZijBR \ ApReR \ 2 36ijS iCR \ AWÂWêRhiR \ 2 =/+AWoZ>ZR65i h >ZR^,AWv R>ZijBPê>Z^
R \ ^2 H B+AWo Q+A 5Ci 5>ZRI>Z^0[C>Z]C24B%z%ACBPY Y e \ ApR0>ZR \ AW^0oZ24AC3BbmR h 36ij1 Q+3_2 ]j>Zij/P^Rf36>@AWoZ^4)
2 L+ME7:Q5 NOP=@?A OPQ=
/ BR \ 2mhsiCoZoZie0>@BP[ 362 ^f/PoZR^4zBeH2v ij1!Q+AC362?R \ 2I2 5"2 v R^Hih ACQ+QPoZ5P>@BP[!R \ 229Y+>65"243624BbR0/+Q+Y.AWR2e3/PoZ2 ^0ijB
R \ 236ijS iCR0oZ24AC3BP>@BP[RfAW^fyY+2 ^v436>@S+2 Y>@B^h2 v R>ZijBcG+) T
OPRfAbBPY.AC36Y$,.- oZ24AC3hBP>cBP[_K 2 +/+AWR>ZijBcG(M0e >ZR \ L ]è \ 24BP2 ]C243eABPijB(- [j362 2 Y+5AWv R>ZijB>q^Q 243!-
hsij31,2 Y")
,K!LM2K 2 +/+AWR>ZijBT4l( M )
9iPY+>:.H 2 Y ,.- oZ24AC3BP>@BP[ K 2 +/+AWR>ZijBUf M )
/ BApY+Y+>qRh>qiPB%zH e 2ev ij1!Q+Ab362eR \ 2e2 "5 2 v R0ih ACQ+QPoZ5j>@BP[R \ 2 ^2,/+Q+Y.AWR2 ê/P^>@BP[S+iCR \ S+Apv y(0e AC36Y3624QPo@A45
K ^2 v R>ZijBY+9 ) THM ACBPYR \ 2eiPB(- oq>@BP2!AWoZ[Cij3_>qR \ 1 K ^2 v R>ZijBV' M )
g\ 20^h>c1,/Po@AWR2 Y36ijS+ibRe Ap^ Rf3AW>@BP2 Y0e >ZR \ J AWv R>ZijBPÂ ]WAW>ZocAbSPoq2eRhie>qR r R /+3BJoq2 sh RITG zbRf/+3B!3_>q[ \ R
TG z.ij3?yW2 24QR \ 2^fAb1,2 \ 24AWY+>@BP[.z"ACBPY2 >qR \ 243e1,iW]C2^h iP3 0e AC36YA4HP; 2 YY+>q^hRfACBPv 2.z ij3036241!Ap>cBijB
R \ 2H^fAC1,2H^fQ iCR4) g\ >Z^ 1,24AbBCR >ZR \ AWY!iPBP2 3_2 Y./+BPY.ACBCRHAWv R>ZijB B+AC12 oq5!BPiCR1i ]P>@BP[eij3 v \ ACBP[C>@BP[
\ 24AWY+>@BP[8AWRAWoZo R \ ApR 0e AW^0BP2 ]C243i /P^2 h /Po">@BAWv \ >Z2 ]j>@BP[>ZR^0iPSPD`2 v Rh>q]b2C)
/ BP^hR24AWY i0 h 362 v 2 >Z]j>@BP[ Q+A 5Ci"5 ^ \0AWRe24Apv \ R>@1,2^R24Q%z%R \ 236ijS+ibReijBPoZ5 362 v 2 >Z]C2 Y;AQ+A45bi5 AWR
R \ 2e24BPYih 24AWv \ Rf36>@AWos) g\ 2^H B+AWo Q+A 5Ci5 362 v 2 >Z]C2 YY+24Q 24BPY+2 YijB \ ie R \ 2R 36>@AWo%v ijBPv o@/PY+2 Y"r
Q&R /@h R \ 236ijS iCRe1,iW]C2 Yc0e >ZR \ >@B A^f1!Apoqo=HP; 2 Y 3AWY+>@/Pêih R \ 2,[CiPAWo Q+ib^>ZR>ZijB%z AQ+A 5Ci5 [C>Z]C24B
e0AWêTC)
&?V g.\ 2 36ijS iCR 3_2 v 2 >q]b2 Y AQ+A 5Ci5 S+AW^h2 Y iPB>ZRêY+>Z^RfACBPv 27h 36ij1 R \ 2,[CijApoke \ 24B>qRev43hAW^ \ 2 Y
>s)w2C)
i
]1:G 2A+; Q+SK 0l
# M
e \ 243_2 0e AW^0R \ 2eoZ24BP[CR \ ih ijBP240e AWoZo iah R \ 2J^ +/+AC36236iPij1)
&! / h R \ 2Rf36>@AWo0R>@1,2 YB- iP/PR$K >@B;R \ 23_2 ^f/PoZR^ Q+362 ^24BbR2 Y;>@B;R \ >q^^2 v R>ZijB%zHR \ >Z^0e AW^!Ash R243!lf](]
^Rh24QP^ M zPR \ 236ijS iCRI362 v 2 >Z]C2 YR \ 2e^fAC12Q+A45bi 5 AW^ sh ij30AJv43AW^ \ z.S+/PR0e >ZR \ A $2] ):!9 S+iPBj/P^msh ij3
BPiCRHv43Ap^ \ >@BP[.)
/sRH^ \ ij/PoZYS+2eBPiCR2 YR \ AWR R \ 2eQ+A 5Ci5 h ij3 v43AW^ \ >@BP[70 e AW^ v \ iC^24B>cBA^h AW>@36oZ5 AC3hSP>qR 3AC36570e A 5^>@1!QPoZ5
Ri[C>Z]C2 A \ >Z[ \ 243JQ+A45bi5Nsh ij324BPY+>@BP[/+Q BP24AC36243eRhi R \ 2[CijAWosz AbBPYA1Aj; >@1,/+1 Q+A 5Cia5 ih iPBPoq5
+] ):+G )
g\ 2eY+>Z^v ij/+BbR.h AWv Rhij3 z e Ap^0^2 R0Ri +] )xUCU+zBe \ >Zv \ [C>Z]C2 êA \ >Z[ \ 243.He 2 >q[ \ R>@BP[Ri AWv Rh>qiPBP^0R \ AWR
oq24ApYRi!R \ 2e[CijAWo%>@BR \ 2^sh 2AhHe 2 ^R0^Rh24QP^4)
T]
g 3AW>@BP>@BP[812 R \ ijY O+/Pv v 2 ^h^ h /Po 36ijS iCR^ \ Q+Y.AWR2 ^RfACyp24B g 36>@AWo"oZ24BP[CR \
Kh 36ij1 9fJfM Ks1,>ZoZoq>ZijBP^ M K h^ R24QP^ M
OPRfACBPY.AC36Y T J+):] OfG+) T
,K0L M T4l 9+) > G+TC6) J
9 X,.-0/ T V 9+:) ] Gf9+6) 9
g ACSPoZ2TCr,O+/+1!1!AC365ih0^f/Pv v 2 ^h^ h /PoH36ijS iCRE ^ K R \ iC^2 A ]C243hAW[C>@BP[!/8]1xU(GA4]b243AW[C2!Q+A 5Ci5;iW]C243eR \ 2
ocAp^RIT] z:]f]f]?Rf3AW>@BP>@BP[R 36>@AWoZ^ M zh 36ij1 9(JY+>:5"243624BbRL!ACBPY Jv ij1,SP>@B+AWR>ZijBP^tRf3AW>@BP2 Y /P^h>cBP[,S+AWv yfe0AC36Y
3624QPo@A45C)JXHiCo@/+1!BP^^ \ i e BP/+1S 243ih /+Q+Y.ApR2 ^1!ApY+2iW]C243?R \ 2,Rf36>@AWoZ^4z"AbBPY R \ 2A ]C243hAW[C2BP/+1S 243
ih ^hR24QPÎ36 2 +/P>@362 YRi H BPYR \ 2e[CiPAWo"SC5R \ 2e24BPY")
g \ Q2 ,.-)h /+BPv R>ZijB e0AW^ 3_24Q+362 ^24BbR2 Y Sb5$JBP24/+3AWoBP2 R eHij3yb^4z%ijBP27hsij324AWv \ A ] Ap>qo@ACSPoZ2AWv R>ZijB%)
aAWv \ BP2 R eHij3y \ AWY lfJ>@B+Q+/PR^ z9 \ >ZY+Y+24B BPijY+2 ^4zHACBP-Y A^>@BP[CoZ2iP/PRfQ+/PR4zACBPY;/P^2 Y ^>Z[j1,iC>ZY.AWo
AWv R>Z] AWRh>qiPB h /+BPv R>ZijBP^!ih0R \ 2`hsij31KiM6 TD#K6T@$ M2e \ >Zv \ 362 ^Rf3_>qv Rh2 Y R \ 2ij/PRfQ+/PRJihIR \ 2
BP2 0R eHij3hyC^ RiS+2 R eH2 24 B ]eACBPYTC) g\ 2lfJ0>@B+Q+/PR^meH24362HY./P20Rie^ Q+3624AWY+>@BP[R \ 2Oe3624Apo+] AWo@/P2 Y^24BPîj3
^RfAWR2 Âpv436iC^^J^2 ]C243AWot>@B+Q+/PRBPiPY+2 ^,24AWv \ K)9 hsij324AWv \ i h0R \ E 2 G3ACBP[C2 ^ kz G hsij3R \ 2!Y+>Z^RfACBPv 2Ri
[CijAWoszjAbBPE Y Jhsij3 R \ 2eACBP[CoZ2RiR \ 2I[CiPAWZo M zj/P^>@BP[ A h iP31 i h"v ijAb36^2v ijY+>@BP[.F) Nij3 A2 ;+AC1!QPoZ2Cz.AJ3ACBP[C2
] AWo@/Pc 2 e0AW^^fQ+3624AWY AWv43_iC^Q ^ 9>@B+Q+/PRBPijY+2 ÂW^!6> h>Z R eH24362AWv R>Z] ApR>@BP[ 9^h>cBP[boq2 - >@B+Q+/PR^>Z[j1,iC>ZY.AWo
/+BP>qRh^4z+24AWv \ e0>ZR \ AJY+6> 5"243624BbR SP>@AW^0]WAWo@/P2C) .g \ 2 ^22 eH24362ev \ ib^24Bî!R \ ApR Ap^0R \ 23ACBP[b2IY+2 v43624Ap^2 Y"z
R \ 2>@B+Q+/PR^>@BPv43624AW^2 YijBP2,A h Rh243HR \ 2ibR \ 2430/+BbR>Zo AWR?]C24365v oqib^2!3ACBP[C2eR \ 2 Y 5 eH24362Apoq o xijB ) g \ >Z^
362 ^Rf36>Zv R2 YR \ 2e>@BPY+>Z]j>ZY./+AWo >@B+Q+/PR] AWo@/P2 êRi!A!3AbBP[C2ei h ],RiTC)
/ BR \ >q^
^ eHij3y,R \ 2e^>@1!QPoZ2ktiCoZR
41!ACB+BQ+36ijS+ACSP>ZoZ>ZR65Y+>q^hRf36>@S+/PR>ZijB e0AW^0/P^2 YRi!Q+36iW]P>ZY+2Q+36ij(S -
ACSP>Zoq>ZR>Z2 ^^ hsij3H^2 oZ2 v R>@BP[AWv Rh>qiPBP^4z
[ % [
K2& \ [N\)M %('*
)
[ % K6T G(M
e \ 24362 AWYWD/P^R^!R \ 23hACBPY+ij1!BP2 ^^,i h^2 oZ2 v R>ZijB%) g \ >Z^ 1,24AbBP^!R \ AWR!AWv R>ZijBP`^ e0>ZR \_\ >Z[ \ 243 ,.-
] AWo@/P2 ^ \ A4]b2e[j3624AWR243HQ+36iPS+ACSP>Zoq>ZR65i h S 2 >@BP[ v \ iC^24B%z.S+/PR?iCR \ 2430AWv Rh>qiPBPÎ1!A 5 S+2ev \ iC^24B>@BP^Rh24AWY")
*I^tR \ 2IACQ+Q+36i ;P>@1!AWR>ZijB,ai hR \ 2 ,.-)h /+BPv Rh>qiPB>c1Q+36iW]C2 ^4zjAbBPY!R \ 2IAC1ij/+BCR ai hA2 ;+QPoZij3AWRh>qiPB 36 2 +/P>@362 Y
[Cij2 êY+i eIB%zNR \ 2,] Apoc/P2i h >Z^362 Y./Pv 2 Y /+BbR>Zo R \ 2J[j362 2 Y+5 Apv R>ZijB \ AWÂWo@1,iC^ReQ+36ijS+AbSP>qoZ>ZR65 TJi h
S+2 >@BP[^2 oZ2 v R2 Y AWRe24AWv \ ^R24Q%) / BR \ 2 362 ^ /PoqRh^Q+3_2 ^24BCRh2 Y"z e0AW^,362 Y./Pv 2 Y oq>@BP24AC3_oq5;S+2 R eH2 24B A
] AWo@/P2ai h%]+6) ]fG,ACBP$ Y ]+6) ]+Ti ]b243 (l ]+:z ]f]f]Rf36>@AWoZ^4z.A h R24 3 e \ >Zv \ >q.R e Apm^ H;j2 YAWR ]+:) ]+Tb)
KJ E=CO
g\ 24362!AC362J^2 ]C243AWo \ 24/+36>Z^R>Zv Q+Ab3AC1,2 R243_ÎR \ ApR 1,/P^ReS+2!^h2 RIY./+36>@BP[Rf3AW>@BP>@BP[ %R \ 2,] Apoc/P2 ^,/P^2 Y
h ij3?R \ 2,Y+>Z^v ij/+BbR^h Apv Rij3 ACBPYR \ 2,2A;=QPoZij3hAWR>ZijB] AWo@/P2 \ A ]C2!S+2 24BY+2 ^hv436>@S+2 Y ACS iW]C2C) g\ 2 ^2
e 243_2v \ iC^24B Ah Rh243IA^f1AWh oZoAC1,ij/+BbRIiah 2A;+Q+2436>@1,24BbRfAWR>ZijB%z S+/PRBPiAWRhR241!QPR2e0AWê1!AWY+2,Ri H BPY
ijQPR>@1!AWoN] AWo@/P2 ^ K 2C) [.) A2h AW^Rh243362 Y./Pv R>ZijB>@B! v4ACB!^ Q+2 2 Y/+Qv ijBC]b2436[C24BPv 2Ri!AeîCo@/PR>ZijBM ) g\ >Z^
oq24A ]C2 ^HR \ 2Rf3Ap>cBP>@BP[!3AWRh2!ACBPY!R \ 2 g d Q+AC3AC1,2 Rh243mL"zfe \ >Zv \ eH24362mhsij/+BPY Rhi \ A ]C2IAJ^>Z[jBP>6H.v4ACBbR
2 5 2 v R iPBR \ 2 AbSP>qoZ>ZR65,i.h R \ 20BP24/+3AWo.BP2 R He ij3yb^ RhiIoZ24AC3B%zPACBPY,R \ 2 =/+Apoq>ZR65iah R \ 2HîCo@/PR>ZijBP^HAC336>Z]C2 Y
AWR4)
g\ 2t[j3ACQ \ ^">@B >Z[.) V^ \ ie R \ 2t] AC36>@AWRh>qiPB>@BJA4]b243AW[C2Q+A 5Ci"5 ^"362 v 2 >Z]C2 Y!Sb536ijS+ibR^ Ash R243"S 2 >cBP[
Rf3AW>@BP2 YijB Gf+] z:](]fe] 3ACBPY+ij1oq5[C24BP243ApR2 Y83_ijij1^ /P^h>cBP[ %/ >@BwÎS+AWv yf0e AC3_Y 3624QPo@A 58AWoZ[CiP36>ZR \ 1) g\ 2
[j3ACQ \ ^0>@B >Z[.) G ^ \ ie R \ 2,362 ^f/PoZR^ê \ 24BijB(- oZ>@BP2 /+Q Y.AWR2 ^K ^2 v R>ZijBV'0M AC362,/P^2 Y>@BP^Rh24AWY") g\ 2
v ijBCRhij/+3 QPoZiCR^ Ab362Hv ijBP^Rf3/Pv Rh2 Y7h 3_ij1 362 ^f/PoZR^tijSPRfAW>@BP2 Y7h 36ij1 +] ) TCzf+] ):+G zPTC):] zCl+):+] zf+9 )6+] zWV.):] ACBPY
TCT

g ;g

4 1
0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2
0.4
1.5
1
0.2
0.5 +
0
!

0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
λ Trials (thousands)

g ;g

4 1
0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2
0.4
1.5
1
0.2
0.5 +
0
!

&
0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
"$#
% g ;g

4 1
0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2
0.4
1.5
1
0.2
0.5 +
0
0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
>q[P/+362!V.rN Xn JXHijBbRij/+3eQPoZiCR^J^ \ ie0>@BP[ \ ie R \ 27H B+AWoQ+A 5Ci5 A hsR2432Gf]+z:](]f]!R 36>@AWoZ^]WAC36>Z2 ^2hsij3
24AWv \ ih R \ 2R \ 3_2 2/+Q+Y.ApR23/PoZ2 ÂCQ+QPoZ>Z2 Y /P^>@BP[ 0 &# 3 :;&8A +4C.h ij3 Y+>:5"243624BbR] AWo@/P2 êih
ACBPY
'
L) $ 5 O+AC1!QPoZ2,Rf3Ap>cBP>@BP[v4/+36]b2 êRfACyW24BEh ij3?24AWv \ /+Q+Y.AWR2!3/PoZ2Cz%v ij3h362 ^fQ+iPBPY+>cBP[RiR \ 2]WAWo@/P2

ihs! ACBPYYL 1!Ab3yW2 Y Sb5 A $ ijB,24Apv \ v ijBbRij/+3 QPoZiCR4) g.\ 2IY+iCRhR2 Y!oZ>@BP2e>q^HR \ 2BPij31!AWoZ>Z^2 YA ]C243Ap[C2
)( +* '
Bj/+1,S+243?ih ^hR24QP^0RfACyp24B>cB24AWv \ Rf36>@AWo sK 1A;j>@1,/+1 Rf3_>cApo"oZ24BP[CR \ 0e AWÎlf](]^hR24QP^ M )
T4l

Yg 1g

4 1
0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2 +
0.4
1.5
1
0.2
0.5
0

0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
4
λ

1
Yg 1g
Trials (thousands)

0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2 +
0.4
1.5
1
0.2
0.5
0
0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
4
"$#
λ

1
1g
Trials (thousands)
Yg
0.98
3.5 0.97
0.96
0.95 0.8
3
Average reward
2.5 0.6
η 2 +
0.4
1.5
1
0.2
0.5
0
0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50
>q[P/+362QG+rN Xn JXHijBbRij/+3eQPoZiCR^J^ \ ie0>@BP[ \ ie R \ 27H B+AWoQ+A 5Ci5 A hsR2432Gf]+z:](]f]!R 36>@AWoZ^]WAC36>Z2 ^2hsij3
24AWv \ ih+R \ 2HR \ 362 2H/+Q+Y.AWRh2 3h/Poq2 ^HACQ+QPoZ>Z2 YD!-)k7+hsij3"Y+>65"243624BbR ] AWo@/P2 ^ ih^J ACBPYL") P O+AC1!QPoZ2
'
Rf3AW>@BP>@BP[ v4/+3_]C2 ^0RfACyp24B7hsij3H24AWv \ /+Q+Y.ApR23/PoZ2Cz.v iP3362 ^fQ ijBPY+>@BP[ Ri,R \ 2e]WAWo@/P2,ih_ ACBPYNL1!AC3yW2 Y
( +* '
SC5A $'ijB24AWv \ v ijBbRij/+3QPoqibR4) g \ 2,Y+iCRhR2 YoZ>@BP2!>ZêR \ 2BPiP31!AWoZ>Z^2 Y A ]C243Ap[C2Bj/+1,S 243ih^R24QP^

RfACyW24B>@B24Apv \ Rf3_>cApo Ks1!A;P>@1,/+1 fR 36>@AWo%oZ24BP[CR \ e Ap^ l(]f],^R24QP^ M )
T9
g 3AW>@BP>@BP[812 R \ ijY O+/Pv v 2 ^h^ h /Po 36ijS iCR^ \ Q+Y.AWR2 ^RfACyp24B g 36>@AWo"oZ24BP[CR \
Kh 36ij1 9fJfM Ks1,>ZoZoq>ZijBP^ M K h^ R24QP^ M
OPRfACBPY.AC36Y T > G+)@T GfO+) >
,K0L M lCl V.:) ] GCl+6) J
9 X,.-0/ lWV l+w) V VjU+6) J
g ACSPoZ2l+r,O+/+1!1!AC365ih0^f/Pv v 2 ^h^ h /PoH36ijS iCRE ^ K R \ iC^2 A ]C243hAW[C>@BP[!/8]1xU(GA4]b243AW[C2!Q+A 5Ci5;iW]C243eR \ 2
ocAp^RT]+z6]f]f]eR 3AW>@BP>@BP[!Rf36>@AWoZ^ M z>h 36iP1 9(JY+>65"243624BbR2LACBPY v ij1,SP>cB+ApR>ZijBPÎ/P^>@BP[ijB(- oZ>cBP2!/+Q Y.AWR2 ^4)
XHiCo@/+1!BP^^ \ ie BP/+1,S+243?ih /+Q Y.AWR2 ^1!ApY+2iW]C243HR \ 2eR 36>@AWoZ^4z AbBPYR \ 2,A4]b243AW[C2BP/+1,S+243?ih^R24QP^
36 2 =/P>@362 YR7i H BPYR \ 2e[CijAWo"Sb5 R \ 2e24BPY")
"$#

4 1
0.98
3.5 0.97
0.96 0.8
3 0.95
Average reward
2.5 0.6
η 2

0.4
1.5

1
0.2
"$#
0.5
0
0 0.2 0.4 0.6 0.8 1 0 1 2 3 4 5 6
λ Updates (millions)
>q[P/+362J+r N X g \ 2Hv ijBbRij/+3 QPoZiCR ^ \ i e?^ R \ 2HQ+A 5Ci5!oZ2 ]C2 oZ^ AWv \ >Z2 ]C2 Y!SC5e9 X ,.-0/Rf3hAW>@BP2 Y36ijS iCR^
Ah Rh243TC)6G 1,>ZoZoZ>ZijB /+Q Y.AWR2 ^4) P g\ 2,^fAC1,2JR \ 362 2,2A;+AC1!QPoZ2![j3AbQ \ ^h 36ij1 >Z[.)3G+zNR \ >ZêR>@1,2
'
( +* '
QPoqibRR2 Y Ap[jAW>@BP^R0/+Q+Y.AWRh2 ^ 3ApR \ 243HR \ ACBR 36>@AWoZ^4)
L8I]+):] zf]+)xlfG+z]+):G z ]+)6OfG+zf]+) >(G+zjTb):]mhsij3 24AWv \ ih R \ 2 R \ 362 2I/+Q+Y.ApR203/PoZ2 ^ /P^2 Y"zPACBPY!^ \ ie;R \ 2 H B+AWo
Q+A45bi 5 A ]C243Ap[C2 Y iW]C243tR \ 2Jo@AW^RT] z:]f]f]Rf36>@AWoZ^4) g\ 2J[j3ACQ \ ^?ijBR \ 236>Z[ \ R0^ \ ie R65+QP>qv4Apo oZ24AC3hBP>cBP[
v4/+36]C2 ^.h ij336ijS iCR^R \ AWR0oZ24AC3hBRhi83624Apv \ R \ 2[CiPAWo"v ijBP^>Z^Rh24BCRoZ5C) /sR>Z^2eHij36R \ 36241241S 2436>@BP[R \ AWR
R \ 2e2A=; QPoZij3hAWR>ZijB h AWv RiP3 \ AWÎ3624AWv \ 2 Y>ZR^1,>@BP>c1,/+1 ]WAWo@/P2ih ]+):]+TeAWR0lf] z:]f]f]JRf36>@AWoZ^4)
< 2^2 2 R \ AWRR \ 2 Q 243!h iP31!ACBPv 2i, h R \ 2 36ijS iCR^Rf3hAW>@BP2 Y /P^>@BP[ S+AWv y(e Ab36Y 3624QPo@A45 >Z^]b24365
^24BP^>ZR>Z]C2Ri R \ 2v \ ib>qv 2ieh oZ24AC3BP>@BP[ Q+AC3Ab1,2 R2436^ ;AbBPY "L z ?e >qR \ ^RfAbBPY.AC36Y,.- oZ24AC3hBP>cBP[;iPBPoq5
Q+36ijY./Pv >@BP[ijBP2,Rf3h/Poq5^ /Pv v 2 ^^ h /Po 36iPS+iCR4) g.\ 2362 ^f/PoZR^^sh ij3 ^f/Pv v 2 ^^ h /Po "36ijS+ibR^ K R \ ib^2A ]C243AW[C>@BP[
/8]x1 UfG Q+A 5Cia5 AWv436iC^êR \ 2H B+AWoT+] z:](]f!] R 36>@AWoZ^ ,M AC362!^f/+11!AC36>Z^2 Y >@B RfAbSPoq2TCzie \ 24362>qRJv4ACB S+2
^2 24BR \ ApR 9 X,.-0/ ACBPYc,K!L0M Q 243!sh ij31 h AC3HS+2 RRh243HR \ ACB^R ACBPY.AC36Yc,.- oZ24AC3BP>@BP[.)
/ Bv iPBCRf3hAW^R4zPR \ 2ev ijBbRij/+3?QPoqibR^msh ij3HijB(- oZ>@BP2!/+Q+Y.AWR>@BP[NPK >Z[.)GfH M ^ \ ie h AC3HoZ2 ^^^24BP^>ZR>Z]j>ZR65Ri
R \ 2Iv \ iC>Zv 2i"h oq24Ab3BP>@BP[ Q+AC3AC12 R2436^4) *I^Hv4ACBS+2^2 24B h 36iP1 RfACSPoZ2el+zCR \ 2IBj/+1,S 243 AbBPYp+/+AWoZ>ZR`5ih
R \ 2e^f/Pv v 2 ^^ h /Po 36iPS+iCR^H>Z^0[j3_24AWRoZ5 >@BPv43624Ap^2 Y"z0e >ZR \ R \ 2S 2 ^RI362 ^ /PoqRh^mh iP39 X,.-0 / Rf3AW>@BP2 Y36ijS iCR^
/P^>@BP[ ijB(- oZ>@BP2!/+Q+Y.AWRh2 ^4)
< \ 24B v iP1!Q+AC36>@BP[R \ 2Q+243!sh ij31!AbBPv 2 i0 h R \ 2ijB(- oZ>@BP2Rf3hAW>@BP2 Y 36iPS+iCR^ z R \ 2 A ]C243AW[C2Q+A45bi"5 ^
362 v 2 >Z]C2 Y Sb5R \ 2 36iPS+iCRêY+iBPiCR^ \ i e R \ 2e \ iCoZ2 QP>Zv Rf/+362C) g\ 2Bj/+1,S+243Jih ' +-A . R ACyW24B>@B
Gf+] z:]f](e] Rf36>@AWoZ^0]WAC36>Z2 ^0v ijBP^h>qY+243hACSPoZ5S+2 R He 2 24BR \ 2eY+>65 243_24BCR0R 3AW>@BP>@BP[ 1,2 R \ iPY+^4)F >Z[.),J ^ \ i0e ^HR \ 2
^fAC1,2t[j3ACQ \ ÂWîsh ij3 >Z[.)fG z S+/PR0e >ZR \ R \ 2m;(`- AP; >q^ ^hv4AWoZ2 >@B,Rh2431,^ ih /+Q+Y.AWRh2 ^ 3AWR \ 243"R \ AbBBP/+1S 243
T V
9 A ;P>@1/+1 Rf36>@AWo a ;+QPoZij3AWR>ZijB *I]b243AW[C2 e( ijApo 0X 3Ap^ \ O+A h 2
oZ24BP[CR \ K ^Rh24QP^ M ] AWo@/P2 Q+A 5Ci5 K R>@1,2 Yij/PR M
T ]f] PB ijBP2 ]+)x&U >WV fU JCl G lf9
]+:) ]+T ]+)x&U >+T fU Jf] U l+T
fl ]f] BPijBP2 ]+)xUbU+T U >f] G TG
]+:) ]+T ]+)xUbfU 9 U >CU U l
Gf]f] BPijBP2 ]+)xUbU+T U >f] G TG
]+:) ]+T ]+)xUbfU 9 CU Uf] U T
g CA SPoZ2Q9 re9 X ,.-0/;Rf3hAW>@BP2 Y 36ijS iCReR2 ^R2 YijB T](]f]3ACBPY+ij1oq5[C24BP243hAWR2 Y 36iPij1,^4z+e0>ZR \ ] AC365P>@BP[
1!A;P>c1,/+1 R 63 >@AWo"oZ24BP[CR \ ^4zNACBPYce0>ZR \ AbBPYce0>ZR \ ij/PR?2A;=QPoZij3ApR>ZijB%)
>q[P/+3623O+r,N X "*
' fR 3Ap>cBP2 Y36ijS iCRH>@BAR65+QP>Zv4AWo%Rf3AW>@BP>@BP[!24Bb]j>@36ijB+124BCR4) (P +*' g\ 2^fAC12 36iPS+iCR
>cBABPiW]C2 oN24BC]P>@36ijB+1,24BbRv iPBP^>Z^R>@BP[!ih"v >@36v4/Po@AC3tijSPD62 v R^>@BA36iPij1'l+):GR>@1,2 ^Ho@AC36[C243 R \ ACB,>ZR \ AW^
^2 24BY./+36>@BP[Rf3AW>@BP>@BP[.)
ih R 36>@AWoZ^4)*^v4ACBS 2^2 24B%zNR \ 2e9 X,.-0/ Rf3AW>@BP2 Y 3_ijS+iCRh^ \ A ]C2ev ijBb]C2436[C2 YRi A,îboc/PRh>qiPB>@B e 2 oZo
/+BPY+243eA 1,>ZoZoZ>ZijB /+Q Y.AWR2 ^4zNv ij1!Q+Ab362 YRi!iW]C243l 1,>ZoZoZ>ZijB hsij3 ,K0L M?ACBPYV81,>ZoZoZ>ZijB hsij30^hRfACBPY.AC36Y
,.- oq24Ab3BP>@BP[.)
g iQ+/PRR \ >Z^,>@B Q+2436^ Q+2 v R>Z]C2Cz R \ 2!v ijBbRij/+3e1!ACQ>cB >Z[.)J3_24Q+362 ^24BbRêR \ 2 A ]C243Ap[C2!Q+A45bi5"^
362 v 2 >Z]C2 Y Sb5R \ 2!9 X ,.-0/ Rf3AW>@BP2 Y 36iPS+iCRÂ h R243,Tb:) G1,>ZoZoq>ZijB;AbQ+QPoq>Zv4AWRh>qiPBP^!ih R \ 2 /+Q+Y.AWRh2 3/PoZ2
R \ 2![j3AbQ \ ^^ h ij3R \ 2!iCR \ 2430R eHi1,2 R \ iPY+ÂC3_2 BPiCRe^ \ i eIBAW^JR \ 2 5 AC362,]j>@36R /+AWoZoq5 SPocAbB+y>s)w2C) BPi
^RfACBPY.AC3_Q 5 9v ij1,SP>@B+AWR>ZijBP^0i hs!AbBPc
Y ,.- oZ24AC3BP>@BP[ R AC/P[ \ R36ijS+ibR^4zjAbBPY ijBPoZQ Y L7hsij3 ,K0L M z \ A4]b2IoZ2 Y
Ri A,36ijS iCR e \ >Zv \ A ]C243AW[b2 ^ AQ+A45bi 5 [j3_24AWR243tR \ ACY B ]+) fU G4e0>ZR \ >@B Tb:) G,1,>ZoqoZ>ZijB/+Q+Y.ApR2 ^4)
IO ( 9;NO9;= 9N=D

OPi.ze \ AWR >Z^ R \ 20S+2 ^hR36ijS+ibR v ijBCR 36iCoZoZ243 R \ AWR v4ACB,S+2IAWv \ >Z2 ]C2 Ye0>ZR \ v ijB+BP2 v Rh>qiPBP>q^hR 362 >@B(hsij36v 241,24BbR
1,2 R \ iPY+^ g ACSPoZ2Q9^f/+1!1!AC3_>q^h2 êR \ 2362 ^ /PoqRh^^h ij3A36ijS+ibReRf3AW>@BP2 Y /P^>@BP[ijB(- oZ>@BP2!9 X,.-0/ e0>ZR \
A,Rf3AW>@BP>@BP[3AWR26ihl+):] z.ACBPY L2 =/+Apo Ri ]+)xlfG+zPijBYG(]+z:]f]f],3AbBPY+ij1,oZ5[C24BP243AWRh2 Y36ijiP1,^4);< >ZR \

ijBPoZ5 T ]f]^R24QPêA ] AW>Zo@ACSPoZ2!Ri3624AWv \ R \ 2[CijApo Q+iC^h>qRh>qiPB%z"R \ 2,36ijS+ibRh AW>ZoZ^l iahR \ 2,R>@1,2Y./P2,Ri

TG
R>@1,2 - ij/PR^4) g\ 2,Bj/+1,S+243?ihR>@1,2 - iP/PR^0Y.36ijQP^RiTb):G >6h lf]f]^R24QRf36>@AWoZêAC362,AWoZoqieH2 Y"z#e0>ZR \ BPi
3624AWo.>@1!Q+3_i ]b241,24BCR hsij3ApoqoZie0>@BP[!>@BPv43624AW^h2 YBj/+1,S+243_^0ih"^R24QP^4) <?iCR2R \ AWRtR \ 2I^h5j^Rh241 Q 243!hsij31,^

S+2 RR24^3 e?>qR \ A^ 1!AWoZo Ab1,ij/+BbRIihH2A;=QPoZij3hAWR>ZijB%z%2 ]C24B e \ 24B h /PoZoZ5 R 3AW>@BP2 Y") g\ >Zê>Z^S 2 v4AC/P^2!R \ 2
oq>@1,>ZR2 Y^24BPîj365e>@(B hsij31!ApR>ZijB,A4]WAW>Zo@ACSPoZ20RiR \ 2 36iPS+iCR 1!ACyW2 ^ R \ 20Q+36ijSPoZ241 BPijB(- 9 AC3yWiW]P>@ACB%z4^hiI>@B
îj1,20^h>qR /+AWR>ZijBP^H>ZR >Z^0BPiCRtv oq24Ab!3 - v4/PmR e \ >Zv \ AWv R>ZijBRieRfACyp2C) / h R \ 2I3_ijS+iCRH1!ACyW2 ^HAeY+2 R243h1,>@BP>q^hR>Zv
Y+2 v >q^h>qiPB%+z e \ >Zv \ o@AWR243HoZ24AWY+^>ZR S+Apv yRiR \ 2e^fAC1,2J^RfAWRh2Czj>Z^R e0>ZoZo [C2 R?v4AC/P[ \ R0>@B AoqiPijQ%)H* oZ>qRhRoZ2
2A;=QPoZij3ApR>ZijB%z \ i eH2 ]C243 zPv4ACB \ 2 o@QR \ 2,36ijS+ibR iP/PR0i h ^f/Pv \ ^>ZRf/+AWR>ZijBPêSb5 [C>Z]P>@BP[8A!Q+3_ijS+ACSP>ZoZ>qR65i h
Q+24!3 hsij31,>@BP[A,Y+6> 5"243624BCRAWv R>ZijB%)
>Z[.) O^ \ i e0^,R \ 2Rf3ApD`2 v RiP365RfACyW24B SC5 R \ 236iPS+iCR,>@B ijB;AR65=QP>Zv4AWoI3hACBPY+ij1,oZ5 [C24BP243AWR2 Y
Rf3AW>@BP>@BP[24Bb]j>@36iPB+1,24BCR z ACBPY >@B ABPiW]C2 o v >c3_v oq2 eHij36oZY %24Bb]j>@36ijB+124BCR h ij^3 e \ >qv \ >ZR3_2 v 2 >q]b2 Y BPi
AWY+Y+>ZR>ZijB+AWo Rf3AW>@BP>@BP[.z+Y+241,iPBP^Rf3AWRh>cBP[R \ 2e[C24BP243AWoZ>ZR65i h >qRh^0v ijBbRf36iCo"Q+iboq>Zv 5C)

S
3

S

Y L J
]
E IOPQD 58.O6

g\ 2 v ijBbRij/+3QPoZiCRîah >Z[.)HV ACBPY G ^ \ ie \ ie R \ 2 v \ iC>Zv 2 ihR 3AW>@BP>@BP[ 3AWRh2 ACBPY g d^-

oq24Ab3BP>@BP[83ApR72 L v4ACB2 5"2 v R0R \ 2e^f/+SP^2 +/P24BCR^f/Pv v 2 ^^0iP3 h Ap>qo@/+362Jih R \ 2e^h5j^Rh241 Ri,v ijBb]C2436[C2eRi!A
^f/Pv v 2 ^^ h /Poîboc/PRh>qiPB%) OPij1,2,]WAWo@/P2 ^^h>c1QPoq5 362 ^f/PoZR>@B]C243_5ôZie v ijBb]C2436[C24BPv 2!Rh>c12 ^ iCR \ 2436ê>@B
v ij1!QPoZ2 R.2 h AW>Zo@/+362RieoZ24AC3B!A^f/Pv v 2 ^^ h /Po Q iCoZ>Zv 5C) g \ >Z^t>q^0S 2 v4AC/P^2ih R \ 2?[C24BP243AWoZ>Z^fAWR>ZijBQ+36ijQ 2436R65
ih BP24/+3AWo BP2 R e iP3yC^ 'z e \ >qv \ 1,24ACBPêR \ ApR0>@(B h ij3h1!AWR>ZijBv4ACBS 2 h ij3_[CiCRR24B+AW^êH2 oZo AWôZ24AC3BbR4)$/@h
R \ 2 Q+Ab3AC1,2 R243_^v \ iC^24BY./+36>@BP[!Rf3AW>@BP>@BP[!AC362e/+BP^f/P>ZRfACSPoZ2Cz+R \ 2e36ijS iCmR e0>ZoZo hsij36[b2 R>@(B h iP31!AWR>ZijBAW^
h AW^hRIAW^0>ZRoq24Ab3BPÎ>ZR4z%ACBPYî S+2,/+B+ACSPoZ2,Ri!v ijBb]C2436[C2eiPBA,^f/Pv v 2 ^h^ h /Po îCo@/PR>ZijB%) .g \ >q^>Z^ e \ 5BPi
Q+36ijai h ^ 5C2 R A2 ;P>q^hR 362 [jAb36Y+>@BP[eR \ 2Hv ijBC]b2436[C24BPv 20i h,.- oZ24AC3hBP>cBP[,ij3 g ^d -ÀWoZ[Cij36>ZR \ 1,^ >@B,v ijB+BP2 v R>ZijBP>Z^R
^5j^hR241,^4)
XHijBP^h 2 =/P24BbRoZ5CzW>ZR >q^ Y+2 ^>@3ACSPoZ20Rhi /P^2HRf3hAW>@BP>cBP[e1,2 R \ iPY+^ R \ AWR AC362oZ2 ^^ ^h24BP^>ZR>Z]C20Ri0R \ 2 v \ iC>Zv 2
i h.Rf3AW>@BP>@BP[Q+AC3Ab1,2 R2436^4z Ri A ]Cib>qY \ A ]j>@BP[Ri Q 24!3 hsij31 3624Q+24AWRh2 YA2 ;+Q+243_>c124BCR^ Rhi02 ^RfACSPoZ>Z^ \ e \ >qv \
] AWo@/P2 7^ eHij3yS+2 ^hR4) g \ 2 362 ^ /PoqRh^Q+362 ^24BbR2 Y >@BR \ 2!o@AW^hRe^2 v R>ZijB^f/P[b[C2 ^ReR \ AW R !&-)8"e/+Q+Y.ApR2 ^
ACBPYR \ 2!/P^2Ji h9 X ,.-0/ ij3 ,K0L M z.AW^0ijQ+Q iC^2 YRi^RfACBPY.Ab36c Y ,.- oZ24AC3BP>@BP[ /+Q Y.AWR2 ^4z \ 2 o@Q 362 Y./Pv 2
R \ >ZÎ^24BP^h>qRh>q]P>ZR65C)
g \ 2] AWo@/P2Ji h /P^2 YY e0AW%^ H;j2 YR \ 3_ij/P[ \ ij/PRHR \ 2A2 ;=Q 2436>@1,24BbRÎQ+362 ^24BbR2 YAWR ]+) UCU+N) < >ZR \ BPi
Y+>q^hv ij/+BCRh>cBP[Nz.R \ 2,h 36ijS iCRHv4ACBAC3h36>Z]C2AWR0îboc/PRh>qiPBPÎR \ ApR 3_24ACQ \ >Z[ \ H B+AWo Q+A 5Ci 5"^4z+S+/PR0Y+i BPibRI/P^2
2 v >Z24BCR,Rf3ApD`2 v Rhij36>Z2 E ^ KsAbBPY \ 24BPv 2R \ 2 36iPS+iCRJ>q^,i hsR24BRh>c12 BY - ij/PR M ) g i>ZoZo@/P^Rf3ApR2R \ >Z^4z >Z[.) >
^ \ i e0^HR \ 2eRf3AW>@BP>@BP[v4/+36]C2 m^ hsij3tR e i,36ijS iCRÎ/P^>@BP[!9 X ,.-0/ e0>ZR \ ACBPY Y e?>qR \ ij/PR0Y+>Z^v ij/+BbR>@BP[.)t*I^
v4ACB S 2^2 24B%ztR \ 28/+BPY+>Z^v iP/+BCR2 Y 36ijS iCR,Y+ij2 ^,v ijBP^>ZY+243AbSPoq_ 5 eHij3_^2Cz 2 ^fQ+2 v >@AWoZoZ5 >@B R \ 2A ]C243Ap[C2
Bj/+1,S+243?i h^Rh24QP^0RfACyW24BQ+243?Rf36>@AWoszNY+2 ^fQP>ZR2,R \ ^2 h Apv RIR \ ApR0R \ 24362J>qîjBPoZ5AT Y+6> 5"243624BPv 2,>cBR \ 2
/+Q+Y.AWR2 êS+2 >@BP[1!AWY+2AWR?24AWv \ Rh>c12^R24Q%)

>@B+AWoZoZ5Czj^hij1,2HR2 ^R^ \ A ]C2H^ \ i eIBJR \ ApR R \ 2?v ijBC]b2436[C24BPv 2Hi h R \ 2IBP24/+3AWo.BP2 R eHij3yb^ 3_2 oq>Z2 ^ \ 24A f] -
>qoZ5 ijBR \ 2A2 ;+QPoqiP3AWR>ZijB /P^2 Y;AWRe24AWv \ ^R AW[C2!i h0oZ24AC3BP>@BP[.H ) /@hI>ZR,>Z^RhijioZi e 24AC36oZ5ijB R \ 24B R \ 2
36ijS+ibR0v4ACB+BPiC.R H BPY>c1Q+36iW]C2 Y Q iCoZ>qv >Z2 ^4iz e \ >ZoZ^RI6> hH>qR>ZÎRhiji \ >Z[ \ AWRA!o@AWR243?^RfAp[C2eR \ 24BR \ 23AC(B -
Y+ij1!BP2 ^^0>@BbR24!3 hs24362 ^ e0>ZR \ R \ ^2 H BP2Rf/+BP>@BP[ 36 2 +/P>@362 YRhi \ A ]C23_2 oq>@ACSPoZ2,Q+iCoZ>Zv >Z2 ^R \ AWRH^f/Pv v 2 ^h^ h /PoZoZ5
oq24ApYRiR \ 2[CijApo G) < \ 24B/P^>@BP[ AktiCoZR
41!AbB+B!Y+>Z^Rf3_>cS+/PRh>qiPB%z.R \ 243_2 h ij3_2CzCR \ 2e3AWR2?i h"v ijBC]b2436[C24BPv 2
>q^Y+>@362 v RoZ5oZ>@B+yW2 Y e0>ZR \ R \ 23hAWR2ei h 3_2 Y./Pv R>ZijBi h )
XHoZ24AC36oZ5JR \ 24362 hsij362Cz R \ 2v \ iC>Zv 20i h.]WAWo@/P2 =^ h ij3 R \ 2 \ 24/+36>Z^R>ZvIQ+AC3Ab1,2 R2436^ /P^h2 Y>@BJv ijB+BP2 v R>ZijBP>Z^R
,.- oq24Ab3BP>@BP[ >Z^Jv436>ZR>Zv4AWo>@Bij3_Y+243eRi[j/+Ab3ACBbR2 2^f/Pv v 2 ^h^ h /Potv ijBb]C2436[C24BPv 2C) g \ 3h/+B ACBPY OPv \ e0AC3_R
K6T4UCfU 9(M Q+3_i ]P>ZY+2 Yoq>@1,>ZR^ h iP3 S+AW^2 YijBR \ 2Rf36>@AWo oq24BP[bR \ ACBPY BP/+1,S+243i hAWv R>ZijBPÂ ] Ap>qo@ACSPoZ2,Ri
A^h5j^Rh241z+AW^^f/+1,>@BP[!iPBP2 - ^R24h Y Q ,.- oZ24AC3BP>@BP[>ZÎS+2 >@BP[/P^2 Y"z.S+/PRI1ij362[C24BP243AWo"362 ^f/PoZRÂC362AW^H5C2 R
/+B+A4]WAW>Zo@ACSPoZ2C)
TJ
g m g
1 1
0.8 0.8
Average reward
Average reward
0.6 0.6
0.4 0.4
0.2 0.2
0 0
0 10 20 30 40 50 0 10 20 30 40 50
Trials (thousands) Trials (thousands)
>q[P/+362p>+re(3hACQ \ ^^ \ ie0>@BP[R \ 22 52 v Reiah R \ 2Y+>Z^v iP/+BCR4h AWv Rij3 )ktiCR \ 36ijS+ibR^2eH24362,Rf3AW>@BP2 Y
/P^>@BP[!9 X,.-0/ e0>ZR \ .' TC):]AbBPY L ]+):G+) g\ 2IY+ibRR2 Yoq>@BP2 ^^ \ ih e R \ 2BPiP31!AWoZ>Z^2 YBP/+1S 243tih
^R24QPÎQ 2430Rf3_>cApo )
2 /= D %+ 7E=Q

g\ 2 3_2 ^f/PoZR^Hih R \ 2R2 ^R^%h iP3/P^>@BP[,ijB(- oZ>@BP2/+Q+Y.ApR>@BP[!v ij1!Q+AC3_2 Y Ri,S+AWv yfe0AC36Y3624QPo@A45AC3620>@BbR243!-
2 ^R>@BP[ +ij(B - oZ>@BP2 /+Q Y.AWR>@BP[ v iPBP^>Z^R24BbRoZ5Q+243!hsij31,^1/Pv \ 1,ij3_2e^f/Pv v 2 ^^ h /PoZoq5iW]C243HA7e0>ZY+2433ACBP[C2
ih Rf3Ap>cBP>@BP[Q+AC3AC12 R2436^ h iP3 Apoqo=9/+Q Y.AWR2,3/PoZ2 ^4) g\ >Z^0>Z^(=/P>ZR2e^f/+3hQ+36>Z^>@BP[.z Ap^ S+Apv y(e0AC36Y3624QPo@A45
\ AW^!R \ 2S 24BP2 H.Ri h \ A ]P>@BP[ AWoZo0^RfApR2 -ÀWv R>ZijB;Q+AW>@36^^RiP362 NY hsij3,R \ 2Rf36>@AWosz0ACBPY S+2 >@BP[ /+Q Y.AWR2 Y
/P^>@BP[ ^f/+Q 2436]P>q^h2 Y oZ24AC3hBP>cBP[S+AW^2 YijBR \ ^2 H B+AWo 36A2 e0AC36Y") b?i e 2 ]b243 zP>q.R e iP/PoqY^2 241 R \ ApR0ij(B - oZ>@BP2
oq24Ab3BP>@BP[ \ AW^tR \ 20AWY+] ACBbRfAW[b2 R \ AWR R \ 2 2 oZ>Z[C>@SP>Zo@>ZR>@2 ÎApv R >cBA^>@1,>ZocAb3 e0A 5eRiARH !H ' H R2431 z
Q+36iW]j>ZY+>@BP[ /+Q Y.AWR2 ^R \ AWR 3_2 Y./Pv 2!R \ 2,24336ij3Rhi
2436iNz">@BP^R24AWYi hHv ijBb]C2436[C>@BP[AW^5+1!QPRibR>Zv4AWoZoq5>@B
Q+36ijQ+iP36R>ZijBRi!R \ 21,24ACB2^ =/+AC3_2 Y24336ij3 )tm36iW]P>qY+>@BP[A!1,ij124BCRf/+1'R243h1 h iP3HR \ 2/+Q+Y.ApR2 ^ Q 24!3 -
h ij3h1,2 YY./+36>@BP[S+Apv (y e0AC36Y 3_24QPocA 5v ij/PoZY \ 2 o@Q AWv \ >Z2 ]C2R \ 2!^fAC122 5"2 v QR KsApê>qRJY+ij2 ê>@B BPij31!Apo
^f/+Q+2436]P>Z^2 Y oZ24AC3BP>@BP[ R AW^fyb^ M zS+/PR Apoq^hi >@BCRf3_ijY./Pv 2 ÂCBPiCR \ 243 \ 24/+36>Z^Rh>qv Q+AC3hAC1,2 R243JR \ AW7R eHij/PoZY
BP2 2 Y^2 RRh>cBP[Y./+36>@BP[ Rf3hAW>@BP>cBP[N)
KJ ;9 587>< 09; 9
A7l+O E=Q
(
g\ 283_2 ^f/PoZR^,^ \ ie R \ 283_2 ocApR>Z]C28Q 243!hsij31!ACBPv 2ih0R \ 2E9Y+>65243_24BCR4hsij31,êihI/+Q Y.AWR23/PoZ2C) / R>Z^
>c1Q+ij36R ACBCRtRi3_241,241,S+243 R \ AWR AWoZo+9eih"R \ 2 ^2/+Q+Y.AWR2e3/PoZ2 ^0AC36202A;+AWv RoZ5!2 +/P>Z] AWoZ24BbRe \ 24B1+4'8C
[j362 2 Y+5 Apv R>ZijBPÂC3_2RfACyp24BY./+3_>cBP[ARf36>@AWos) g\ 2!Y+>65"243624BPv 2!>@BR \ 2!/+Q+Y.ApR2 îPv v4/+36êijBPoZ5 e \ 24B
2A;=QPoZij3ApRij365AWv R>ZijBPÎAb362eRfACyW24B%)
\?^>@BP[^RfAbBPY.AC36Q
Y ,.- oZ24AC3BP>@BP
[ e?>qR \ L!^2 RRhi
24362i hsij3BPiP(B -`Q+iCoZ>Zv 58Apv R>ZijBP^4fz eH2 >Z[ \ R2 oZ>Z[C>@SP>Zo@>ZR>@2 ^
AC362!ijBPoZ5;AWoZoZi eH2 Y Ri S+/P>ZoZY /+Q e \ 24B R \ 2 36iPS+iCRJRfACyW2 ^,A^h 2 =/P24BPv 2i h0[j3_2 2 Y+5 Q+iCoZ>Zv 5;AWv R>ZijBP^
R \ 2 5 AC362
2436ij2 Y Y e \ 24B ACBA2 ;=QPoZij3ApRij365 Apv R>ZijB,>Z^0RfACyp24B%) .g \ >q^H^RhijQP^R \ 2e362 ^f/PoZR^Hi h"A2 ;+QPoZij3AWRiP365
AWv R>ZijBP^ h 36ij1 S 2 >@BP[ ^2 24BjSb5,24AC36oZ>Z243AWv R>ZijBP^ zjS+/PR Apoq^hi1,24ACBR \ AWR ^RfApR2 ^ ^2 2IAv ijBCRh>cBP/+AWo+iW]C24!3 -
2 ^R>@1!AWR>ZijBi h R \ 2Q+A 5Ci 5"^0A ] AW>Zo@ACSPoZ2Cz"AW^0R \ 2 58AC3_2AWo e0A 5j^HRf3Ap>cBP2 YijBR \ 21!A ;P>@1,/+1 +4%A&k# %A
,.- ] AWo@/P2 ApR24Apv \ ^R24I Q K g \ 3/+B ACBPY OPv \ e0AC3_R
T4UC(U 9fM c ) b?i e 2 ]b243 z >cB;Av ijB+BP2 v Rh>qiPBP>q^hR!^5P^R241z
[C24BP243AWoZ>Z^fAWR>ZijBiPv v4/+36^4(z e \ >Zv \ 124ACBP^R \ ApRR \ 202 5"2 v R^Hi h S+AWYA2 ;=QPoZij3ApRij365!AWv R>ZijBPm^ e0>ZoZo"S+2^2 24B
SC5BP24AC3hSC5^R AWR2 ^02 ]C24B6> hL>ZÎ^2 RRi
243_i RiRf365Ri8Q+3_2 ]C24BCR?R \ >Z^4z.^hi R \ >Z^>Zêi h oZ>@1,>ZR2 Y ]WAWo@/P2Cz
ACBPY^>@1!QPoZ53_2 ^f/PoZRê>@BR \ 2>@(B hsij31!ApR>ZijBoq24Ab3BCRSb5[CiPijYA2 ;+QPoZij3AWRiP365AWv R>ZijBP^S 2 >cBP[/P^2 YoZ2 ^^
k@sJA<KM<DE$CpmA@;JAI E$C;@;n KM<6J ': GS>7<DEP@;>AIM2C@7g

TO
2 52 v Rh>q]b2 oq5C) g.\ 2iW]C243AWoZo%2 5"2 v RI>Z^R \ AWR0^hRfACBPY.AC36Yc,.- oZ24AC3hBP>cBP[v iPBC]C243_[C2 ^0oZ2 ^^(=/P>Zv yCoZ5Cz"ACBPYi ]b243
A^f1!AWoZoZ2433AbBP[C2!ihHRf3AW>@BP>@BP[ Q+AC3hAC1,2 R2436^K 2 ^fQ 2 v >@AWoZoq5;BPiCR>Zv4ACSPoZ2 e0>ZR \ S+AWv y(e Ab36Y 3624QPo@A 5MeR \ ACB
9.X ,.-0/ ijm3 ,K!LMHR65+Q+2/+Q Y.AWR2 ^4)
,K!LM >q^t>@B2 5"2 v RAv ij1,SP>@B+AWR>ZijBi h.^RfACBPY.Ab36Y,.- oq24Ab3BP>@BP[AbBPY9 X,.-0/ )fb0ieH2 ]C243 zC/+Q Y.AWR>@BP[
R \ 2!o@AW^RJ[j362 2 Y+5 Q+3_2 Y+>qv Rh>qiPB;S+AW^2 Y ijBR \ 2!BPA2 ;jR,Q+362 Y+>Zv R>ZijB%z Y+2 ^fQP>ZR2R \ 72 h AWv ReR \ AWReR \ 2!v ij3362 -
^fQ+ijBPY+>@BP[[j362 2 Y+5AWv R>ZijB e0AW^BPibRQ+24!3 hsij31,2 Y"z >ZÂv ij(B h /P^>@BP[v ijBPv 24QPR z S+/PRAbQ+Q+24AC36^JRY i eHij3y
e 2 oZoH>@B Q+3hAWv R>Zv 2CzAWoZR \ ij/P[ \ >ZR>Z^ BPibRv oZ24AC43 e \ AWRJR \ 28Q+3_2 Y+>qv Rh>qiPBP^ 1!AWY+2 Sb5 >@BbR2431,2 Y+>@AWRY2 ,.-
h /+BPv R>ZijBP^HAWv Rf/+AWoZoZ5!3624Q+362 ^24BbR4) g \ 203/PoZ2>q^ ^hoq>Z[ \ RoZ5 \ AC3_Y+243 Rie>@1!QPoZ241,24BCRtR \ ACBJ2 >ZR \ 243 ^hRfACBPY.AC36Y
,.- oq24Ab3BP>@BP[ ij3t9 X ,.-0/ zNACBPY>@BR \ 2 ^2eA2 ;+Q+2436>@1,24BbRêACQ+Q+24AC3_^0Ri,i 5"243BPi ApY+] ACBbRfAW[C2iW]C243H/P^>@BP[
R \ 2e9 X ,.-0/ R65+Q+2ei h/+Q+Y.ApR2 ^4)
< >ZR \ 9iPY+6> H.2 Y XHijB+BP2 v R>ZijBP>Z^Q R ,.- oZ24AC3hBP>cBP[NzR \ 2362 ^f/PoZR^,i h0A2 ;+QPoZij3AWRiP365 AWv R>ZijBP^!AC3_2!^2 24B
SC524AC36oZ>Z243e^hRfAWR2 ^4iz e \ >Zv \ R \ 24362 hsij362!oZ24AC3BR \ 2!A2 ;+Q+2 v R2 Y;Q+A 5Ci 5 A ] Ap>qo@ACSPoZ2>@B24AWv \ ^R AWR2,[C>Z]C24B
R \ 2!v4/+33624BbR,oZ2 ]C2 oi h0A2 ;+QPoZij3AWR>ZijB%)=a 5"2 v R>Z]C2 oZ5Cz R \ 2 3_>q^ yi h AQ+AC36R>Zv4/Po@AC3Jv ij/+36^h2 i hIAWv Rh>qiPBP^>Z^
S+/P>qoZR,>@B%z AWê6> h ^hoq>Z[ \ RA2 ;+QPoZij3AWRh>qiPBv4ACBoZ24AWY RiY+>Z^fAW^Rh243 z%R \ 24BR \ 2Q+362 Y+>Zv R2 Y Q+A 5Cai 5.^ hsij3R \ AWR
^ 2 =/P24BPv 2i h AWv Rh>qiPBP3^ e0>ZoZo S 2!v ij3362 ^ Q+ijBPY+>@BP[CoZ5oZi eH243 z"/+BPoZ>@yW` 2 h iP3^RfACBPY.AC3_Y ,.- oZ24AC3hBP>cBP[Nkz e \ >qv \
AW>@1,^0ijBPoZ5Ri!oZ24AC3hBR \ 2Y+2 R243h1,>@BP>q^hR>Zv[P362 2 Y+58Q iCoZ>Zv 5C)
I2 3

S
/ B R \ >Z^83_24Q+ij36R zme 2 \ A ]C2Q+362 ^24BbR2 Y 1,2 R \ iPY+^hsij3,iPB(- oq>@BP2 /+Q Y.AWR>@BP[ /P^>@BP[ BP24/+3AWoeBP2 R eHij3yb^
h ij3?362 >@B(h ij3_v 241,24BCReoZ24AC3hBP>cBP[N) g\ 2 ^2,1,2 R \ iPY+^ \ A ]C2,S+2 24BY+241,ijBP^hRf3AWR2 YijBA3624AWoZ>Z^R>Zv!1,ijSP>ZoZ2
36ijS+ibRQ+36ijSPoZ241ztACBPY \ A ]C2^ \ iIe BR \ AWRJijB(- oZ>@BP2oq24Ab3BP>@BP[ >Z^>@B h AWv RA1,ij3622 v >Z24BCR!1,2 R \ iPY
ih Q+243!sh ij31>cBP[/+Q+Y.AWRh2 êR \ ACBS+AWv yf0e AC36Y3624QPo@A 5 1,2 R \ iPY+^QK)/%>@B;T UCUClfM z >cBR2431,êS+iCR \ iah^Rij3!-
AW[C2362 +/P>@36241,24BbR^ ACBPY ^24BP^>ZR>Z]j>ZR65;RiRf3Ap>cBP>@BP[;Q+AC3AC12 R2436^4)_ed B(- oZ>@BP2 oZ24AC3hBP>cBP[ Apoq^hi \ AW^!R \ 2
AWY+] ACBbRfAW[b2R \ AWR>ZRev ij/PoZY S+2!/P^2 Y >cBv ijBbR>@BP/Pij/PôZ5ijQ+243ApR>@BP[^5j^hR241,^ê \ 24362!BPi24BPYih Rf36>@AWo
v ijBPY+>ZR>ZijBPÎiPv v4/+3 )
/ B,AWY+Y+>ZR>ZijBRiIR \ >q^ zC^2 ]C243hAWojY+>6"
5 243624BbR/+Q+Y.AWRh2 3h/Poq2 ^ \ A ]C2 S 2 24B,v ijBP^>ZY+24362 Y"zb>cBPv o@/PY+>@BP[ ABP2Ae
h ij3h1 K 9 X,.-0/M S+Ap^2 Y81ij362H^Rf36iPBP[CoZ5iPB g d^- oZ24AC3BP>@BP[,R \ ACB,R \ 2 BPij3h1!AWo,.- oZ24AC3BP>@BP[!2 +/+AWR>ZijB%)
g\ >Z^ \ Ap^S+2 24B^ \ iIe B,Riev ijBb]C2436[b2 RhiîCo@/PR>ZijBP^t>@B!v ijBP^h>qY+243hACSPoZ57h 2AHe 243t/+Q+Y.AWR2 ^tR \ ACB,^hRfACBPY.AC36Y
,.- oq24Ab3BP>@BP[ iP3!R \ 2 ,K!LM /+Q Y.AWR2 3/PoZ2CzeACBPY Ri S 21ij362 36ijS+/P^RRhi R \ 2v \ iC>Zv 2 i,h Rf3AW>@BP>@BP[
Q+AC3AC1,2 Rh2436^4);/ Bv ij1!Q+AC3_>q^hijB?e >qR \ ,K!L M zb9 X,.-0/ w^HQ+243!sh ij31!AbBPv 2I>@1!Q+36iW]C24124BCRt>q^BPiCRîJv oZ24AC3
v4/PR4z S+/PR^hR>Zoqo \ AW^,R \ 2 AWY+]WACBbRfAW[C2!ia?h Av ij1!Q+/PRfApR>ZijB+AWoZoZ5 ^>@1!QPoZ243!/+Q+Y.ApR283h/Poq2362 +/P>@36>@BP[ oZ2 ^^
^Rij3Ap[C2Cz+AW^me 2 oZo AW^0A,v oZ24AC36oZ5Y+2 H BP2 YRfAC36[b2 R ,.-)h /+BPv R>ZijB sh ij3HACBC5[C>Z]C24BQ+iCoZ>Zv 5C)
d."h v ij/+36^2CzbR \ 2e362 ^f/PoZR^ Ap^ Aev iP1!Q+AC36>ZîjBi"h Y+>:"5 243624BbR ,.- oZ24AC3hBP>cBP[/+Q+Y.AWR2e3/PoZ2 ^0AC362mh AC3ih 36ij1
v ijBPv o@/P^>Z]C2Cz AbBPYYh /+36R \ 243Hv ij1Q+AC3AWR>Z]C2^Rf/PY+>Z2 êBP2 2 YRi!S 21!AWY+2C) 0b iHe 2 ]C243 zPv4/+33624BbRoZ5 e iP3y,>Z^
S+2 >@BP[v4AC3h36>Z2 Y iP/PRIRiv ij1!Q+AC362JR \ 2!Q+243!sh ij31ACBPv 2ih R \ 2Y+>6"5 243624BbR /+Q Y.AWR2!3/PoZ2 êijBAY+>q^hv4362 R2
^RfAWR2 - ^ Q+AWv 2 Q+36ijSPoZ241z%e \ 24362Q+362 oZ>@1,>@B+AC365;362 ^f/PoZR^,^f/+Q+Q ij36R,R \ 2`h AWv RJR \ AWRJ9X.,.-0/ ACBPY ,K!LM
/+Q+Y.AWR2 êoZ24AWYRi`h AW^R243?v ijBb]C2436[C24BPv 2,R \ ACB^RfACBPY.Ab36Y$,.- oZ24AC3BP>@BP[_sK ACBPY h Ab3mh AW^R243 v ijBC]b2436[C24BPv 2
R \ ACB^>@1!QPoZ2,ijBP2 - ^R24QY,.- oZ24AC3hBP>cBP[B M )
73 K S P O
g\ ACB+yb^ 1,/P^Rt[Ci,RiX \ 24B g\ AC1 AbBPY8-H>Zv \ O+/PRRhijB4sh ij3tR \ 2 >@3 \ 2 o@Q(h /Po"v ij1!124BCR^HijBR \ >Z^mHe ij3yN)
g\ >Z^ He ij3y>q^.h /+BPY+2 Y Sb58A,[P3ACBbR h 36ij1 R \ 2OPv >Z24BPv 2AbBPYaBP[C>@BP2 2436>@BP[-H2 ^24Ab36v \ XHij/+BPv >Zos)
T>
R Y

Rm4
MS
2S
2
S RT
.ij3Hv iP1!QPoZ2 R24BP2 ^^4z+R \ 2v4AWoZv4/Po@AWR>ZijBi_h 8; \ACBPY \ 24BPv 2R \ 2e2 oZ>Z[C>@SP>Zo@>ZR65 R 3AWv 2 ^0>Z^0[C>Z]C24B \ 24362C)
< 2,Y+2 H BP2 ABP24/+3ApoBP2 R0eHij3hyAWêS+2 >@BP[ Av iCoZoZ2 v R>ZijB iah >@BCRh2436v ijB+BP2 v R2 Y '"" Ab33ACBP[C2 Y>@B
&C z=e \ >Zv \ eH2!o@ACS+2 o ! ! +11Z1Bh 36ij1#R \ 2 ij/PR Q+/PRo@A 5C243eRiR \ 2>cB+Q+/PRocA 5C243 ) * e 2 >Z[ \ RiPB A
v ijB+BP2 v R>ZijB h 36ij1 ocA 5C243
Ri>Z^0o@ACS+2 oZoZ2 Y & )atAWv \ /+BP>qReQ+2430h ij3h1,^0R \ 2^sh iCoZoZi ?e >cBP[`h /+BPv R>ZijB%z
d K &)M 6K TJ(M
6K TO(M
e \ 24362 >Z^0R \ 2iP/PRfQ+/PRmh 36ij1'o@A45b243ACBPY K 16MH>ZÎA H?!&kA^h /+BPv R>ZijB%)
g\ 2eij/PR Q+/PR0[j3AWY+>Z24BbR \ >ZÎY+2 H BP2 Y e)x3 )wR4)"R \ 2eij/PRfQ+/PR0o@A 5C243meH2 >Z[ \ RÎAW^4z
" 5 K )M K6T>(M
"'&
e \ 24362 5 K 1:tM >Z^R \ 22H 36^hR0Y+>65 243_24BCR>@AWo ih R \ 2^h>q[P1,iC>ZYYh /+BPv R>ZijB%) g\ 24362 sh ij362Cz>h ij3 R \ 22H 36^hR \ >ZY+Y+24B
ocA 5C243 e 2 >Z[ \ R^4z R \ 2e[j3AWY+>Z24BbR0>ZÎ^h>c1QPoq5Cz
" 5 K AM & 5 K M e K6T4U(M
"'& e
g\ 2 ^2,] AWo@/P2 ÂC3_2AWY+Y+2 YRi!R \ 2v4/+33624BbR2 oq>Z[C>@SP>Zo@>ZR>@2 ^ )(24BP243AWoZoZ5Cz%R \ 243623He ij/PoZY S+2,ijBP2Jij/PRfQ+/PR
[j3AWY+>Z24BbR7h iP3,24AWv \ ij/PRfQ+/PRhzACBPY \ 24BPv 2e2 oZ>Z[C>@SP>Zoq> R>Z2 ^cHe ij/PoZY S 2362 =/P>@362 Y sh ij3J24AWv \ e 2 >Z[ \ R4z
îR \ ApR3e \ 24B 24AWv \ ij/PR Q+/PRwêR241!Q ij3AWotY+>65 243_24BPv 224336ij3 zG_6z AC336>Z]C2 Y"z R \ 2è 2 >Z[ \ R^Jv ij/PoZY S+2
.& e ^ d _ e
/+Q+Y.AWR2 YAWv v ij36Y+>@BP[Ri.z
sK lf](M
e \ 24362 e >Z^ R \ 22 oq>Z[C>@SP>ZoZ>@R65 ijB He 2 >q[ \ R & e e \ >Zv \ v ij3362 ^ Q+ijBPY+^Riij/PRfQ+/PRh)?b iHe 2 ]C243 z >@B
,.- oq24Ab3BP>@BP[.z"R \ 243_2>ZêijBPoZ5A!^>@BP[CoZ2!Rh241!Q+ij3hAWo"Y+>6"5 243624BPv 2!24336iP3e \ >Zv \ >qêv4Apoqv4/Po@AWRh2 YNe )x3 ) R4)0R \ 2
ij/PRfQ+/PR e \ >Zv \ Q+36iPY./Pv 2 Y R \ 2v4/+33624BbR8Q+362 Y+>Zv R>ZijB \) ?b 24BPv 2 iPBPoq5 ijBP2iP/PRfQ+/PR [P3AWY+>Z24BCR>Z^
v4AWoZv4/PocApR2 Y AWRH24AWv \ R>@1,2e^R24Q%zNACBPYijBPoZ5ijBP2e2 oZ>Z[C>@SP>qo >qR65>Z^ 3_2 =/P>@362 YQ+243.e 2 >Z[ \ R4)
J K L m
*Io@S+/P^4z.)"O%) _K T4U>+TM )$kF $*5Wk!&'5&A (9!0 ! "# z"k g a ktiPijyb^4z%9v4(3Aem-0?b >Zoqosz%v \ ACQ(-
R243J z=Q+Q%) TC9 U! TCO U+)
dA45PACB%zjm )_K T4UCUClfM ) g\ 2v ijBb]C2436[C24BPv 2ih g dQK!LMih ij3 [C24BP243AWo="L z #*k7 " j rW9 V.T">9(CJ l+)
%/ >@B%zk/ )6K T4UCUblf M )0OP2 o6hT- >@1!Q+3_i ]P>@BP[ 3624AWv R>Z]C2,AW[C24BbR^ S+Ap^2 YijB362 >@B(sh ij36v 241,24BbReoZ24AC3BP>@BP[.z QPocAbB+BP>cBP[
ACBPYR24Apv \ >@BP[.zk # *5"
N" j rNlCUf#9 >9bl+TC)
%/ >@B%z / ) _K T4UCUf9fM ) (9" X !#1H 7N"Xn! (9!502!"%$N "'& '5(& :;! 3 z m \ d R \ 2 ^>Z^4z
X0AC3hBP2 [C>Z292 oZoqiPB\BP>q]b2436^>ZR65Cz.m >ZRR^ S+/+36[ \ z.m 24B+BP^5PoZ] ACBP>@A+)
m 24BP[.z)N)ACBPY6< >ZoqoZ>@AC1,^ zI-e)*N) 6K T4UCUpV' M ) / BPv436241,24BbRfAWoI1,/PoZR>6- ^R24Q ,.- oZ24AC3BP>@BP[.z "B< ) XHi \ 24B
ACBPY Jb ) ?b >@36^ \ K 2 Y+^ M z #*k7 N%" ' !#%A" !X *5+9W7 *-, & k!7
.K!.Xn#0/ )132#4Czj9ij3_[jACB65eAC/(h 1ACB+B%z.<?2Ae kH3/+BP^S0e >Zv y.z%<Nz.\IOP*,z.Q+Q%)%lClf#J +lfC9 l+)
T4U
-0/+1,2 o \ CA 36R4z d!)ja0) z'b?>cBbRijB%z+()=a?)jACBPY < >ZoqoZ>@AC1,^ z -e).)K6T U>fJfM )k 9&8&k % 0 ' %A8 9!# %)
"jz iCos) TCzP9B/ g mt362 ^^4)
O+/PRRijB%zP-e)CO%) K6T4U>&>fM )/%24AC3BP>@BP[,RieQ+362 Y+>Zv R0SC5JR \ 2I1,2 R \ iPY+îah R241!Q ij3AWoPY+>65"243624BPv 2 ^4z # *5"
N%&%7k& r.#U jVCVN)
O+/PRRijB%zP-e)CO%) K6T4U >bfU M -) / 1!QPoZ241,24BbRfAWRh>qiPB!Y+2 RfAp>qoZ^ti h R \ 2 g dQK!LM Q+3_ijv 2 Y./+362 h ij3 R \ 20v4AW^2Hih%]C2 v Rij3
Q+362 Y+>Zv R>ZijBPÂCBPYS+Apv yPQ+36ijQ+AW[PAWR>ZijB%z %# *5"# ( +U!&% & )
1Wz.( g a /"AbS+ij3ApRij36>Z2 ^4)
g\ 3/+B%z O%) K_T4UCUW'V M )I*eB ACQ+Q+3_ijAWv \ Rhi oZ24AC3BP>@BP[ 36ijS iCReB+A4]P>Z[jAWR>ZijB%z 9!# , ++)+ .Y! X 7#% !X
,%7 " 5 (9!502!" 7ACn I H zP9 /+BP>qv \ z%(e2431!AbBC5C)
g\ 3/+B%z%O%) ACBPYOPv \ e0AC36R
Cz *k) K_T4UCfU 9fM K) /s^^ /P2 ê>cB/P^>@BP[ h /+BPv Rh>qiPB AbQ+Q+36i ;j>@1!AWRh>qiPB hsij30362 >@(B hsij36v 2 -
1,24BbReoZ24AC3BP>@BP[.Fz !# !X *5 !&'5 *-.K!&# "!7 F !A 4' HR?H 4#*!!Z+z /"A eI3624BPv 2
a3hSPocAb/+1z b?>qoZoZ^Y.AWoZ2Cz < .)
g\ 3/+B%z0O%)HkI2) K6T4UbUCfl M ) a v >Z24BCRA2 ;+QPoqiP3AWR>ZijB >@B 362 >@(B h ij3_v 241,24BCRoZ24AC3BP>@BP[. z %# *5"#%& ( +-!%
.F $N)".) 1 @)Cz.X0Ab3BP2 [C>Z2 - 92 oqoZijB\BP>Z]C2436^>ZR65C)
< AWRfyb>@BP^4z+Xe) .) bJ)+Xe H C@%A (9:;A@ zjm \ d R \ 2 ^>Z^4z 5>cBP[ w^0XHiCoZoZ2 [C2Cz
) K6T4U >C(U M ) N"YXn!
X0AC1,S+36>ZY+[C2\BP>Z]C2436^>ZR65Cz%\ 5,)
lf]

On-Line Q-Learning Using Connectionist Systems

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

On-Line Q-Learning Using Connectionist Systems

Uploaded by

Copyright:

Available Formats

CT ) H- 2 ^2 RAWoZo"2 oZ>q[b>cSP>Zo@>ZR> 2 ^4z g ]+)

>Z[j/+362l r deB(- oZ>@BP2!v ijB+BP2 v Rh>qiPBP>q^hR /+Q Y.AWR2Apoq[bij36>ZR \ 1 sh ij3,K!LM )

15deg Angle to goal

>Z[j/+36279+rG< \ AWR0R \ 236iPS+iCRIyPBPie0^0ACS+iP/PR0>ZR^I^ /+336ij/+BPY+>@BP[C^

L) $ 5 O+AC1!QPoZ2,Rf3Ap>cBP>@BP[v4/+36]b2 ^eRfACyW24BEh ij3?24AWv \ /+Q+Y.AWR2!3/PoZ2Cz%v ij3h362 ^fQ+iPBPY+>cBP[RiR \ 2]WAWo@/P2

Bj/+1,S+243?ih ^hR24QP^0RfACyp24B>cB24AWv \ Rf36>@AWo sK 1A;j>@1,/+1 Rf3_>cApo"oZ24BP[CR \ 0e AW^Ilf](]^hR24QP^ M )

SC5A $'ijB24AWv \ v ijBbRij/+3QPoqibR4) g \ 2,Y+iCRhR2 YoZ>@BP2!>Z^eR \ 2BPiP31!AWoZ>Z^2 Y A ]C243Ap[C2Bj/+1,S 243ih^R24QP^

ijBPoZ5 T ]f]^R24QP^eA ] AW>Zo@ACSPoZ2!Ri3624AWv \ R \ 2[CijApo Q+iC^h>qRh>qiPB%z"R \ 2,36ijS+ibRh AW>ZoZ^l iahR \ 2,R>@1,2Y./P2,Ri

E IOPQD 58.O6

You might also like

On-Line Q-Learning Using Connectionist Systems

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

On-Line Q-Learning Using Connectionist Systems

Uploaded by

Copyright:

Available Formats



CT ) H- 2 ^2 RAWoZo"2 oZ>q[b>cSP>Zo@>ZR> 2 ^4z  g  ]+)

>Z[j/+362l r deB(- oZ>@BP2!v ijB+BP2 v Rh>qiPBP>q^hR /+Q Y.AWR2Apoq[bij36>ZR \ 1 sh ij3,K!LM )

15deg Angle to goal

>Z[j/+36279+rG< \ AWR0R \ 236iPS+iCRIyPBPie0^0ACS+iP/PR0>ZR^I^ /+336ij/+BPY+>@BP[C^

L) $  5 O+AC1!QPoZ2,Rf3Ap>cBP>@BP[v4/+36]b2 ^eRfACyW24BEh ij3?24AWv \ /+Q+Y.AWR2!3/PoZ2Cz%v ij3h362 ^fQ+iPBPY+>cBP[RiR \ 2]WAWo@/P2

Bj/+1,S+243?ih ^hR24QP^0RfACyp24B>cB24AWv \ Rf36>@AWo sK 1A;j>@1,/+1 Rf3_>cApo"oZ24BP[CR \ 0e AW^Ilf](]^hR24QP^ M )

SC5 A $'ijB24AWv \ v ijBbRij/+3QPoqibR4) g \ 2,Y+iCRhR2 Y oZ>@BP2!>Z^eR \ 2BPiP31!AWoZ>Z^2 Y A ]C243Ap[C2Bj/+1,S 243ih^R24QP^

ijBPoZ5 T ]f]^R24QP^eA ] AW>Zo@ACSPoZ2!Ri3624AWv \ R \ 2[CijApo Q+iC^h>qRh>qiPB%z"R \ 2,36ijS+ibRh AW>ZoZ^l iahR \ 2,R>@1,2Y./P2,Ri

 E IOPQD 58 .O 6 

You might also like

CT ) H- 2 ^2 RAWoZo"2 oZ>q[b>cSP>Zo@>ZR> 2 ^4z g ]+)

>Z[j/+362l r deB(- oZ>@BP2!v ijB+BP2 v Rh>qiPBP>q^hR /+Q Y.AWR2Apoq[bij36>ZR \ 1 sh ij3,K!LM )

>Z[j/+36279+rG< \ AWR0R \ 236iPS+iCRIyPBPie0^0ACS+iP/PR0>ZR^I^ /+336ij/+BPY+>@BP[C^

L) $ 5 O+AC1!QPoZ2,Rf3Ap>cBP>@BP[v4/+36]b2 ^eRfACyW24BEh ij3?24AWv \ /+Q+Y.AWR2!3/PoZ2Cz%v ij3h362 ^fQ+iPBPY+>cBP[RiR \ 2]WAWo@/P2

Bj/+1,S+243?ih ^hR24QP^0RfACyp24B>cB24AWv \ Rf36>@AWo sK 1A;j>@1,/+1 Rf3_>cApo"oZ24BP[CR \ 0e AW^Ilf](]^hR24QP^ M )

SC5A $'ijB24AWv \ v ijBbRij/+3QPoqibR4) g \ 2,Y+iCRhR2 YoZ>@BP2!>Z^eR \ 2BPiP31!AWoZ>Z^2 Y A ]C243Ap[C2Bj/+1,S 243ih^R24QP^

ijBPoZ5 T ]f]^R24QP^eA ] AW>Zo@ACSPoZ2!Ri3624AWv \ R \ 2[CijApo Q+iC^h>qRh>qiPB%z"R \ 2,36ijS+ibRh AW>ZoZ^l iahR \ 2,R>@1,2Y./P2,Ri

E IOPQD 58.O6