6SHFWUDO&OXVWHULQJLQ(GXFDWLRQDO'DWD0LQLQJ


6+8%+(1'875,9(',=$&+$5<$3$5'26
*È%2516È5.g=<1(,/7+())(51$1
:RUFHVWHU3RO\WHFKQLF,QVWLWXWH8QLWHG6WDWHV
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

6SHFWUDO&OXVWHULQJLVDJUDSKWKHRUHWLFWHFKQLTXHIRUPHWULFPRGLILFDWLRQVXFKWKDWLWJLYHVDPXFKPRUHJOREDO
QRWLRQ RI VLPLODULW\ EHWZHHQ GDWD SRLQWV DV FRPSDUHG WR RWKHU FOXVWHULQJ PHWKRGV VXFK DV NPHDQV ,W WKXV
UHSUHVHQWV GDWD LQ VXFK D ZD\ WKDW LW LV HDVLHU WR ILQG PHDQLQJIXO FOXVWHUV RQ WKLV QHZ UHSUHVHQWDWLRQ ,W LV
HVSHFLDOO\ XVHIXO LQ FRPSOH[ GDWDVHWV ZKHUH WUDGLWLRQDO FOXVWHULQJ PHWKRGV ZRXOG IDLO WR ILQG JURXSLQJV ,Q
SUHYLRXV ZRUN ZH KDYH VKRZQ WKH XWLOLW\ RI XVLQJ NPHDQV FOXVWHULQJ IRU H[SORLWLQJ VWUXFWXUH LQ WKH GDWD WR
DIIHFWDVLJQLILFDQWLPSURYHPHQWLQSUHGLFWLRQDFFXUDF\RQHGXFDWLRQDOGDWDVHWV,QWKLVZRUNZHVKRZWKDWE\
XVLQJ 6SHFWUDO &OXVWHULQJ ZH DUH DEOH WR IXUWKHU LPSURYH WKH VWXGHQW SHUIRUPDQFH SUHGLFWLRQ :H HYDOXDWH DQ
HGXFDWLRQDOGDWDPLQLQJSUHGLFWLRQWDVNSUHGLFWLQJVWXGHQWVWDWHWHVWVFRUHVIURPIHDWXUHVGHULYHGIURPDWXWRU
DQGDOVRSUHVHQWVRPHSUHOLPLQDU\UHVXOWVRQVRPHRWKHU('0WDVNVXVLQJVSHFWUDOFOXVWHULQJ
&DWHJRULHVDQG6XEMHFWGHVFULSWRUV,>$UWLILFLDO,QWHOOLJHQFH@
.H\ :RUGV DQG 3KUDVHV (GXFDWLRQDO 'DWD 0LQLQJ ,QWHOOLJHQW 7XWRULQJ 6\VWHPV %RRWVWUDS $JJUHJDWLQJ
&OXVWHULQJ6SHFWUDO&OXVWHULQJ(QVHPEOH/HDUQLQJ0L[WXUHRI([SHUWV
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
,1752'8&7,21
7KHKLJKO\LQWHUGLVFLSOLQDU\ILHOGRI(GXFDWLRQDO'DWD0LQLQJ('0KDVUHVXOWHGIURP
D IXVLRQ RI PDQ\ GLIIHUHQW DUHDV VRPH RI ZKLFK LQFOXGH 0DFKLQH /HDUQLQJ &RJQLWLYH
6FLHQFHDQG3V\FKRPHWULFV7KHPDLQWDVNLQ('0LVWRFRQVWUXFWFRPSXWDWLRQDOPRGHOV
DQGWRROVWRPLQHGDWDWKDWRULJLQDWHGLQDQHGXFDWLRQDOVHWWLQJ:LWKUDSLGO\LQFUHDVLQJ
GDWD UHSRVLWRULHV IURP GLIIHUHQW HGXFDWLRQDO FRQWH[WV SDSHU WHVWV HOHDUQLQJ ,QWHOOLJHQW
7XWRULQJ6\VWHPVHWFJRRGSUDFWLFHVLQ('0FDQSRWHQWLDOO\DQVZHULPSRUWDQWUHVHDUFK
TXHVWLRQVDERXWVWXGHQWOHDUQLQJ7KLVJRDORI('0LVSURYLQJLQVWUXPHQWDOLQFRPELQLQJ
WKHNQRZOHGJHGHULYHGIURPWKHGDWDWRFRPELQHZLWKWKHRULHVIURPFRJQLWLYHSV\FKRORJ\
WRIRUPXODWHWKHEHVWOHDUQLQJVHWWLQJVDQGPHWKRGRORJLHV
:LWKLQ GDWD PLQLQJ FOXVWHULQJ LV SHUKDSV RQH RI WKH PRVW LPSRUWDQW WRROV IRU ERWK
H[SORUDWRU\DQGFRQILUPDWRU\DQDO\VLV,WLVDWHFKQLTXHWRGLVFHUQPHDQLQJIXOSDWWHUQVLQ
XQODEHOHG GDWD E\ JURXSLQJ WRJHWKHU GDWD SRLQWV WKDW DUH ³VLPLODU´ ,Q ('0 FOXVWHULQJ
KDV EHHQ XVHG LQ D YDULHW\ RI FRQWH[WV 5LWWHU HW DO  ,Q DQ DOUHDG\ LQIOXHQWLDO ZRUN
HVVHQWLDOO\XVHGWKHLPSOLFLWLQIRUPDWLRQFRPSUHVVLRQDOEHLWORVV\KDQGHGE\FOXVWHULQJ
WRUHGXFHWKH.QRZOHGJH7UDFLQJSDUDPHWHUVSDFH>5LWWHU@ZLWKRXWFRPSURPLVLQJWKH
SHUIRUPDQFH RI WKH V\VWHP 'RPLQJXH] HW DO XVHG FOXVWHULQJ DV D WRRO WR JHQHUDWH
LQGLYLGXDOL]HGKLQWVIRUVWXGHQWV>'RPLQJXH]@,QDQRWKHULQWHUHVWLQJZRUN6KLKHWDO
HPSOR\HG FOXVWHULQJ IRU XQVXSHUYLVHG GLVFRYHU\ RI VWXGHQW OHDUQLQJ WDFWLFV >6KLK @
&OXVWHULQJKDVDOVREHHQXVHGIRUFXUULFXOXPSODQQLQJ>0DXOO@IRUHVWLPDWLQJVNLOOVHW
SURILOHV>1XJHQW@DPRQJVWQXPHURXVRWKHUWDVNV+RZHYHULQWHUHVWLQJO\PRVWRIWKHVH
ZRUNV HPSOR\ NPHDQV FOXVWHULQJ H[SHFWDWLRQ PD[LPL]DWLRQ EDVHG FOXVWHULQJ RU
VXEVSDFHFOXVWHULQJ7KLVSDSHUDLPVWRLQWURGXFHWRWKHILHOGRI('0WKHXWLOLW\KDQGHG
E\ VSHFWUDO FOXVWHULQJ RYHU RWKHU FOXVWHULQJ DOJRULWKPV ZKLFK LV DOVR DQ HDV\ WR
LPSOHPHQWDOJRULWKPZLWKQXPHURXVWRROER[HVDYDLODEOHDVZHOO>&KHQ@
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

$XWKRUV¶DGGUHVVHV67ULYHGLHPDLOVKXEKHQGXBWULYHGL#LHHHRUJ=$3DUGRVHPDLO]SDUGRV#FVZSLHGX
* 1 6iUN|]\ HPDLO JVDUNR]\#FVZSLHGX 1 7 +HIIHUQDQ HPDLO QWK#FVZSLHGX 'HSDUWPHQW RI
&RPSXWHU6FLHQFH:RUFHVWHU3RO\WHFKQLF,QVWLWXWH,QVWLWXWH5RDG:RUFHVWHU0$±8QLWHG6WDWHV

7RXQGHUVWDQGWKHZHDNQHVVRIPHWKRGVVXFKDVNPHDQVDXVHIXOZD\RIORRNLQJDW
FOXVWHULQJLVWKHIROORZLQJ&RQVLGHUDVHWRI GLVWULEXWLRQV VXFK
WKDWHDFKRIWKHVHGLVWULEXWLRQVKDVDQDVVRFLDWHGZHLJKWWKHFROOHFWLRQRIZKLFKLVJLYHQ
E\  VXFK WKDW  6XSSRVH D GDWDVHW LV JHQHUDWHG E\ VDPSOLQJ
WKHVH GLVWULEXWLRQVVXFKWKDWDSRLQWLQWKLVGDWDVHWPLJKWEHSLFNHGIURPGLVWULEXWLRQ
 ZLWK SUREDELOLW\  7KH REMHFWLYH RI FOXVWHULQJ PHWKRGV LV WR LGHQWLI\ WKHVH 
GLVWULEXWLRQV JLYHQ D GDWDVHW 0HWKRGV VXFK DV NPHDQV DQG ([SHFWDWLRQ 0D[LPL]DWLRQ
(0 DUH EDVHG RQ HVWLPDWLQJ H[SOLFLW PRGHOV RI WKH GDWD :KLOH NPHDQV ILQGV WKH
FOXVWHUVE\DVVXPLQJWKDWWKHVHWRIGLVWULEXWLRQV WKDWJHQHUDWHGWKHGDWDZDVDVHWRI
VSKHULFDO *DXVVLDQV (0 DOJRULWKPV LQ JHQHUDO OHDUQ D PL[WXUH RI *DXVVLDQV ZLWK
DUELWUDU\VKDSHV0RUHIRUPDOO\NPHDQVILQGVWKHFOXVWHUVE\PLQLPL]LQJWKHGLVWRUWLRQ
IXQFWLRQ 

  
:KHUH  LV WKH FOXVWHU FHQWURLG WR ZKLFK D SRLQW  KDV EHHQ DVVLJQHG ,Q VSLWH RI WKH
JUHDW SRSXODULW\ RI WKH NPHDQV DOJRULWKP YHU\ IHZ WKHRUHWLFDO JXDUDQWHHV RQ LWV
SHUIRUPDQFH DUH NQRZQ >'DVJXSWD @ ,Q SUDFWLFH KRZHYHU NPHDQV SHUIRUPV ZHOO RQ
GDWDWKDWDWOHDVWDSSUR[LPDWHO\IROORZVLWVDVVXPSWLRQRIEHLQJJHQHUDWHGE\DPL[WXUHRI
ZHOOVHSDUDWHG VSKHULFDO *DXVVLDQV >&KDXGKXUL @ 7KLV FRXSOHG ZLWK LWV VLPSOLFLW\
PDNHVLWD KDQG\WRROIRUDGDWDPLQHU+RZHYHUNPHDQVSHUIRUPVSRRUO\ ZKHQWKHVH
DVVXPSWLRQV RI GDWD JHQHUDWLRQ DUH QRW PHW ZKLFK LV XVXDOO\ WKH FDVH LQ UHDO ZRUOG
GDWDVHWV)LJLOOXVWUDWHVWKLVSUREOHPE\DWR\V\QWKHWLFGDWDVHW


)LJ5HVXOWVRIXVLQJNPHDQVRQV\QWKHWLFGDWDVHWVNPHDQVLVXQDEOHWRLGHQWLI\FOXVWHUVZKHQWKHGDWDLV
GLVWULEXWHG LQ FRQFHQWULF JURXSV OHIW ZKLOH LW FOHDUO\ ILQGV WKH FOXVWHUV LQ ZHOO VHSDUDWHG DQG WLJKW VSKHULFDO
*DXVVLDQVULJKW7KHFOXVWHUVLGHQWLILHGDUHLQGLFDWHGE\GLIIHUHQWFRORUV%RWKVHWVKDYHSRLQWV
6SHFWUDO &OXVWHULQJ PDNHV QR VXFK DVVXPSWLRQV IRU GDWD JHQHUDWLRQ ,W LQVWHDG ILQGV
JURXSLQJV E\ DQDO\]LQJ WKH WRS HLJHQYHFWRUV RI WKH DIILQLW\ PDWUL[ DQG KHQFH XVXDOO\
UHWXUQVEHWWHUUHVXOWV
7KH UHVW RI WKH SDSHU LV RUJDQL]HG DV IROORZV 7KH QH[W VHFWLRQ GLVFXVVHV 6SHFWUDO
&OXVWHULQJJLYLQJDWXWRULDORYHUYLHZRIWKHVDPH6HFWLRQXVHVWKHVSHFWUDOFOXVWHULQJ
PHWKRGWRLPSURYHWKHSUHGLFWLRQRISRVWWHVWVFRUHVHPSOR\LQJVWXGHQWIHDWXUHVIURPDQ
,QWHOOLJHQWWXWRUXVLQJDERRWVWUDSDJJUHJDWLQJPHWKRGGHYHORSHGE\WKHDXWKRUV>7ULYHGL
3DUGRV@>7ULYHGL3DUGRV@6HFWLRQLVDGLVFXVVLRQRIUHVXOWVDQGIXWXUHZRUN
63(&75$/&/867(5,1*
2QH RI WKH PRVW LPSRUWDQW GHYHORSPHQWV LQ 0DFKLQH /HDUQLQJ LQ WKH SDVW GHFDGH KDV
EHHQ WKH XVH RI VSHFWUDO PHWKRGV LQ FOXVWHULQJ 7KH\ KDYH FUHDWHG D QHZ ZDYH RI
H[FLWHPHQWWRXQGHUVWDQGWKHSUREOHPRIFOXVWHULQJDQGWKHQRWLRQRIVLPLODULW\EHWZHHQ
SRLQWV EHWWHU DQG IRUPXODWH LW SUHFLVHO\ 2QH PDMRU UHDVRQ IRU WKLV H[FLWHPHQW LV WKDW


 

VSHFWUDO FOXVWHULQJ LV EDVHG RQ VROLG JUDSK WKHRUHWLF SULQFLSOHV *LYHQ LWV VWUHQJWKV LW
ZRXOGEHKLJKO\EHQHILFLDOWRWKH('0FRPPXQLW\LILWLVXVHGPRUHZLGHO\LQWKHVDPH
7KH EURDG LGHD RI FOXVWHULQJ LV HVVHQWLDOO\ WR JURXS SRLQWV WKDW DUH ³VLPLODU´ LQ RQH
FOXVWHUDQGSRLQWVWKDWDUH³GLVVLPLODU´LQWRGLIIHUHQWFOXVWHUV7KHQRWLRQRIVLPLODULW\WKDW
LV HPSOR\HG LQ NPHDQV LV WKH (XFOLGHDQ GLVWDQFH EHWZHHQ GDWD SRLQWV DQG WKH FOXVWHU
FHQWURLGVWRZKLFKWKH\DUHDVVLJQHGWRZKLFKJHWXSGDWHGLQHDFKLWHUDWLRQ,QDVHQVH
WKHLGHDRIVLPLODULW\XVHGLQNPHDQVUHVWULFWVZKDWFRXOGEHNQRZQDERXWWKHJHRPHWU\
RIWKHGDWD,QNPHDQVZHZRUNZLWKWKHGDWDGLUHFWO\LQVSHFWUDOFOXVWHULQJKRZHYHUZH
ZRUN ZLWK D UHSUHVHQWDWLRQ RI WKH GDWD WKDW JLYHV D PRUH JOREDO DQG KHQFH EHWWHU
HQFRGLQJ RI WKH VLPLODULWLHV EHWZHHQ SRLQWV 7KLV ³VLPLODULW\´ LQ VSHFWUDO FOXVWHULQJ LV
UHSUHVHQWHGLQWKHIRUPRIDJUDSKFDOOHGWKHVLPLODULW\JUDSKUHSUHVHQWHGE\ 
ZKHUH  LV WKH VHW RI YHUWLFHV DQG  LV WKH VHW RI HGJHV 7KH LGHD LV WKDW SRLQWV LQ WKH
GDWDVHWFDQEHUHSUHVHQWHGE\DJUDSKZLWKHDFKGDWDSRLQWDVDYHUWH[RIWKHJUDSK DQG
WKHHGJHVFRQQHFWLQJWKHPHQFRGLQJDQRWLRQRIVLPLODULW\ EHWZHHQWKHP7ZR
SRLQWVDUHFRQQHFWHGLQWKHJUDSKLIWKHVLPLODULW\RUZHLJKWEHWZHHQWKHPLVHLWKHUQRQ
]HUR RU DERYH VRPH WKUHVKROG 7KH FOXVWHULQJ SUREOHP FDQ WKHQ EH UHVWDWHG XVLQJ
LQIRUPDWLRQ IURP WKH VLPLODULW\ JUDSK DV :H ZDQW WR ILQG SDUWLWLRQV RI WKLV JUDSK VXFK
WKDW ZHLJKWV EHWZHHQ SRLQWV LQ WKH VDPH JURXS DUH KLJK DQG WKRVH EHWZHHQ SRLQWV LQ
GLIIHUHQW JURXSV DUH ORZ %HIRUH WDONLQJ KRZ ZH FOXVWHU XVLQJ WKLV UHSUHVHQWDWLRQ ZH
LQWURGXFHVRPHQRWDWLRQDQGGLVFXVVKRZWKHJUDSK LVXVHGWRUHSUHVHQWWKHGDWDVHW
*LYHQWKHVLPLODULW\JUDSK RI GDWDSRLQWV WKHUHDUHHVVHQWLDOO\WZR
WKLQJVDERXWLWWKDWWHOOXVVRPHWKLQJDERXWWKHJOREDOVWUXFWXUHRIWKHGDWD
 7KHGHJUHHRIDYHUWH[DGDWDSRLQWLQRXUFDVH7KHGHJUHHRIDYHUWH[WHOOVXV
WKH VXP RI ZHLJKWV RI DOO WKH HGJHV WKDW RULJLQDWH IURP D YHUWH[ L WR DOO RWKHU
YHUWLFHVM,WLVJLYHQE\

7KLV GHILQLWLRQ LV VRPHZKDW QRQVWDQGDUG EXW PRUH JHQHUDO 7KH VWDQGDUG
GHILQLWLRQ IRU GHJUHH RI D YHUWH[ LV RQO\ GHILQHG IRU  DQG WKXV LV
RQO\WKHFRXQWRIYHUWLFHVDJLYHQYHUWLFHVLVFRQQHFWHGWR*LYHQWKLVGHILQLWLRQ
WKH GHJUHH PDWUL[ RI WKH VLPLODULW\ JUDSK LV WKH GLDJRQDO PDWUL[  ZLWK WKH
GHJUHHV RQWKHGLDJRQDO

,QWXLWLYHO\WKHGHJUHHPDWUL[RIDJUDSKWHOOVXVKRZPDQ\SRLQWVHDFKSRLQWLV
FRQQHFWHG WR ZH FRXOG FRQQHFW DOO SRLQWV RU FKRRVH WR FRQQHFW NQHDUHVW
QHLJKERUV RI HDFK SRLQW DQG E\ ³KRZ PXFK´ KHQFH WKH VXPPDWLRQ RI WKH
ZHLJKWV
 7KHZHLJKWHGVLPLODULW\PDWUL[RUWKHDIILQLW\PDWUL[RIWKHVLPLODULW\JUDSK 
RQWKHRWKHUKDQGLVDUHSUHVHQWDWLRQRIVLPLODULW\EHWZHHQDOOWKHSRLQWV(DFK
HOHPHQW LQ WKH DIILQLW\ PDWUL[ LV JLYHQ E\  ZKLFK LV WKH ZHLJKW RU HGJH
EHWZHHQWZRSRLQWV DQG $FRPPRQZD\RIUHSUHVHQWLQJWKHZHLJKWLV

 
  

 
1RWLFH WKDW  LV VLPSO\ WKH H[SRQHQWLDWHG (XFOLGHDQ GLVWDQFH EHWZHHQ WZR
SRLQWV SRLQWV LQ  VFDOHG E\ D SDUDPHWHU FDOOHG WKH VFDOLQJ RU ZHLJKLQJ
SDUDPHWHU  7KLV SDUDPHWHU LV WR EH WXQHG DQG YDU\LQJ LW FKDQJHV WKH ZHLJKW
EHWZHHQSRLQWV$SRLQWWRQRWHLVWKDWLIDOOWKHSRLQWVDUHFRQQHFWHGWKHQDOO 
VXFKWKDW ZLOOEHQRQ]HURYDOXHV,ISRLQWVDUHFRQQHFWHGWRRQO\WKHLUN
QHDUHVWQHLJKERUVDQGQRWHYHU\RWKHUSRLQWWKHQPRVWRIWKHPDWUL[ ZLOOEH
SRSXODWHGE\]HURV
7KHPDWULFHV DQG WHOOXVVRPHWKLQJDERXWWKHJOREDOVWUXFWXUHRIWKHGDWDEXW ZH
GRQ¶WZRUNZLWKWKHPGLUHFWO\:HLQVWHDG ZRUNZLWKWKHJUDSK/DSODFLDQPDWUL[JLYHQ
E\


7KH DERYH LV WKH XQQRUPDOL]HG YHUVLRQ RI WKH /DSODFLDQ 7KHUH DUH WZR QRUPDOL]HG
YHUVLRQVWKDWDUHUHSUHVHQWHGDV

 
7KHILUVW LVFDOOHGWKHV\PPHWULF/DSODFLDQ ZKLOHWKHVHFRQGLVFDOOHGWKHUDQGRPZDON
/DSODFLDQ7KH/DSODFLDQLQDZD\FRPELQHVERWKWKHGHJUHHDQGWKHDIILQLW\PDWUL[DQG
DOVRKDVVRPHPDWKHPDWLFDOO\LQWHUHVWLQJSURSHUWLHVVXFKDVEHLQJSRVLWLYHVHPLGHILQLWH
WKDWPDNHLWHDVLHUWRZRUNZLWK>0RKDU@6LQFHWKH/DSODFLDQLVDUHSUHVHQWDWLRQRIWKH
VLPLODULW\EHWZHHQWKHGDWDSRLQWVZHFDQQRZZRUNZLWKLWWRILQGJURXSVLQWKHGDWD
*LYHQWKHDERYHEDFNJURXQGFOXVWHUVLQDGDWDVHWFDQEHIRXQGE\WKHIROORZLQJPHWKRG
>1J@
 )RU WKH GDWDVHW KDYLQJ  GDWD SRLQWV FRQVWUXFW WKH VLPLODULW\ JUDSK  7KH
VLPLODULW\JUDSKFDQEHFRQVWUXFWHGLQWZRZD\VE\FRQQHFWLQJHDFKGDWDSRLQW
WRWKHRWKHU GDWDSRLQWVRUE\FRQQHFWLQJHDFKGDWDSRLQWWRLWVNQHDUHVW
QHLJKERUV$URXJKHVWLPDWHRIDJRRGYDOXHRIWKHQXPEHURIQHDUHVWQHLJKERUV
LV  7KH VLPLODULW\ EHWZHHQ WKH SRLQWV LV JLYHQ E\ HTXDWLRQ  7KLV ZLOO
JLYHWKHPDWUL[ 
 *LYHQWKHVLPLODULW\JUDSKFRQVWUXFWWKHGHJUHHPDWUL[ 
 8VLQJ DQG ILQG 
 /HW EHWKHQXPEHURIFOXVWHUVWREHIRXQG&RPSXWHWKHILUVW HLJHQYHFWRUV
RI 6RUWWKHHLJHQYHFWRUVDFFRUGLQJWRWKHLUHLJHQYDOXHV
 ,I DUHWKHWRSHLJHQYHFWRUVRI WKHQFRQVWUXFWDPDWUL[ VXFK
WKDW 1RUPDOL]HURZVRIPDWUL[ WREHRIXQLWOHQJWK
 7UHDWWKHURZVLQWKHQRUPDOL]HGPDWUL[ DVSRLQWVLQD GLPHQVLRQDOVSDFH
DQGXVHNPHDQVWRFOXVWHUWKHVH
 ,I DUHWKH FOXVWHUV7KHQDVVLJQDSRLQWLQWKHRULJLQDOGDWDVHW 
WRFOXVWHU LIDQGRQO\LIWKH URZRIWKHQRUPDOL]HG LVDVVLJQHGWRFOXVWHU

,WLVQRWHZRUWK\WKDWZHGRQ¶WFOXVWHUWKHRULJLQDOGDWDVHWGLUHFWO\:HILUVWWUDQVIRUPLWWR
ILQGLWVWRS HLJHQYHFWRUV7KHVHEHLQJ WKH PRVWLPSRUWDQWHLJHQYHFWRUVRI HQFRGH
WKH PD[LPXP LQIRUPDWLRQ DERXW LW $W WKH VDPH WLPH WKLV UHGXFHV WKH GLPHQVLRQDOLW\
ZKLFKZLWKRXWWKURZLQJDZD\PXFKLQIRUPDWLRQZKLFKPDNHVWKHWDVNRIFOXVWHULQJPXFK
HDVLHU7RLOOXVWUDWHWKHSRZHUJLYHQE\WKLVFKDQJHRIUHSUHVHQWDWLRQZHGHPRQVWUDWHLW
RQ D WR\ GDWDVHW )LJ  $ GHWDLOHG WXWRULDO WKDW H[SODLQV YDULRXV VSHFWUDO FOXVWHULQJ


 

DOJRULWKPVDQGVRPHSRLQWRIYLHZVRQZK\LWZRUNVLVE\/X[EXUJ>/X[EXUJ@,QWKH
QH[WVHFWLRQZHGLVFXVVDVSHFLILFDSSOLFDWLRQRIVSHFWUDOFOXVWHULQJLQ('0


)LJ5HVXOWRIXVLQJVSHFWUDOFOXVWHULQJRQDV\QWKHWLFGDWDVHW7KLVV\QWKHWLFVHWKDVSRLQWV7KHFRORUV
LQGLFDWHWKHFOXVWHUVIRXQGE\VSHFWUDOFOXVWHULQJ6XFKJURXSVFDQQRWEHIRXQGE\NPHDQVFOXVWHULQJ

,03529,1*35(',&7,2121678'(173(5)250$1&(,132677(676
%D\HVLDQ .QRZOHGJH 7UDFLQJ >&RUEHWW @ KDV ORQJ EHHQ XVHG WR PRGHO VWXGHQW
NQRZOHGJH LQ DQ LQWHOOLJHQW WXWRULQJ V\VWHP ,76 7KLV NQRZOHGJH HVWLPDWH LV XVHG WR
FDOLEUDWHWKHDPRXQWRIWUDLQLQJDVWXGHQWJHWVWRHQVXUHVNLOOPDVWHU\2QHRIWKHJRDOVRI
VXFK PRGHOLQJ LV WR HQVXUH WKDW VWXGHQWV SHUIRUP ZHOO RQ DFWXDO SRVW WHVWV ,Q IDFW LW LV
UHDVRQDEOHWRVD\WKDWSHUKDSVRQHRIWKHPRVWLPSRUWDQWPHDVXUHVRIVXFFHVVRIDQ,76LV
KRZZHOOSHUIRUPDQFHRQLWWUDQVIHUVWRDFWXDOSRVWWHVWV
7UDGLWLRQDOO\ SHUIRUPDQFH RQ D SRVWWHVW LV SUHGLFWHG E\ XVLQJ SUDFWLFH WHVWV 7KH
SHUFHQWDJHRITXHVWLRQVDQVZHUHGFRUUHFWO\RQWKHVHSUDFWLFHWHVWVJLYHDFUXGHHVWLPDWH
RI KRZ ZHOO D VWXGHQW ZRXOG SHUIRUP RQ WKH DFWXDO SRVW WHVW ,PSURYLQJ WKLV HVWLPDWH
ZRXOGEHKLJKO\EHQHILFLDOWRERWKVWXGHQWVDQGHGXFDWRUV)RUWKHLPSURYHPHQWRIVXFK
DVVHVVPHQW G\QDPLF DVVHVVPHQW >*ULJHUHQNR @ KDV EHHQ DGYRFDWHG DV DQ HIIHFWLYH
PHWKRG7KHELJLGHDRIG\QDPLFDVVHVVPHQWLVWKDWDVVHVVPHQWLVEDVHGRQWKHDPRXQWRI
KHOSVWXGHQWVUHTXLUHWRJHWTXHVWLRQVFRUUHFWDQGLWHQDEOHVWKHWXWRUWRDVVHVVDVLWDVVLVWV
7KLV LV D PDMRU DGYDQWDJH DV LW QRW RQO\ QRW RQO\ DOORZV VWXGHQWV WR OHDUQ ZKLOH EHLQJ
DVVHVVHG EXW FDQ DOVR SUHGLFW VWXGHQW SHUIRUPDQFH RQ SRVWWHVWV EHWWHU 7UDGLWLRQDO
WHVWLQJ LQ ZKLFK RQO\ WKH SHUFHQWDJH RI TXHVWLRQV LV FRQVLGHUHG LV FDOOHG VWDWLF
DVVHVVPHQW7KHQRWLRQRIG\QDPLFDVVHVVPHQW PDNHVLQWXLWLYHVHQVHDV LW JLYHVD ILQHU
JUDLQHG HVWLPDWH RI D VWXGHQW¶V NQRZOHGJH ,I D VWXGHQW JHWV D TXHVWLRQ ZURQJ LW PLJKW
QRW LPSO\ WKDW WKH VWXGHQW KDV QR NQRZOHGJH SHUWDLQLQJ WR WKH TXHVWLRQ 7KH OHYHO RI
NQRZOHGJHWKDWWKHVWXGHQWKDVPLJKWEHHVWLPDWHGE\PHDVXULQJWKHDPRXQWRIKHOSWKDW
WKHVWXGHQWUHTXLUHGWRJHWWKHTXHVWLRQFRUUHFW*LYHQWKHLQWHUDFWLYHQDWXUHRI,76WKH\
DUHWKHLGHDOWHVWEHGIRUPHDVXULQJWKHXWLOLW\RIG\QDPLFDVVHVVPHQW
)HQJ HW DO >)HQJ @ UHSRUWHG WKH UHVXOW WKDW GDWD IURP DQ ,76 FRXOG EHWWHU SUHGLFW
VWDWHWHVWVFRUHV0&$6RU0DVVDFKXVHWWV6WDWH7HVW6FRUHVLQWKHLUH[SHULPHQWLILWRQO\
FRQVLGHUHGWKHH[WUDPHDVXUHVFROOHFWHGLQG\QDPLFDVVHVVPHQWDVFRPSDUHGWRWKHVWDWLF
DVVHVVPHQWFRQGLWLRQ7KHSDSHUKDGDZHDNQHVVWKDWWLPHZDVQHYHUKHOGFRQVWDQW)HQJ
+HIIHUQDQZHQWRQHVWHSDKHDGDQGFRQWUROOHGIRUWLPHLQIROORZLQJZRUN>)HQJ@
7KH\UHSRUWHGEHWWHUSUHGLFWLRQVRQWKH0&$6VWDWHWHVWVFRUHVE\WKHG\QDPLFFRQGLWLRQ
EXW QRW D VWDWLVWLFDOO\ UHOLDEOH GLIIHUHQFH 7KLV ZRUN HIIHFWLYHO\ VKRZHG WKDW G\QDPLF
DVVHVVPHQWOHGWREHWWHUSUHGLFWLRQVRQWKHSRVWWHVW7KLVSUHGLFWLRQZDVGRQHE\ILWWLQJD
OLQHDU UHJUHVVLRQ PRGHO RQ WKH G\QDPLF DVVHVVPHQW IHDWXUHV DQG PDNLQJ SUHGLFWLRQV RQ
WKH 0&$6 WHVW VFRUHV 7KH\ FRQFOXGHG WKDW ZKLOH '\QDPLF $VVHVVPHQW JDYH JRRG
DVVHVVPHQW RI VWXGHQWV WKH 0&$6 SUHGLFWLRQV PDGH XVLQJ WKRVH IHDWXUHV ZHUH RQO\

PDUJLQDOO\ VWDWLVWLFDOO\ VLJQLILFDQW DV FRPSDUHG WR WKH VWDWLF FRQGLWLRQ 7ULYHGL HW
DO>7ULYHGL @ LQYHVWLJDWHG IXUWKHU LI WKH G\QDPLF DVVHVVPHQW GDWD FRXOG EH EHWWHU
XWLOL]HGWRLQFUHDVHSUHGLFWLRQDFFXUDF\RYHUWKHVWDWLFFRQGLWLRQDQGKHQFHHVWDEOLVKWKH
VXSHULRULW\RIG\QDPLFDVVHVVPHQW7KH\XVHGDQHZO\LQWURGXFHGPHWKRG>7ULYHGL@
WKDWFOXVWHUVVWXGHQWVXVLQJWKHNPHDQVDOJRULWKPDQGXVHVPXOWLSOHFOXVWHUPRGHOVDQG
WKHQ HQVHPEOHV WKH SUHGLFWLRQV PDGH E\ HDFK FOXVWHU PRGHO WR DFKLHYH D UHOLDEOH
LPSURYHPHQW +HUH ZH VKRZ WKDW E\ XVLQJ VSHFWUDO FOXVWHULQJ ZH IXUWKHU LPSURYH WKH
SUHGLFWLRQ RQ WKH 0&$6 SRVWWHVW EDVHG RQ WKH G\QDPLF IHDWXUHV 7KH LPSURYHPHQW
REWDLQHGE\XVLQJVSHFWUDOFOXVWHULQJLVQRWRQO\VLJQLILFDQWRYHUWKHVWDWLFFRQGLWLRQEXW
DOVRRYHUUHVXOWVREWDLQHGXVLQJNPHDQVDIWHU. SYDOXHRQDSDLUHGWWHVW
'DWDDQG0HWKRGRORJ\
7KHGDWDXVHGIRUWKLVVWXG\ZDVWKHVDPHDVXVHGE\)HQJHWDO>)HQJ@DQG7ULYHGLHW
DO >7ULYHGL @7KH GDWD LV IURP WKH  VFKRRO \HDU DQG ZDV FROOHFWHG XVLQJ WKH
$66,67PHQWV WXWRU LQ WZR VFKRROV LQ 0DVVDFKXVHWWV $66,67PHQWV >5D]]DT @ LV DQ
,76GHYHORSHGDW:RUFHVWHU3RO\WHFKQLF,QVWLWXWH0$86$7KHGDWDLVIRUVWXGHQWV
DQGWKHIHDWXUHVLQFOXGHGWKHYDULRXVG\QDPLFIHDWXUHV>)HQJ@7KHVHIHDWXUHVZHUH
6WXGHQW¶V SHUFHQW FRUUHFW RQ PDLQ SUREOHPV  1XPEHU RI SUREOHPV GRQH  3HUFHQW
FRUUHFW RQ WKH KHOS TXHVWLRQV  $YHUDJH WLPH VSHQW SHU LWHP  $YHUDJH QXPEHU RI
DWWHPSWV SHU LWHP  $YHUDJH QXPEHU RI KLQWV SHU LWHP 7KH ILUVW IHDWXUH ZDV D VWDWLF
IHDWXUHDQG ZDVXVHGWR PDNHSUHGLFWLRQVRQWKHVWDWLFFRQGLWLRQ ZKLOHWKHRWKHUV ZHUH
XVHG WR PDNH SUHGLFWLRQV LQ WKH G\QDPLF FRQGLWLRQ 7KH SUHGLFWLRQ PDGH ZDV IRU WKH
0&$6WHVWVFRUHVWKDWZDVDYDLODEOHIRUWKHVDPHVWXGHQWVLQWKHIROORZLQJ\HDU$IROG
FURVVYDOLGDWLRQZDVGRQH
7KH PHWKRGRORJ\ XVHG IRU PDNLQJ WKH SUHGLFWLRQ LV D QHZ ERRWVWUDS DJJUHJDWLRQ
HQVHPEOHPHWKRG>7ULYHGL3DUGRV@7KHSURFHGXUHLVVXPPDUL]HGDVIROORZV
 &OXVWHUWKHWUDLQLQJGDWDLQWR FOXVWHUV
 )RU HDFK FOXVWHU WUDLQ D VHSDUDWH OLQHDU UHJUHVVLRQ PRGHO XVLQJ WKH SRLQWV IURP
WKDWFOXVWHUDVWKHWUDLQLQJVHW
 (DFK VXFK WUDLQHG SUHGLFWRU VXFK DV /LQHDU 5HJUHVVLRQ UHSUHVHQWV D PRGHO RI
WKHFOXVWHUDQGLVKHQFHDSSURSULDWHO\FDOOHGDFOXVWHUPRGHO
7KLVLVUHSUHVHQWHGLQILJXUHEHORZ


)LJ7KHILUVWVWHSLQWKHPHWKRGRORJ\IRUXVLQJFOXVWHULQJWRERRWVWUDSDQGPDNLQJDSUHGLFWLRQRQWKH
WUDLQLQJ VHW 7KH VFDOH RI FOXVWHULQJ FDQ EH YDULHG WR JHQHUDWH D QXPEHU RI SUHGLFWLRQV WKDW FDQ WKHQ EH
DJJUHJDWHG



 

7KLVFROOHFWLRQRIFOXVWHUPRGHOVWKDWPDNHDSUHGLFWLRQRQWKHHQWLUHWHVWVHWLVFDOOHG
DSUHGLFWLRQPRGHO30.WKHVXEVFULSWGHQRWHVWKHQXPEHURIFOXVWHUVLQHDFK3UHGLFWLRQ
0RGHO0DNLQJDSUHGLFWLRQIRUDWHVWSRLQWZRXOGLQYROYH/RFDWLQJWKHFOXVWHUWRZKLFK
WKHSRLQWEHORQJVDQGWKHQXVLQJWKHPRGHOWUDLQHGIRUWKDWFOXVWHUWRPDNHWKHSUHGLFWLRQ
IRULW+RZHYHUE\XVLQJWKHQXPEHURIFOXVWHUVDVDIUHHSDUDPHWHUZHJHQHUDWHDVHWRI
 SUHGLFWLRQ PRGHOV 30 30 « 30. VXFK WKDW HDFK KDV D GLIIHUHQW QXPEHU RI
FOXVWHU PRGHOV $QG WKXV ZH FDQ REWDLQ  GLIIHUHQW SUHGLFWLRQV RQ WKH WHVW VHW 7KHVH
SUHGLFWLRQVDUHWKHQDYHUDJHGWRREWDLQDVLQJOHVWURQJSUHGLFWLRQ
7KLV PHWKRG FDQ EH WKRXJKW RI OLNH DQ DGDSWLYH PL[WXUH RI ORFDO H[SHUWV >-DFREV
+LQWRQ@WKDWXVHVFOXVWHULQJWRERRWVWUDS%XWXQOLNHLQRWKHUEDJJLQJPHWKRGVZKLFK
VHOHFWDUDQGRPVXEVHWWRERRWVWUDSWKLVPHWKRGKDVDVSHFLILFH[SHUWIRUHDFKFOXVWHURI
WKHGDWD%\YDU\LQJWKHJUDQXODULW\RIWKHFOXVWHULQJZHDUHDEOHWRREWDLQDPL[WXUHRI
H[SHUWV RQ WKH GDWD DW GLIIHUHQW OHYHOV HDFK RI ZKLFK JLYHV D SUHGLFWLRQ RQ WKH WHVW VHW
ZKLFKDUHWKHQDYHUDJHGWRJHWRQHSUHGLFWLRQ
5HVXOWV8VLQJ6SHFWUDO&OXVWHULQJ
7KH UHVXOWV RI FOXVWHULQJ WKH GDWD XVLQJ ERWK NPHDQV DQG VSHFWUDO FOXVWHULQJ DUH
UHSUHVHQWHGLQILJXUHEHORZ6LQFHWKHGDWDLVKLJKGLPHQVLRQDODQGWKHDFWXDOSDUWLWLRQV
FDQQRWEHSLFWXUHGWKLVYLVXDOL]DWLRQLVGRQHE\GRLQJDPXOWLGLPHQVLRQDOVFDOLQJRQWKH
GDWDVHW WR WKUHH GLPHQVLRQV ZLWK HDFK FOXVWHU LGHQWLILHG E\ D GLIIHUHQW FRORU 7KLV
YLVXDOL]DWLRQLVIRU. 
 

 

 

 
=

 

 

 
 
 
 
 
   
   
<   <  
; ;

   


 

 
<

<

 

 

   
         
; ;


   

 

 
=

=
 

 

   
           
< <

)LJ7KHLPDJHVRQWKHOHIWFROXPQDUHIRUNPHDQVDQGWKRVHRQWKHULJKWDUHIRUVSHFWUDOFOXVWHULQJ7KHWRS
URZUHSUHVHQWVWKHSORWRIWKH$66,67PHQWGDWDVFDOHGGRZQE\PXOWLGLPHQVLRQDOVFDOLQJWRWKUHHGLPHQVLRQV
DQGWKHFOXVWHUVLGHQWLILHGE\ERWKNPHDQVDQGVSHFWUDOFOXVWHULQJ7KHURZVEHORZDUHVLPSO\GLIIHUHQWSODQDU
YLHZVRIWKHSORWLQURZ
7KHVSHFWUDOFOXVWHULQJHQVHPEOHUHVXOWVDUHQRWRQO\VLJQLILFDQWRYHUWKHVWDWLFFRQGLWLRQ
. LQILJXUHEXWDOVRDUHVLJQLILFDQWIRUWKHNPHDQVJHQHUDWHGHQVHPEOHEH\RQG.
ZLWKSLQHDFKFDVHRQDSDLUHGWWHVW
 
306SHFWUDO
30NPHDQV

6SHFWUDO(QVHPEOHG
NPHDQV(QVHPEOHG

0HDQ$EVROXWH(UURU









 
      
. 
)LJ7KHSORWVRIWKHIROGFURVVYDOLGDWHGHUURUVE\WKHYDULRXVSUHGLFWLRQPRGHOVDQGHQVHPEOHVIURPWR
.   IRU ERWK NPHDQV DQG VSHFWUDO FOXVWHULQJ 7KH . HQVHPEOH SUHGLFWLRQ LV WKH DYHUDJH RI SUHGLFWLRQV
UHWXUQHGE\SUHGLFWLRQPRGHOVIURPWR.

&21&/86,21',6&866,21$1')8785(:25.
7KH PHWKRGRORJ\ GHVFULEHG LQ WKH SDSHU ZDV HPSOR\HG RQ VRPH RWKHU ('0 WDVNV DV
ZHOO VXFK DV PDNLQJ DQ LQWXWRU SUHGLFWLRQ RQ WKH .'' &XS  GDWDVHW DQG RQ WKH
3HUIRUPDQFH)DFWRU$QDO\VLV3)$WDVN>*RQJ@3UHOLPLQDU\UHVXOWVVXPPDUL]HGIRU
3)$EHORZKDYHLQGLFDWHGDQLPSURYHPHQWLQWKHSUHGLFWLRQDFFXUDFLHV
7DEOH3UHOLPLQDU\ZRUNRQ3HUIRUPDQFH)DFWRU$QDO\VLV
6SHFWUDO .  .  .  .  . 
(QVHPEOH
$8&     



 

7KH UHVXOWV LQGLFDWH DQ LPSURYHPHQW RYHU WKH EDVH FRQGLWLRQ DV PRUH SUHGLFWLRQ
PRGHOVDUHDYHUDJHG%XWWKLVUHVXOWLVQRWFURVVYDOLGDWHGDQGLVDZRUNLQSURJUHVV$OVR
JLYHQWKHSURKLELWLYHVL]HRIWKHGDWDVHWVSHFWUDOFOXVWHULQJZDVQRWXVHGIRUDOOWKHURZV
LQWKHWUDLQLQJVHWEXWDUDQGRPVXEVHWRI WKHP ZDVXVHG7KLV ZDVGRQHWRVDYHWLPH
KRZHYHUWKLVLVWKHUHDVRQWKHGHWDLOHGUHVXOWVDUHQRWUHSRUWHGLQWKLVZRUN$OVRPRUH
ZRUN QHHGV WR EH GRQH WR XVH VSHFWUDO FOXVWHULQJ PHWKRGV PRUH HIILFLHQWO\ IRU PDVVLYH
GDWDVHWVVXFKDVWKH.''FXSGDWDVHW
$ GHHSHU ZD\ RI ORRNLQJ DW FOXVWHULQJ LV HVVHQWLDOO\ DV D VFKHPH IRU ORVV\ GDWD
FRPSUHVVLRQ,PSURYHPHQWLQSUHGLFWLRQDFFXUDF\XVLQJVSHFWUDOFOXVWHULQJRYHUNPHDQV
LQGLFDWHV WKDW VSHFWUDO FOXVWHULQJ LV D EHWWHU LQIRUPDWLRQ FRPSUHVVLRQ PHWKRG WKDQ N
PHDQV DQG KHQFH WHOOV VRPHWKLQJ GHHSHU DERXW WKH VWUXFWXUH RI WKH GDWD WKDW NPHDQV
PLVVHV 7KLV ZRXOG PHDQ DQ LQWHUHVWLQJ DSSOLFDWLRQ WR UHGXFH WKH NQRZOHGJH WUDFLQJ
VSDFH OLNH E\ 5LWWHU HW DO >5LWWHU @ DQG VHH KRZ LW FRPSDUHV ZLWK SHUIRUPDQFH
UHWXUQHGE\NPHDQVFOXVWHULQJ
7KHREMHFWLYHRIWKLVZRUNZDVWRLQWURGXFHWRWKHGRPDLQRI('0WKHJUHDWXWLOLW\RI
XVLQJ VSHFWUDO FOXVWHULQJ :H XVHG VSHFWUDO FOXVWHULQJ WR HQKDQFH WKH SHUIRUPDQFH RI D
QHZ HQVHPEOH PHWKRG SURSRVHG LQ DQ HDUOLHU ZRUN E\ WKH DXWKRUV :KLOH WKH REMHFWLYH
ZDV WR LQWURGXFH WKH XVH RI VSHFWUDO FOXVWHULQJ D YHU\ VLJQLILFDQW UHVXOW RI WKH ZRUN LV
SURYLQJ WKH HIILFDF\ RI '\QDPLF $VVHVVPHQW DV FRPSDUHG WR VWDWLF DVVHVVPHQW 7KHVH
UHVXOWV VKRZ WKDW DQ ,76 WKDW FDQ DVVHVV DV LW DVVLVWV RIIHUV D VLJQLILFDQW DGYDQWDJH WR
VWXGHQWVDQGWHDFKHUV7KLVLVLPSRUWDQWEHFDXVHLWFDQQRWRQO\VDYHWLPHWKDWLVZDVWHG
RQDVVHVVPHQWIRULQVWUXFWLRQEXWLWFDQDOVREHDEHWWHUSUHGLFWRURIWKHLUSHUIRUPDQFHLQ
SRVWWHVWV
7KHUHVXOWVIRUWKHWDVNRISUHGLFWLQJWKHSRVWWHVWVFRUHVKDYHEHHQYHU\HQFRXUDJLQJ
KRZHYHU WKHUH DUH VRPH DUHDV WKDW QHHG IXUWKHU ZRUN DQG FRXOG LPSURYH SUHGLFWLRQ
DFFXUDF\IXUWKHU2QHVXFKDUHDRISRVVLEOHLPSURYHPHQWLVDOORZLQJIRUIX]]\FOXVWHULQJ
7R PDNH D SUHGLFWLRQ WKH FOXVWHU FORVHVW WR D WHVW D SRLQW ZDV LGHQWLILHG DQG WKHQ WKH
H[SHUWIRUWKDWFOXVWHUZDVXVHGWRPDNHDSUHGLFWLRQRQLW,QPDQ\UHDOZRUOGH[DPSOHV
PHPEHUVKLSRIDGDWDSRLQWWRDSDUWLFXODUFOXVWHULVDWULFN\TXHVWLRQWRDQVZHU$PRUH
UHDOLVWLFYLHZLVWRDOORZIRUIX]]\FOXVWHULQJ7KDWLVJLYHQDWHVWSRLQWZHGHWHUPLQHLWV
SUREDELOLW\ RI RFFXUULQJ LQ HDFK RI WKH FOXVWHUV 7KHQ ZH FDQ REWDLQSUHGLFWLRQVE\WKH
FOXVWHUPRGHOH[SHUWIRUHDFKRIWKHFOXVWHUVDQGREWDLQRQHSUHGLFWLRQIRURQHWHVWSRLQW
WKDW LV D ZHLJKWHG DYHUDJH RI WKH SUHGLFWLRQV UHWXUQHG E\ HDFK FOXVWHU PRGHO HDUOLHU D
SUHGLFWLRQZDVPDGHRQWKHWHVWSRLQWE\RQO\RQHFOXVWHUPRGHOZLWKWKHZHLJKWVEHLQJ
WKHSUREDELOLW\WKDWWKHSRLQWOLHVLQWKDWFOXVWHU:KLOHIX]]\FRXQWHUSDUWVWRWKHNPHDQV
DOJRULWKP VXFK DV IX]]\ FPHDQV DUH ZHOO NQRZQ WKH LGHD RI GRLQJ IX]]\ VSHFWUDO
FOXVWHULQJ LV VRPHWKLQJ WR EH H[SORUHG &OHDUO\ VSHFWUDO FOXVWHULQJ XVHV NPHDQV DW D
ORZHUGLPHQVLRQDOUHSUHVHQWDWLRQRIWKHODSODFLDQDQGIX]]\FPHDQVFDQEHXVHGDWWKLV
OHYHO+RZHYHUWKHHIIHFWLYHQHVVRIGRLQJWKHVDPHLVQRWNQRZQ
$QRWKHU SRVVLEOH DUHD RI LPSURYHPHQW LV XVLQJ PHWKRGV WR PHUJH FOXVWHUV WKDW DUH
VSDUVHO\ SRSXODWHG >&KHQJ @ %\ WKLV PHWKRG ZH FRXOG LPSURYH ERWK WKH TXDOLW\ RI
FOXVWHULQJLIWKHWDVNLVSXUHO\XQVXSHUYLVHGDQGSUHGLFWLRQDFFXUDF\LIWKHWDVNOLNHLQ
WKHDSSOLFDWLRQGLVFXVVHGLVDSUHGLFWLRQWDVN
,QWKLVZRUNZHFRPELQHSUHGLFWLRQVE\DYHUDJLQJWKHP&OHDUO\WKLVLVDVXERSWLPDO
FKRLFH ,GHDOO\ ZH ZRXOG ZDQW WR SLFN WKRVH SUHGLFWLRQV PDGH E\ SUHGLFWLRQ PRGHOV
ZKLFK DUH JRRG LQ SUHGLFWLRQ DQG KDYH OHVV FRUUHODWLRQ ZLWK HDFK RWKHU DUH GLYHUVH
6LQFHWKHPHWKRGXVHGWRPDNHWKHSRVWWHVWSUHGLFWLRQVZDVDQHQVHPEOHPHWKRGLWFDQ
EH XVHG WR FRPELQH WKH SUHGLFWLRQV WKHPVHOYHV 3UHOLPLQDU\ ZRUN XWLOL]LQJ WKLV LGHD RI
XVLQJFOXVWHULQJ WRERRWVWUDSWKHSUHGLFWLRQVUHWXUQHGE\YDULRXVSUHGLFWLRQ PRGHOVKDV
VKRZQSURPLVH

$&.12:/('*(0(176
:H DUH JUDWHIXO WR WKH IROORZLQJ IXQGHUV IRU VXSSRUWLQJ WKLV UHVHDUFK
KWWSZZZZHEFLWDWLRQRUJ\P<IU :H ZRXOG DOVR OLNH WR WKDQN WKH 3LWWVEXUJK
6FLHQFHRI/HDUQLQJ&HQWUHIRUWKH&RJQLWLYH7XWRU.''GDWDVHW&RPPHQWVDERXWWKLV
ZRUN E\ 'U &DUROLQD 5XL] 'U 6HUJLR $OYDUH] DQG 'U $OH[DQGUX 1LFXOHVFX0L]LO ZHUH
HVSHFLDOO\KHOSIXODQGDUHDSSUHFLDWHG
5()(5(1&(6

&+$8'+85, .  '$6*837$ 6 9$77$1, $  /HDUQLQJ 0L[WXUH RI *DXVVLDQV XVLQJ WKH NPHDQV
$OJRULWKP,Q&R55YRODEVKWWSDU[LYRUJDEV
&+(1:<621* < %$, +/,1 & - &+$1* ( <   $&&(37('3DUDOOHO6SHFWUDO&OXVWHULQJLQ
'LVWULEXWHG6\VWHPV,Q,(((7UDQVDFWLRQVRQ3DWWHUQ$QDO\VLVDQG0DFKLQH,QWHOOLJHQFH
&+(1*'.$11$1 5 9(03$/$ 6:$1* * $'LYLGHDQG0HUJH0HWKRGRORJ\IRU&OXVWHULQJ,Q
7KH-RXUQDORIWKH$&09RO9
&25%(77 $ 7  $1'(5621 - 5   .QRZOHGJH 7UDFLQJ 0RGHOLQJ WKH $FTXLVLWLRQ RI 3URFHGXUDO
.QRZOHGJH,Q8VHU0RGHOLQJDQG8VHU$GDSWHG,QWHUDFWLRQSS
'$6*837$ 6  /HDUQLQJ 0L[WXUH RI *DXVVLDQV ,Q 7KH WK $QQXDO ,((( V\PSRVLXP RQ )RXQGDWLRQV RI
&RPSXWHU6FLHQFH
'20,1*8(=$ .<$&() . &855$1 - 5 'DWD0LQLQJIRU ,QGLYLGXDOL]HG+LQWVLQH/HDUQLQJ,Q
3URFHHGLQJVRIWKH7KLUG,QWHUQDWLRQDO&RQIHUHQFHRQ(GXFDWLRQDO'DWD0LQLQJ
)(1* 0  +())(51$1 1 7 .2(',1*(5 . 5  $GGUHVVLQJ WKH DVVHVVPHQW FKDOOHQJH LQ DQ RQOLQH
V\VWHP WKDW WXWRUV DV LW DVVHVVHV 8VHU 0RGHOLQJ DQG 8VHU$GDSWHG ,QWHUDFWLRQ 7KH -RXUQDO RI
3HUVRQDOL]DWLRQ5HVHDUFK
)(1*0+())(51$1 1 7  &DQ:H *HW%HWWHU $VVHVVPHQW)URP$7XWRULQJ6\VWHP&RPSDUHGWR
7UDGLWLRQDO 3DSHU 7HVWLQJ" &DQ :H +DYH 2XU &DNH EHWWHU DVVHVVPHQW DQG (DW LW WRR VWXGHQW OHDUQLQJ
GXULQJWKHWHVW3URFHHGLQJVRIWKHUG,QWHUQDWLRQDO&RQIHUHQFHRQ(GXFDWLRQDO'DWD0LQLQJ
*21* <  %(&. - ( +())(51$1 1 7    $FFHSWHG +RZ WR &RQVWUXFW 0RUH $FFXUDWH 6WXGHQW
0RGHOV&RPSDULQJDQG2SWLPL]LQJ.QRZOHGJH7UDFLQJDQG3HUIRUPDQFH)DFWRU$QDO\VLV,Q,QWHUQDWLRQDO
-RXUQDORI$UWLILFLDO,QWHOOLJHQFHLQ(GXFDWLRQ$FFHSWHG
*5,*(5(1.2(/67(,1%(5*5-'\QDPLF7HVWLQJ,Q3V\FKRORJLFDO%XOOHWLQSS
-$&2%6 5 $  -25'21 0 , 12:/$1 6 - +,1721 *(  $GDSWLYH 0L[WXUH RI /RFDO ([SHUWV ,Q
1HXUDO&RPSXWDWLRQ9RO1R
/8;%85* 8  $ 7XWRULDO RQ 6SHFWUDO &OXVWHULQJ ,Q 6WDWLVWLFV DQG &RPSXWLQJ .OXZHU $FDGHPLF
3XEOLVKHUV+LQJKDP0$86$9RO,VVXH
0$8// . ( 6$/9,'$5 0 * 6801(5 7  2QOLQH &XUULFXOXP 3ODQQLQJ %HKDYLRU RI 7HDFKHUV ,Q
3URFHHGLQJVRIWKH7KLUG,QWHUQDWLRQDO&RQIHUHQFHRQ(GXFDWLRQDO'DWD0LQLQJ
02+$5 % 7KH /DSODFLDQ 6SHFWUXP RI *UDSKV ,Q *UDSK 7KHRU\ &RPELQDWRULFV DQG $SSOLFDWLRQV 

1*$<-25'210,:$,66 < 2Q6SHFWUDO&OXVWHULQJ$QDO\VLVDQGDQ$OJRULWKP,Q$GYDQFHVLQ
1HXUDO,QIRUPDWLRQ3URFHVVLQJ6\VWHPV
18*(17 5 '($1 1 $<(56 (  6NLOO6HW 3URILOH &OXVWHULQJ 7KH (PSW\ .0HDQV $OJRULWKP ZLWK
$XWRPDWLF6SHFLILFDWLRQRI6WDUWLQJ&OXVWHU&HQWHUV,Q3URFHHGLQJVRIWKH7KLUG,QWHUQDWLRQDO&RQIHUHQFH
RQ(GXFDWLRQDO'DWD0LQLQJ
5$==$4/)(1*018==2-21(6*+())(51$117.2(',1*(5.5-81.(5%5,77(56.1,*+7
$ $1,6=&=<. & &+2.6(< 6 /,9$. 7 0(5&$'2 ( 7851(5 7 ( 83$/(.$55 :$/2126.,
-$ 0$&$6(. 0 $ $1' 5$60866(1 . 3  7KH $VVLVWPHQW 3URMHFW %OHQGLQJ $VVHVVPHQW DQG
$VVLVWLQJ ,Q & . /RRL * 0F&DOOD % %UHGHZHJ  - %UHXNHU (GV 3URFHHGLQJV RI WKH WK
,QWHUQDWLRQDO&RQIHUHQFHRQ$UWLILFLDO,QWHOOLJHQFHLQ(GXFDWLRQ$PHVWHUGDP,623UHVVSS
5,77(5 6 +$55,6 7 . 1,;21 7 ',&.,621 ' 0855$< 5& 72:/( %  5HGXFLQJ WKH
.QRZOHGJH 7UDFLQJ 6SDFH ,Q 3URFHHGLQJV RI WKH 6HFRQG ,QWHUQDWLRQDO &RQIHUHQFH RQ (GXFDWLRQDO 'DWD
0LQLQJ
6+,+ % .2(',1*(5 . 5 6&+(,1(6 5  8QVXSHUYLVHG 'LVFRYHU\ RI 6WXGHQW /HDUQLQJ 7DFWLFV ,Q
3URFHHGLQJVRIWKH7KLUG,QWHUQDWLRQDO&RQIHUHQFHRQ(GXFDWLRQDO'DWD0LQLQJ
75,9(',63$5'26=$+())(51$1 1 7  VXEPLWWHG7KH8WLOLW\RI&OXVWHULQJLQ3UHGLFWLRQ7DVNV
,Q7KH6HYHQWK$&06,*.''&RQIHUHQFHRQ.QRZOHGJH'LVFRYHU\DQG'DWD0LQLQJ
75,9(',63$5'26=$+())(51$117DFFHSWHG&OXVWHULQJ6WXGHQWVWR*HQHUDWHDQ(QVHPEOHWR
,PSURYH 6WDQGDUG 7HVW 6FRUH 3UHGLFWLRQV ,Q 7KH )LIWHHQWK ,QWHUQDWLRQDO &RQIHUHQFH RQ $UWLILFLDO
,QWHOOLJHQFHLQ(GXFDWLRQ