1RYHPEHU_9ROXPH_1XPEHU

7HDFKHU(YDOXDWLRQ:KDW
V)DLU":KDW
V(IIHFWLYH"3DJHV

2EVHUYLQJ&ODVVURRP3UDFWLFH
&KDUORWWH'DQLHOVRQ
&ODVVURRPREVHUYDWLRQVFDQIRVWHUWHDFKHUOHDUQLQJ²LIREVHUYDWLRQV\VWHPVLQFOXGHFUXFLDO
FRPSRQHQWVDQGREVHUYHUVNQRZZKDWWRORRNIRU
-HQQLIHU/RSH]ORRNVXSTXLFNO\DV0V$QGHUVRQWKHSULQFLSDOVWHSVLQWRKHUWKJUDGHFODVVURRP
6KHJODQFHVDURXQGQHUYRXVO\:KDWPLJKWWKLVORRNOLNHWR0V$QGHUVRQ"
$WWKHEHJLQQLQJRIWKHOHVVRQ²DQLQWURGXFWLRQWRWKHWRSLFVRIEXR\DQF\DQGGHQVLW\²WKHVWXGHQWV
SXVKHGWKHLUGHVNVWRJHWKHUWRPDNHWDEOHV2QHDFKWDEOHLVDGLVKSDQIXOORIZDWHU-HQQLIHU
DOZD\VKRSHVIRUWKHEHVWRQGD\VOLNHWKLVVKH
VQRWRULRXVZLWKPHPEHUVRIWKHFXVWRGLDOVWDIIIRU
YDULRXVDGYHQWXUHVLQKHUFODVVURRP%XWWRGD\DOOLVZHOO7KHVWXGHQWVHDFKKDYHDOXPSRIFOD\
DQGWKH\
YHZHLJKHGWKHLUOXPSVRQDSDQEDODQFHWRVDWLVI\WKHPVHOYHVWKDWWKH\DOOKDYHURXJKO\
WKHVDPHDPRXQWRUPDVVRIFOD\
7KHVWXGHQWVSXWWKHLUFOD\LQWKHZDWHUDQGZDWFKLWVLQN7KH\
UHFKDOOHQJHGWRPDNHLWIORDWZKLFK
WKH\GLVFRYHUWKH\FDQGRLIWKH\IDVKLRQWKHLUFOD\LQWRWKHVKDSHRIDERDW7KH\ILQGWKLVH[FLWLQJ
DQGWKH\LPPHGLDWHO\WDFNOHWKHQH[WFKDOOHQJH&DQWKH\PDNHDJRRGERDW²JRRGPHDQLQJRQH
WKDWZLOOKROGDORWRIFDUJRLQWKHIRUPRISDSHUFOLSV"7KH\H[SORUHYDULRXVTXHVWLRQV6KRXOGWKH
ERDWKDYHWKLQRUWKLFNVLGHV"7KLQVLGHVLW
VSRVVLEOHWRHQFORVHPRUHYROXPHZLWKWKHVDPH
DPRXQWRIPDWHULDO6KRXOGWKH\VKDSHLWOLNHDERZORUOLNHDFDQRH"/LNHDERZOIRUWKHVDPH
UHDVRQFDQRHVDUHIRUUDSLGV
7KHVWXGHQWVKDYHEHFRPHTXLWHSURILFLHQW7KH\
UHFRQVWUXFWLQJERDWVZLWKSDSHUWKLQZDOOVDQG
HYHQWRSVVRWKHZDWHUZRQ
WUXVKLQDQGWKH\
UHVNHWFKLQJWKHLUGHVLJQVRQWKHERDUGVKRZLQJWKH
QXPEHURISDSHUFOLSVHDFKRQHZLOOKROG-HQQLIHULVLPSUHVVHGDQGH[SUHVVHVKHUDGPLUDWLRQ7KH
ERDWVKROGWKHQWKHQDQGILQDOO\PRUHWKDQSDSHUFOLSV
%XWKHUHLV0V$QGHUVRQLQWKHURRPWRGRDQXQDQQRXQFHGREVHUYDWLRQ:KDWPXVWVKHEH
WKLQNLQJ"1RZVKHFRPHVRYHUWR-HQQLIHUPRWLRQLQJKHUDVLGHDQGVD\LQJLQDZKLVSHU,
OOFRPH
EDFNZKHQ\RX
UHWHDFKLQJ

,QVWDQW5HSOD\7KH3ULQFLSDO
V3RLQWRI9LHZ
0V$QGHUVRQSDXVHVDW-HQQLIHU/RSH]
VGRRUEHIRUHVWHSSLQJLQ7KHFODVVURRPORRNVDOLWWOH
FKDRWLF:KDWDUHWKHVHVWXGHQWVGRLQJ"7KH\ORRNEXV\IRUVXUHDQGWKH\VHHPWREHKDYLQJD
JRRGWLPH%XW-HQQLIHU²ZKDWLVVKHGRLQJ"7KHGLVWULFWLVVXSSRVHGWREHJHWWLQJVWDUWHGZLWKWKH
&RPPRQ&RUH6WDWH6WDQGDUGV,VWKLVZKDWWKH\
UHVXSSRVHGWRORRNOLNH"%XWLWVHHPVDVWKRXJK
WKHWHDFKHU
VQRWGRLQJDQ\WKLQJ0V$QGHUVRQGHFLGHVWRFRPHEDFNODWHUZKHQ-HQQLIHULV
WHDFKLQJ$QGVKHKRSHVWKDWZKHQVKHGRHVVKHFDQEHVXUHVKHLVHYDOXDWLQJ-HQQLIHU
VWHDFKLQJ
FRUUHFWO\

7KH&UX[RIWKH3UREOHP
7KLVHSLVRGHJHWVWRWKHKHDUWRIWKHLVVXHIDFLQJERWKWHDFKHUVDQGVXSHUYLVRUVLQWKLVQHZHUDRI
KLJKVWDNHVWHDFKHUHYDOXDWLRQ$IWHUDOOIRUDV\VWHPRIWHDFKHUHYDOXDWLRQWREHGHIHQVLEOHHLWKHU
SURIHVVLRQDOO\RUOHJDOO\LWPXVWEHIDLU²WKDWLVWKHMXGJPHQWVWKDWDUHPDGHDERXWDWHDFKHU
V
SUDFWLFHPXVWDFFXUDWHO\UHIOHFWWKHWHDFKHU
VWUXHOHYHORISHUIRUPDQFH$QGEHFDXVHWKH
TXLQWHVVHQWLDOVNLOORIWHDFKLQJLVWHDFKLQJDQGLWFDQEHREVHUYHGZHVKRXOGFRQGXFWWKRVH
REVHUYDWLRQVZLWKLQWHJULW\DQGVNLOO
,GHQWLI\LQJJRRGSUDFWLFHWKURXJKREVHUYDWLRQLVOHVVIHDVLEOHZLWKRWKHUMREUROHVLQHGXFDWLRQ)RU
H[DPSOHLI\RX
UHWU\LQJWRDVVHVVWKHVNLOOVRIDSULQFLSDOVFKRROQXUVHRUPHQWRUWKHUH
VQRWRQH
VLQJOHSODFH²VXFKDVDFODVVURRP²\RXFRXOGJRWRREVHUYHWKHHVVHQWLDOVNLOOVHPERGLHGLQWKDW
UROHWKH\
UHVSUHDGRXWRYHUPDQ\ORFDWLRQV3ULQFLSDOVLQWHUDFWZLWKPDQ\GLIIHUHQWLQGLYLGXDOV²
WHDFKHUVVWXGHQWVSDUHQWVDQGFRPPXQLW\PHPEHUV²DQGWKH\HQJDJHLQPDQ\GLIIHUHQWW\SHVRI
DFWLYLWLHV²FRQGXFWLQJPHHWLQJVRUJDQL]LQJWKHVFKHGXOHSODQQLQJDEXGJHWDQGVRRQ²ZLWKVXFK
YDULHW\WKDWQRVLQJOHLWHPFDQEHDVWDQGLQIRUWKHHQWLUHMRE,QFRQWUDVWWKHZRUNRIWHDFKHUVLV
HDVLHUWRFKDUDFWHUL]HDVWKDWZKLFKKDSSHQVLQWKHLUFODVVURRPVZLWKVWXGHQWV
,W
VWUXHWKDWWHDFKLQJLVVXSSRUWHGE\DORWRIEHKLQGWKHVFHQHVZRUNEXWQHYHUWKHOHVVZHFDQ
REVHUYHWKHLQWHUDFWLYHZRUNZLWKVWXGHQWVDQGWKLVLVWKHKHDUWRIWHDFKLQJ7KHUHIRUHFODVVURRP
REVHUYDWLRQLVDFUXFLDODVSHFWRIDQ\V\VWHPRIWHDFKHUHYDOXDWLRQ1RPDWWHUKRZVNLOOHGDWHDFKHU
LVLQRWKHUDVSHFWVRIWHDFKLQJ²VXFKDVFDUHIXOSODQQLQJZRUNLQJZHOOZLWKFROOHDJXHVDQG
FRPPXQLFDWLQJZLWKSDUHQWV²LIFODVVURRPSUDFWLFHLVGHILFLHQWWKDWLQGLYLGXDOFDQQRWEH
FRQVLGHUHGDJRRGWHDFKHU

&OHDU6WDQGDUGVRI3UDFWLFH
3UHFLVHO\ZKDWWKHREVHUYHUVXSHUYLVRUPHQWRURUFRDFKORRNVIRULQDQREVHUYDWLRQLVDIXQFWLRQ
RIWKHLQVWUXFWLRQDOIUDPHZRUNWKDWWKHVFKRROGLVWULFWRUVWDWHKDVDGRSWHG8QOHVVWKHUHLVDFOHDU
DQGDFFHSWHGGHILQLWLRQRIJRRGWHDFKLQJWHDFKHUVZRQ
WNQRZKRZWKHLUSHUIRUPDQFHZLOOEH
HYDOXDWHGDQGREVHUYHUVZRQ
WNQRZZKDWWRORRNIRU

)RUH[DPSOHLQWKH'DQLHOVRQ)UDPHZRUNIRU7HDFKLQJ WZRRIWKHIRXUGRPDLQVRIWHDFKLQJWKH

FODVVURRPHQYLURQPHQWDQGLQVWUXFWLRQDUHREVHUYDEOHLQDWHDFKHU
VFODVVURRPSUDFWLFH(DFKRI
WKRVHWZRGRPDLQVFRQWDLQVILYHVPDOOHUFRPSRQHQWVZKLFKVKRZREVHUYHUVH[DFWO\ZKDWWRORRN
IRUZKHQWKH\VWHSLQWRDFODVVURRPVXFKDVZKHWKHUWKHWHDFKHUKDVHVWDEOLVKHGDQHQYLURQPHQW
RIUHVSHFWDQGUDSSRUWPDQDJHGFODVVURRPSURFHGXUHVXVHGYDULRXVTXHVWLRQLQJDQGGLVFXVVLRQ
WHFKQLTXHVRUHQJDJHGVWXGHQWVLQOHDUQLQJ

5HVHDUFK%DVHGDQG9DOLGDWHG
7KHVHWHDFKLQJSUDFWLFHVDUHJURXQGHGLQDVROLGUHVHDUFKEDVH(PSLULFDOVWXGLHVKDYHVKRZQWKDW
HDFKFRPSRQHQWRIWKH)UDPHZRUNIRU7HDFKLQJLVDVVRFLDWHGZLWKLPSURYHGVWXGHQWOHDUQLQJ,W
V
DOVRYDOLGDWHGDVDQ\LQVWUXPHQWXVHGIRUKLJKVWDNHVWHDFKHUHYDOXDWLRQVKRXOGEH7KDWLVKLJK
OHYHOVRIWHDFKHUSHUIRUPDQFHRQWKHLQVWUXFWLRQDOIUDPHZRUNDVDZKROHVKRXOGSUHGLFWKLJKOHYHOV
RIVWXGHQWOHDUQLQJ
7KLVLPSHUDWLYHLPSRVHVVLJQLILFDQWGHPDQGVRQWKHGHYHORSHUVRIHYDOXDWLRQLQVWUXPHQWVEHFDXVH
VXFKUHVHDUFKVKRXOGEHFRQGXFWHGE\LQGHSHQGHQWGLVLQWHUHVWHGSDUWLHVXVLQJUHVSHFWHG
SV\FKRPHWULFWHFKQLTXHV)RUH[DPSOHWKH'DQLHOVRQIUDPHZRUNKDVEHHQVXEMHFWHGWRDQXPEHU
RIVXFKVWXGLHVLQFOXGLQJWKRVHFRQGXFWHGE\WKH0HDVXUHVRI(IIHFWLYH7HDFKLQJ0(73URMHFW
DQGWKH&RQVRUWLXPRQ&KLFDJR6FKRRO5HVHDUFK 

+LJKO\(YROYHG
$Q\HYDOXDWLRQV\VWHPXVHGIRUKLJKVWDNHVSHUVRQQHOGHFLVLRQVVKRXOGEHKLJKO\HYROYHG)RU
H[DPSOHGRHVLWFODULI\ZKDWZLOOVHUYHDVHYLGHQFHIRUHDFKLWHPLQWKHLQVWUXFWLRQDOIUDPHZRUN
VXFKDVREVHUYDWLRQVSODQQLQJGRFXPHQWVRUFRQIHUHQFHV"$UHWKHZRUGVLQWKHUXEULFFOHDU
HQRXJKWRHQDEOHERWKWHDFKHUVDQGVXSHUYLVRUVWRGLIIHUHQWLDWHRQHOHYHORISURILFLHQF\IURPWKH
QH[W"7KHODQJXDJHPXVWEHVXIILFLHQWO\SUHFLVHWRHQDEOHREVHUYHUVWROLQNVSHFLILFWHDFKHUV
RU
VWXGHQWV
ZRUGVRUDFWLRQVWRVSHFLILFHOHPHQWVRUFRPSRQHQWVRIWKHLQVWUXFWLRQDOIUDPHZRUN
,QGHILQLQJJRRGWHDFKLQJHGXFDWRUVPXVWDOVRWDNHLQWRDFFRXQWPDMRUGHYHORSPHQWVLQVWDWHDQG
QDWLRQDOSROLF\VXFKDVWKH&RPPRQ&RUH6WDWH6WDQGDUGVZKLFKVWDWHVDQGWKH'LVWULFWRI
&ROXPELDKDYHIRUPDOO\DGRSWHG7KHVWDQGDUGVUHODWHSULPDULO\WRZKDWVWXGHQWVZLOOOHDUQDQG
FRQVHTXHQWO\KDYHWKHLUJUHDWHVWLPSDFWRQLVVXHVRIFXUULFXOXPDQGVWXGHQWDVVHVVPHQW+RZHYHU
EHFDXVHWKHVWDQGDUGVHPSKDVL]HUHDVRQLQJDQGSUREOHPVROYLQJVNLOOVDVZHOODVGHYHORSLQJ
GHHSFRQFHSWXDOXQGHUVWDQGLQJWKH\KDYHLPSOLFDWLRQVIRULQVWUXFWLRQ7KHPHWKRGVDWHDFKHUXVHV
WRKHOSVWXGHQWVOHDUQWKHWHFKQLTXHVRIDUJXPHQWDWLRQIRUH[DPSOHDUHGLIIHUHQWIURPWKHPHWKRGV
KHRUVKHXVHVWRWHDFKORZOHYHONQRZOHGJHDQGVNLOOVE\URWH/HDUQLQJIDFWVVXFKDV6SDQLVK
YRFDEXODU\ZRUGVRUWKHPXOWLSOLFDWLRQWDEOHVGHPDQGVLQVWUXFWLRQWKDWIRFXVHVRQPHPRUL]LQJDQG
XVLQJPQHPRQLFGHYLFHV%XWWHDFKLQJVWXGHQWVWRIRUPXODWHDQGWHVWK\SRWKHVHVDQGWRWDNHDQG
GHIHQGDSRVLWLRQUHTXLUHVDEURDGUHSHUWRLUHRIWHDFKLQJVWUDWHJLHV
$GHILQLWLRQRIWHDFKLQJWKDW
VUHVSRQVLYHWRHYROYLQJFRQGLWLRQVLQWKHILHOGZLOOLPSRVHGLIIHUHQW
FKDOOHQJHVIRUREVHUYHUVRISUDFWLFH,W
VDG\QDPLFHQYLURQPHQWLQZKLFKWKHDVSHFWVRIWHDFKLQJ
GHHPHGLPSRUWDQWWRVWXGHQWOHDUQLQJDQGWRDSDUWLFXODUW\SHRIVWXGHQWOHDUQLQJ²QDPHO\KLJK
OHYHOVNLOOV²HYROYHRYHUWLPH

+DYLQJ&OHDU/HYHOVRI3HUIRUPDQFH
/HYHOVRISHUIRUPDQFHGHVFULEHKRZDWHDFKHU
VSUDFWLFHSURJUHVVHVIURPLQH[SHULHQFHGDQG
LQH[SHUWWRH[SHULHQFHGDQGH[SHUW:LWKUHVSHFWWRWKHVWDQGDUGVRISUDFWLFHLW
VQRWWKDWWHDFKHUV
HLWKHUGRWKHPRUGRQ
WGRWKHP²LW
VWKDWWKH\GRWKHPZHOORUSRRUO\7KHOHYHOVRISHUIRUPDQFH
GHVFULEHWKDWFRQWLQXXP
%HFDXVHWKHOHYHOVRISHUIRUPDQFHGHVFULEHDWHDFKHU
VVNLOOLQWKHYDULRXVDVSHFWVRIWHDFKLQJLW
V
HVVHQWLDOWKDWREVHUYHUVEHDEOHWRGLVWLQJXLVKRQHOHYHOIURPWKHQH[W7KLVLQWXUQPDNHVLWPRUH
OLNHO\WKDWDQ\WZRWUDLQHGREVHUYHUVZLOODJUHHZLWKHDFKRWKHU7KLVLVILUVWDPDWWHURIFODULW\RI
ODQJXDJHWKHODQJXDJHXVHGLQWKHGLIIHUHQWOHYHOVVKRXOGSHUPLWIRFXVHGWUDLQLQJIRUREVHUYHUVVR
WKHLUOHYHOVRIDJUHHPHQWDQGDFFXUDF\DUHKLJK
7KXVLQWKH'DQLHOVRQ)UDPHZRUNIRU7HDFKLQJDVWDWHPHQWDWWKHSURILFLHQWOHYHOLQ&RPSRQHQWF
HQJDJLQJVWXGHQWVLQOHDUQLQJVWDWHVWKDWWKHOHDUQLQJWDVNVDQGDFWLYLWLHVDUHGHVLJQHGWR
FKDOOHQJHVWXGHQWWKLQNLQJLQYLWLQJVWXGHQWVWRPDNHWKHLUWKLQNLQJYLVLEOH7KLVLVPRUHDGYDQFHG
WKDQZKDWWKHODQJXDJHGHVFULEHVDWWKHEDVLFOHYHO7KHOHDUQLQJWDVNVDQGDFWLYLWLHVUHTXLUHRQO\
PLQLPDOWKLQNLQJE\VWXGHQWVDQGOLWWOHRSSRUWXQLW\IRUWKHPWRH[SODLQWKHLUWKLQNLQJ7KHVH
GLIIHUHQFHVDUHFOHDUDQGPD\EHLOOXVWUDWHGE\VSHFLILFH[DPSOHVGXULQJREVHUYHUWUDLQLQJ
7KHUHDUHRWKHUFKDOOHQJHVFRQFHUQLQJFODULW\RIODQJXDJH6RPHUXEULFVXVHWKHODQJXDJHRI
IUHTXHQF\WHDFKHUVGRDFHUWDLQWKLQJQHYHURFFDVLRQDOO\IUHTXHQWO\RUDOZD\V7KLV
ODQJXDJHVXJJHVWVWKDWDQHYDOXDWRUFDQREVHUYHWKHVDPHWHDFKHUPXOWLSOHWLPHVLW
VQRWVXLWDEOH
IRUDVLQJOHREVHUYDWLRQRIWHDFKLQJ)RUUXEULFVWRDSSO\WRLQGLYLGXDOOHVVRQVWKHODQJXDJHLQWKH
GLIIHUHQWOHYHOVRISHUIRUPDQFHPXVWEHTXDOLWDWLYHO\QRWTXDQWLWDWLYHO\GLIIHUHQW)RULQVWDQFHLQWKH
H[DPSOHFLWHGOHDUQLQJWDVNVDWWKHSURILFLHQWOHYHODUHGHVLJQHGWRFKDOOHQJHVWXGHQWWKLQNLQJ
ZKHUHDVWKRVHDWWKHEDVLFOHYHOUHTXLUHRQO\PLQLPDOWKLQNLQJE\VWXGHQWV7KHVHDUHTXDOLWDWLYH
GLIIHUHQFHV
)LQDOO\WKHUXEULFVPXVWEHUREXVWHQRXJKWRZLWKVWDQGWKHGHPDQGVSODFHGRQWKHV\VWHPDVD
ZKROHLQSDUWLFXODULWPXVWEHSRVVLEOHWRWUDLQREVHUYHUVWRPDNHDFFXUDWHMXGJPHQWVUHJDUGLQJ
ZKDWWKH\VHHDQGKHDU)RUH[DPSOHDVDFRQVHTXHQFHRISDUWLFLSDWLQJLQWKH0(7VWXG\ZH
IRXQGZHKDGWRWLJKWHQWKHODQJXDJHLQWKHUXEULFVRIWKH)UDPHZRUNIRU7HDFKLQJWRDWWDLQKLJK
HQRXJKUDWHVRILQWHUUDWHUDJUHHPHQWDQGDFFXUDF\,WZDVQRWVXIILFLHQWWRVD\WKDWDWHDFKHU
GHPRQVWUDWHGDGHHSXQGHUVWDQGLQJRIWKHFRQWHQWUDWKHUUHYLVHGODQJXDJHVSHFLILHVWKDWD
WHDFKHUPXVWEHDEOHWRDUWLFXODWHFRQQHFWLRQVEHWZHHQWKHWRSLFEHLQJWDXJKWDQGRWKHUWRSLFV
ZLWKLQDQGRXWVLGHWKHGLVFLSOLQH)XUWKHUZHGLVFRYHUHGWKDWSURYLGLQJWHDFKHUH[DPSOHVRIWKH
OHYHOVRISHUIRUPDQFHIDFLOLWDWHGREVHUYHUWUDLQLQJ

7KH6NLOOV2EVHUYHUV1HHG
2EVHUYHUVQHHGWRDFTXLUHDQXPEHURIVNLOOVWRFRQGXFWIDLUDQGUHOLDEOHREVHUYDWLRQVRIWHDFKLQJ
7KH\QHHGWUDLQLQJDQGSRVVLEO\DQDVVHVVPHQWRIWKHLUVNLOOVWRHQVXUHWKH\FDQFRQGXFWWKHVH
REVHUYDWLRQVZLWKILGHOLW\6HYHUDOVWDWHVQRZUHTXLUHWKDWHYDOXDWRUVEHFHUWLILHGDVREVHUYHUV
EHIRUHEHLQJSHUPLWWHGWRHYDOXDWHWHDFKHUVIRUKLJKVWDNHVSHUVRQQHOGHFLVLRQV7KLVUHTXLUHPHQW
PDNHVJRRGVHQVH$IWHUDOO\RXFDQ
WREWDLQDGULYHU
VOLFHQVHZLWKRXWSDVVLQJDWHVW:K\VKRXOGD
VXSHUYLVRUEHDEOHWRPDNHKLJKVWDNHVSHUVRQQHOGHFLVLRQVZLWKRXWGHPRQVWUDWLQJWKHVNLOOWRGR
VRDFFXUDWHO\"
6RZKDWDUHWKRVHQHFHVVDU\VNLOOV"

&ROOHFWLQJ(YLGHQFH
:KHQREVHUYLQJLQDFODVVURRPHYDOXDWRUVPXVWQRWHZKDWWKH\VHHDQGKHDUWKHUH,W
VLPSRUWDQW
WKDWZKDWWKH\ZULWHGRZQDFWXDOO\LVHYLGHQFH²DQGQRWRSLQLRQLQWHUSUHWDWLRQRUELDV7KLVLVQRWD
VLPSOHPDWWHULW
VFKDOOHQJLQJWRUHFRUGMXVWWKHIDFWVPD
DP
7KHUHDUHWKUHHW\SHVRIHYLGHQFHZRUGVVSRNHQE\WKHWHDFKHURUVWXGHQWVVXFKDV&DQDQ\RQH
WKLQNRIDQRWKHULGHD"DFWLRQVVXFKDV7KHVWXGHQWVWRRNVHFRQGVWROLQHXSE\WKHGRRUDQG
WKHDSSHDUDQFHRIWKHFODVVURRPVXFKDV%DFNSDFNVDUHVWUHZQLQWKHPLGGOHRIWKHIORRU
%XWLW
VGLIILFXOWWRUHFRUGRQO\HYLGHQFH9LUWXDOO\DOOHGXFDWRUVILQGWKH\LQFOXGHVRPHLQWHUSUHWDWLRQ
RURSLQLRQLQWKHLUQRWHV)RUH[DPSOHDQREVHUYHUPLJKWQRWHWKDWWKHVWXGHQWVDUHHQJDJHG
GXULQJWKHVFLHQFHOHVVRQRQEXR\DQF\DQGGHQVLW\EXWWKDW
VQRWVWULFWO\VSHDNLQJHYLGHQFH,W
V
QRWZKDWWKHVWXGHQWVRUWHDFKHUVDLGRUGLG,QVWHDGLW
VDQLQWHUSUHWDWLRQRIZKDWWKHREVHUYHU
KHDUGDQGVDZ
:KDWWKHREVHUYHUDFWXDOO\VDZZDVVWXGHQWVIDVKLRQLQJWKHLUFOD\LQWRGLIIHUHQWVKDSHVOHDQLQJ
IRUZDUGLQWKHLUGLVFXVVLRQVZLWKRQHDQRWKHUDQGGUDZLQJVNHWFKHVRIWKHLUGHVLJQVRQWKHERDUG
7KRVHLWHPVZRXOGEHWKHHYLGHQFHZKLFKWKHREVHUYHUSUREDEO\FRUUHFWO\KDVLQWHUSUHWHGDV
VWXGHQWHQJDJHPHQW7KLVGLVWLQFWLRQLVLPSRUWDQWEHFDXVHZKHQREVHUYHUVGLVDJUHHDERXWD
WHDFKHU
VOHYHORISHUIRUPDQFHLW
VHVVHQWLDOWRNQRZZKHWKHUWKHGLIIHUHQFHVVWHPIURPDGLIIHUHQFH
LQWKHHYLGHQFHFROOHFWHGRULQKRZWKHREVHUYHUKDVLQWHUSUHWHGWKDWHYLGHQFH

,QWHUSUHWLQJ(YLGHQFH$JDLQVW/HYHOVRI3HUIRUPDQFH
7KHHYLGHQFHDQREVHUYHUFROOHFWVLQWKHFODVVURRPLVQRWLQLWVHOIJRRGRUEDG:KDWOHDGVWRD
MXGJPHQWDERXWWKHTXDOLW\RIWHDFKLQJLVLQWHUSUHWLQJWKDWHYLGHQFHDJDLQVWWKHUXEULFRUWKHOHYHOV
RISHUIRUPDQFH7KHTXHVWLRQIRUWKHREVHUYHULVQRWZKDWKDSSHQHGWKDW
VWKHHYLGHQFHEXWZKDW
GRHVLWPHDQ"7KDWLVZKLFKFROOHFWLRQRIZRUGVLQWKHUXEULFEHVWVXPPDUL]HVRUFKDUDFWHUL]HV
ZKDWWKHREVHUYHUREVHUYHG"
7KLVTXHVWLRQLVDWWKHKHDUWRIREVHUYHUWUDLQLQJ,W
VHVVHQWLDOWKDWGLIIHUHQWLQGLYLGXDOVXVLQJWKH
VDPHIUDPHZRUNFDQDJUHHRQWKHOHYHORITXDOLW\RIZKDWWKH\REVHUYH²WKDWLVWKDWWKH\VHOHFWWKH
VDPHOHYHORISHUIRUPDQFHIRUZKDWWKH\REVHUYHGIRUWKHVDPHUHDVRQ,IWKHVWXGHQWVRQWKHLURZQ
LQLWLDWLYHSXVKHGWKHLUGHVNVWRJHWKHUWRPDNHWDEOHVDQGJDWKHUHGWKHPDWHULDOVWKH\QHHGHGWKH
FOD\WKHWXEVRIZDWHUWKHVHLWHPVZRXOGEHHYLGHQFHRIKLJKOHYHOVRISHUIRUPDQFHRQ
&RPSRQHQWHLQWKH'DQLHOVRQ)UDPHZRUNRUJDQL]LQJSK\VLFDOVSDFHWKHDUUDQJHPHQWRI
IXUQLWXUHDQGXVHRISK\VLFDOUHVRXUFHVDQG&RPSRQHQWFPDQDJLQJFODVVURRPSURFHGXUHV
PDQDJHPHQWRIPDWHULDOVDQGVXSSOLHV%HFDXVHWKHVWXGHQWVGLGWKHVHWKLQJVRQWKHLURZQWKHLU
DFWLRQVZRXOGSURYLGHHYLGHQFHRIGLVWLQJXLVKHGSUDFWLFHRQWKHSDUWRIWKHWHDFKHUEHFDXVHWKH
WHDFKHUZRXOGKDYHHVWDEOLVKHGWKHVHURXWLQHVDQGWDXJKWWKHVWXGHQWVWRIROORZWKHP
2IFRXUVHIRUORZLQIHUHQFHLWHPVLW
VHDV\WRJHWKLJKOHYHOVRILQWHUUDWHUDJUHHPHQW2EVHUYHUV
FDQSUREDEO\DJUHHRQZKHWKHUWKHFODVVVWDUWHGRQWLPH%XWIRUDQ\WKLQJPRUHVLJQLILFDQWVXFKDV
ZKHWKHUWKHWHDFKHUXVHGTXHVWLRQLQJDQGGLVFXVVLRQWRGHHSHQXQGHUVWDQGLQJWKHUH
VOLNHO\WREH
OHVVFRQVHQVXVDPRQJREVHUYHUVHYHQDIWHUVRPHGHJUHHRIWUDLQLQJ

&RQGXFWLQJ3URIHVVLRQDO&RQYHUVDWLRQVZLWK7HDFKHUV
0DQ\VXSHUYLVRUVHYHQZKHQDGHTXDWHO\WUDLQHGWRFRQGXFWFODVVURRPREVHUYDWLRQVFRQIHVVWR
QRWNQRZLQJZKDWWRGRQH[W:KDWQRZ"WKH\VD\+RZGR,KDYHDFRQYHUVDWLRQZLWKDWHDFKHU
WKDWZLOOUHVXOWLQOHDUQLQJDQGLPSURYHGSUDFWLFH"
&OHDUO\WKHUH
VDUROHIRUIHHGEDFNDVLQ,QRWLFHG\RXGLUHFWHGWZRWKLUGVRI\RXUTXHVWLRQVWRZDUG
WKHULJKWKDQGVLGHRIWKHURRP:HUH\RXDZDUHRIWKDW"%XWWKHRYHUZKHOPLQJIRFXVRID
FRQYHUVDWLRQIROORZLQJDOHVVRQVKRXOGEHGLDORJXHZLWKDVKDULQJRIYLHZVDQGSHUVSHFWLYHV$IWHU
DOOWHDFKHUVPDNHKXQGUHGVRIGHFLVLRQVHYHU\GD\,IZHDFFHSWWKDWWHDFKLQJLVDPRQJRWKHU
WKLQJVFRJQLWLYHZRUNWKHQWKHFRQYHUVDWLRQVEHWZHHQWHDFKHUVDQGREVHUYHUVPXVWEHDERXWWKH
FRJQLWLRQ
5DWKHUWKDQEHLQJDQRSSRUWXQLW\IRUDVXSHUYLVRUWRVLPSO\WHOODWHDFKHUZKDWKHRUVKHWKRXJKW
DERXWWKHOHVVRQ,UHDOO\OLNHGWKHZD\\RXGLG;WKHFRQYHUVDWLRQVIROORZLQJDQREVHUYDWLRQDUH
WKHEHVWRSSRUWXQLW\WRHQJDJHWHDFKHUVLQWKLQNLQJWKURXJKKRZWKH\FRXOGVWUHQJWKHQWKHLU
SUDFWLFH7KHUHIRUHDFRPSUHKHQVLYHDSSURDFKWRREVHUYHUWUDLQLQJVKRXOGLQFOXGHDWWHQWLRQWRWKH
LQWHUDFWLYHVNLOOVRISURIHVVLRQDOFRQYHUVDWLRQLQYLWLQJWHDFKHUVWRUHIOHFWRQWKHLUSUDFWLFHDQG
VWUHQJWKHQLWLQZD\VGHVFULEHGE\WKHLQVWUXFWLRQDOIUDPHZRUNWKH\XVH

7KH7HDFKHUDV/HDUQHU
0DQ\WHDFKHUVKDYHEHHQYLFWLPVRIDQREVHUYDWLRQVXSHUYLVLRQDQGHYDOXDWLRQSURFHVVLQZKLFK
WKHREVHUYDWLRQZDVVRPHWKLQJGRQHWRUDWKHUWKDQZLWKWKHP7KLVLVDVKDPHDQGUHSUHVHQWVDQ
HQRUPRXVPLVVHGRSSRUWXQLW\
$OWKRXJKIHZWHDFKHUVW\SLFDOO\UHTXLUHUHPHGLDWLRQWKHYDVWPDMRULW\RIWHDFKHUVFDQVWUHQJWKHQ
WKHLUSHUIRUPDQFH,QIDFWEHFDXVHWHDFKLQJLVVRGHPDQGLQJDQGFRPSOH[DOOWHDFKLQJFDQEH
LPSURYHGQRPDWWHUKRZEULOOLDQWDOHVVRQLVLWFDQDOZD\VEHHYHQEHWWHU$QGXQOHVVZHXVHWKH
REVHUYDWLRQSURFHVVIRUWKDWSXUSRVHLW
VIDLUWRLQTXLUHZK\HGXFDWRUVHYHQHQJDJHLQLW
&RPSOLDQFHZLWKVWDWHODZPD\EHDQLPSRUWDQWOHJDOUHDVRQUHIOHFWLQJWKHDFNQRZOHGJHGQHHGWR
LGHQWLI\WKHIHZWUXO\XQGHUSHUIRUPLQJWHDFKHUV%XWLIZHGRQ
WXVHWKHREVHUYDWLRQSURFHVVWR
VWUHQJWKHQSUDFWLFHRYHUDOOWKHV\VWHPFDQ
WEHFDOOHGHGXFDWLYH
6RKRZGRVFKRROVPDNHDQREVHUYDWLRQSURFHVVDVHGXFDWLRQDODVSRVVLEOHIRUWHDFKHUV"
,QDQVZHULQJWKLVTXHVWLRQLW
VLPSRUWDQWWRUHFRJQL]HWKDWSURIHVVLRQDOOHDUQLQJLVOHDUQLQJ²DQG
WKDWOHDUQLQJUHTXLUHVWKHOHDUQHUWREHDQDFWLYHSDUWLFLSDQWLQWKHSURFHVV:LWKWKLVLQPLQGLW
V
LQVWUXFWLYHWRUHYLHZWKHW\SLFDOREVHUYDWLRQVFHQDULR+HUHWKHREVHUYHUJRHVWRWKHFODVVURRP
WDNHVQRWHVRQWKHHYHQWVRIWKHOHVVRQJRHVEDFNWRKLVRUKHURIILFHZULWHVXSWKHQRWHVDQGWKHQ
UHWXUQVWRWKHFODVVURRPDQGWHOOVWKHWHDFKHUDERXWWKHOHVVRQ6RPHWLPHVWKHVXSHUYLVRUGRHVQ
W
HYHQWDONZLWKWKHWHDFKHUEXWVLPSO\OHDYHVWKHREVHUYDWLRQUHSRUWLQWKHWHDFKHU
VPDLOER[
,QWKLVVFHQDULRWKHWHDFKHULVGRLQJQRWKLQJ²H[FHSWWHDFKLQJWKHFODVVZKLFKKHRUVKHLVXQGHU
FRQWUDFWWRGR,QWKHREVHUYDWLRQSURFHVVWKHWHDFKHULVFRPSOHWHO\SDVVLYH6RLW
VKDUGO\
VXUSULVLQJWKDWWHDFKHUVUDUHO\OHDUQPXFKIURPWKHSURFHVV

&KDQJLQJWKH6FULSW
+RZFRXOGZHVWUHQJWKHQWKHSURFHVVVRWKHWHDFKHUSOD\VDQDFWLYHUROH"/HW
VJREDFNWRWKH
FODVVURRPREVHUYDWLRQGHVFULEHGDWWKHEHJLQQLQJRIWKLVDUWLFOHDQGVHHKRZLWORRNVZKHQLW
V
EDVHGLQDFOHDUIUDPHZRUNDQGLWVJRDOLVWRVWUHQJWKHQWKHWHDFKHU
VSUDFWLFH
0V$QGHUVRQLVWDNLQJGHWDLOHGQRWHVDERXWZKDWVKHVHHVWKHVWXGHQWVGRLQJLQWKHFODVVURRP
7KH\
UHFUHDWLQJGLIIHUHQWVKDSHVIRUWKHLUERDWVUHVSHFWIXOO\FKDOOHQJLQJRQHDQRWKHUWRWU\GLIIHUHQW
GHVLJQVDGGLQJSDSHUFOLSVXQWLOWKHERDWVVLQNDQGGUDZLQJWKHLUGHVLJQVRQWKHERDUG6KH
VDOVR
QRWLQJZKDWWKHWHDFKHULVGRLQJ-HQQLIHULVFLUFXODWLQJDPRQJWKHVWXGHQWVFKDOOHQJLQJWKHPWR
FRQVLGHURWKHUDOWHUQDWLYHV%HFDXVHWKH\ERWKXQGHUVWDQGWKHLQVWUXFWLRQDOIUDPHZRUNWKH\NQRZ
WKDWWKHVWXGHQWV
ERDWGHVLJQVDUHHYLGHQFHRI&RPSRQHQWFHQJDJLQJVWXGHQWVLQOHDUQLQJDQG
WKDW-HQQLIHU
VFLUFXODWLQJDPRQJWKHVWXGHQWVRIIHULQJLQVLJKWVDQGIHHGEDFNLVHYLGHQFHRI
&RPSRQHQWGXVLQJDVVHVVPHQWLQLQVWUXFWLRQ
$IWHUZDUG0V$QGHUVRQJLYHVDFRS\RIKHUQRWHVWR-HQQLIHU-HQQLIHUORRNVWKHPRYHUDQGSRLQWV
RXWWKDWDIWHUPDNLQJDQHZGHVLJQIRUDERDWWKHVWXGHQWVZHUHDOVRSUHGLFWLQJKRZPDQ\SDSHU
FOLSVLWZRXOGKROGEHIRUHVLQNLQJ0V$QGHUVRQDGGVWKLVSLHFHRILQIRUPDWLRQWRKHUQRWHV
(DFKRIWKHPWKHQWDNHVKHUQRWHVDQGDOLJQVHDFKSLHFHRIHYLGHQFHWRFRPSRQHQWVLQ'RPDLQV
DQGRIWKH)UDPHZRUNIRU7HDFKLQJGHWHUPLQLQJZKLFKOHYHORISHUIRUPDQFH²XQVDWLVIDFWRU\
EDVLFSURILFLHQWRUGLVWLQJXLVKHG²WKH\WKLQNWKHHYLGHQFHUHIOHFWVOLQNLQJWKHHYLGHQFHIRUHDFK
FRPSRQHQWWRWKHODQJXDJHRIWKHUXEULFDQGWKHFUXFLDODWWULEXWHV7KH\GRWKLVZRUNLQGHSHQGHQWO\
LQSUHSDUDWLRQIRUWKHLUFRQYHUVDWLRQKLJKOLJKWLQJWKHZRUGVLQWKHUXEULFWKH\WKLQNEHVWFKDUDFWHUL]H
WKHHYLGHQFH
7KHQH[WGD\WKHWZRPHHWIRUWKHLUSRVWREVHUYDWLRQRUUHIOHFWLRQFRQIHUHQFHLQZKLFKWKH\
FRPSDUHWKHLUKLJKOLJKWVDQGGLVFXVVWKHUDWLRQDOHIRUKDYLQJVHOHFWHGWKHSDUWLFXODUOHYHOVRI
SHUIRUPDQFH)RUH[DPSOHWKH\DJUHHWKDWEHFDXVHYLUWXDOO\DOOWKHVWXGHQWVZHUHLQWHOOHFWXDOO\
HQJDJHGLQWKHDFWLYLW\-HQQLIHU
VSHUIRUPDQFHIRU&RPSRQHQWFLVDWWKHGLVWLQJXLVKHGOHYHO
7KHIUDPHZRUNGRFXPHQWUHSUHVHQWVDWKLUGSRLQWEHWZHHQWKHWHDFKHUDQGWKHREVHUYHU7KDWLV
WKHREVHUYHULVQRWPHUHO\UHSRUWLQJWRWKHWHDFKHUZKDWKHRUVKHWKRXJKWDERXWWKHOHVVRQEXWLV
DOVRUHODWLQJVSHFLILFHYLGHQFHIURPWKHOHVVRQWRVSHFLILFZRUGVDQGSKUDVHVLQWKHOHYHOVRI
SHUIRUPDQFH
)XUWKHUWKHREVHUYHUPXVWEHVXIILFLHQWO\RSHQPLQGHGWRDGMXVWKLVRUKHULQWHUSUHWDWLRQRIWKH
HYLGHQFHLIWKHWHDFKHUPDNHVDFRQYLQFLQJFDVHIRUDQDOWHUQDWLYHYLHZ$IWHUDOOWKHREVHUYHUFDQ
W
EHWKHUHHYHU\GD\GRHVQ
WNQRZZKDWKDSSHQHGWKHGD\EHIRUHDQGGRHVQ
WNQRZKRZDFHUWDLQ
VWXGHQWXVXDOO\EHKDYHV)RUH[DPSOH-HQQLIHUNQRZVWKDWWKHVWXGHQWV
UHVSHFWIXOIHHGEDFNWRRQH
DQRWKHUUHSUHVHQWVDELJVWHSIRUWKHPLQ&RPSRQHQWDFUHDWLQJDQHQYLURQPHQWRIUHVSHFWDQG
UDSSRUWDQGVKHSRLQWVWKLVRXWWR0V$QGHUVRQ

:KDW
V3RVVLEOH
9LUWXDOO\HYHU\VWDWHUHTXLUHVREVHUYDWLRQVRIWHDFKLQJDVDVLJQLILFDQWFRQWULEXWRUWRKLJKVWDNHV
MXGJPHQWVDERXWWHDFKHUTXDOLW\7REHGHIHQVLEOHWKHV\VWHPVWKDW\LHOGWKHVHREVHUYDWLRQVPXVW
KDYHFOHDUVWDQGDUGVRISUDFWLFHLQVWUXPHQWVDQGSURFHGXUHVWKURXJKZKLFKWHDFKHUVFDQ
GHPRQVWUDWHWKHLUVNLOODQGWUDLQHGDQGFHUWLILHGREVHUYHUVZKRFDQPDNHDFFXUDWHDQGFRQVLVWHQW
MXGJPHQWVEDVHGRQHYLGHQFH
,QDGGLWLRQLW
VSRVVLEOHWRGHVLJQDSSURDFKHVWRFODVVURRPREVHUYDWLRQWKDW\LHOGLPSRUWDQW
OHDUQLQJIRUWHDFKHUVE\LQFRUSRUDWLQJSUDFWLFHVDVVRFLDWHGZLWKSURIHVVLRQDOOHDUQLQJ²QDPHO\
VHOIDVVHVVPHQWUHIOHFWLRQRQSUDFWLFHDQGSURIHVVLRQDOFRQYHUVDWLRQ:KHQWKHVHSUDFWLFHVDUH
SXWLQWRSODFHFODVVURRPREVHUYDWLRQFDQPDNHDGUDPDWLFFRQWULEXWLRQWRWKHFXOWXUHRIDVFKRRO

(QGQRWHV
)RUDFRPSOHWHGLVFXVVLRQRIWKHIUDPHZRUNVHHP\ERRN(QKDQFLQJ3URIHVVLRQDO3UDFWLFH$

)UDPHZRUNIRU7HDFKLQJ$6&'

7KHVHVWXGLHVLQFOXGH5HWKLQNLQJ7HDFKHU(YDOXDWLRQLQ&KLFDJR/HVVRQV/HDUQHGIURP

&ODVVURRP2EVHUYDWLRQV3ULQFLSDO7HDFKHU&RQIHUHQFHVDQG'LVWULFW,PSOHPHQWDWLRQ
&RQVRUWLXPRQ&KLFDJR6FKRRO5HVHDUFKDWWKH8QLYHUVLW\RI&KLFDJR8UEDQ(GXFDWLRQ,QVWLWXWH
1RYHPEHUDQG*DWKHULQJ)HHGEDFNIRU7HDFKLQJ&RPELQLQJ+LJK4XDOLW\2EVHUYDWLRQV
ZLWK6WXGHQW6XUYH\VDQG$FKLHYHPHQW*DLQV%LOODQG0HOLQGD*DWHV)RXQGDWLRQ

&KDUORWWH'DQLHOVRQLVDQLQWHUQDWLRQDOO\UHFRJQL]HGH[SHUWLQWKHDUHDRIWHDFKHUHIIHFWLYHQHVV6KHDGYLVHV
VWDWHHGXFDWLRQGHSDUWPHQWVDQGQDWLRQDOPLQLVWULHVDQGGHSDUWPHQWVRIHGXFDWLRQERWKLQWKH8QLWHG6WDWHV
DQGLQWHUQDWLRQDOO\
&RS\ULJKW‹E\&KDUORWWH'DQLHOVRQ