IEEE ICC 2013 - Selected Areas in Communications Symposium

%DG'DWD'HWHFWLRQ0HWKRGIRU6PDUW*ULGV
EDVHGRQ'LVWULEXWHG6WDWH(VWLPDWLRQ

<XQ*X7LQJ/LX
'DL:DQJ;LDRKRQJ*XDQ=KDQER;X
0LQLVWU\RI(GXFDWLRQ.H\/DEIRU,QWHOOLJHQW1HWZRUNVDQG1HWZRUN6HFXULW\
6FKRRORI(OHFWURQLFDQG,QIRUPDWLRQ(QJLQHHULQJ;L¶DQ-LDRWRQJ8QLYHUVLW\
;L¶DQ6KDDQ[L&KLQD
^\JXWOLXGDLZDQJ[KJXDQ]E[X`#VHL[MWXHGXFQ


$EVWUDFW²%DG 'DWD ,QMHFWLRQ %', LQ 6PDUW *ULG LV
FRQVLGHUHGWREHWKH PRVW GDQJHURXVF\EHUDWWDFNDVLW PLJKW
OHDG WR HQHUJ\ WKHIW RQ WKH HQG XVHUV IDOVH GLVSDWFK RQ WKH
GLVWULEXWLRQ SURFHVV DQG GHYLFH EUHDNGRZQ RQ WKH SRZHU
JHQHUDWLRQ 6WDWH (VWLPDWLRQ DQG %DG 'DWD 'HWHFWLRQ ZKLFK
DUH DSSOLHG WR UHGXFH WKH REVHUYDWLRQ HUURUV DQG GHWHFW IDOVH
GDWD LQ WKH WUDGLWLRQDO SRZHU JULG FRXOG QRW GHWHFW WKH EDG
GDWD LQ VPDUW JULG ,Q WKLV SDSHU WKUHH %', DWWDFN FDVHV LQ
,(((EXVV\VWHPDUHGHVLJQHGWRE\SDVVWKHWUDGLWLRQDOEDG
GDWD GHWHFWLRQ 7KH SRWHQWLDO ULVNV RQ HFRQRP\ DQG VHFXULW\
DUH DQDO\]HG H[SORLWLQJ WKH 0$732:(5 $ QHZ PHWKRG
SUREOHPLQ;LDQJHWDO>@SUHVHQWHGDQDSSURDFKIRU
GHWHFWLRQ DQG LGHQWLILFDWLRQ RI PXOWLSOH EDG GDWD 7KLV
DSSURDFK ZDV GHYHORSHG LQ RUGHU WR LPSURYH WKH
SHUIRUPDQFH RI VWDWLF VWDWH HVWLPDWRUV RI SRZHU V\VWHPV
&XWVHP DQG 3DYHOOD >@ SUHVHQWHG DQ LGHQWLILFDWLRQ PHWKRG
ZKLFK DWWHPSWV WR DOOHYLDWH VRPH GLIILFXOWLHV VXFK DV
PXOWLSOH DQG LQWHUDFWLQJ EDG GDWD 7ZR LGHQWLILFDWLRQ
WHFKQLTXHV ZHUH GHULYHG DQG IXUWKHU LQYHVWLJDWHG DQG
DVVHVVHG E\ PHDQV RI D UHDOLVWLF LOOXVWUDWLYH H[DPSOH ,W LV
EHOLHYHG WKDW WKHVH WHFKQLTXHV DUH VXIILFLHQW WR GHWHFW DQG
EDVHG RQ 'LVWULEXWHG 6WDWH (VWLPDWLRQ '6( LV SURSRVHG WR
GHWHFW %', QDPHG DV '6(EDVHG EDG GDWD GHWHFWLRQ 7KH
SRZHU V\VWHP LV GLYLGHG LQWR VHYHUDO VXEV\VWHPV DQG D &KL
VTXDUHV WHVW LV DSSOLHG WR GHWHFW WKH EDG GDWD UHVSHFWLYHO\ LQ
HDFKVXEV\VWHP6LPXODWLRQUHVXOWVGHPRQVWUDWHWKDWWKH'6(
EDVHGEDGGDWDGHWHFWLRQFDQGHWHFWDOOEDGGDWDLQWKUHHDWWDFN
FDVHV 0RUHRYHU LW FDQ ORFDWH WKH EDG GDWD LQ VSHFLILF
VXEV\VWHPZKLFKLVKHOSIXOIRUWKHIXUWKHULGHQWLILFDWLRQ
.H\ZRUGV ² 6PDUW *ULG VHFXULW\ EDG GDWD GHWHFWLRQ
GLVWULEXWHGVWDWHHVWLPDWLRQEDGGDWDLQMHFWLRQ
,,1752'8&7,21
,QVPDUWJULGVWKHLQIRUPDWLRQWHFKQLTXHVDUHDSSOLHGWR
WKH SRZHU V\VWHP WR SURYLGH D GHVLUDEOH LQIUDVWUXFWXUH IRU
UHDOWLPH PHDVXUHPHQW WUDQVPLVVLRQ GHFLVLRQ DQG FRQWURO
9DULRXV DWWDFNV VXFK DV HDYHVGURSSLQJ LQIRUPDWLRQ
UHFRYHUIURPVHQVRUPHDVXUHPHQWPDQLSXODWLRQ
+RZHYHU /LX HW DO >@ GHPRQVWUDWHG WKDW DQ DGYHUVDU\
DUPHG ZLWK WKH NQRZOHGJH RI WKH QHWZRUN FRQILJXUDWLRQV
FDQDFFRPSOLVKPDOLFLRXVGDWDDWWDFNRQVWDWHHVWLPDWLRQWKDW
XVHV '& SRZHU IORZ PRGHOV ZLWKRXW EHLQJ GHWHFWHG 6LQFH
/LX HW DO ILUVWO\ DGGUHVVHG WKH H[LVWHQFH RI ³XQGHWHFWDEOH´
F\EHUDWWDFN RQ SRZHU V\VWHP VWDWH HVWLPDWLRQ ZKLFK KDV
DWWUDFWHGDORWRIUHVHDUFKHIIRUWVLQUHFHQW\HDUV6RPHQHZ
PHWKRGV KDYH EHHQ SURSRVHG WR VROYH WKH DERYH SUREOHP
ZKLFK FDQQRW EH GHWHFWHG E\ WUDGLWLRQDO EDG GDWD GHWHFWLRQ
.RVXW DQG KLV FROOHDJXHV >@ GLYLGHG PDOLFLRXV DWWDFNV LQWR
WZR UHJLPHV WKH VWURQJ DWWDFN UHJLPH DQG WKH ZHDN DWWDFN
UHJLPH7KH\VWXGLHGWKH SDUDPHWHUZKLFKLVWKHVL]HRIWKH
VPDOOHVW XQREVHUYDEOH PDOLFLRXV GDWD DWWDFN 7KH\ DOVR
VWXGLHGWKHJHQHUDOL]HGOLNHOLKRRGUDWLRWHVWDVDGHWHFWRUDQG
SURYLGHG UHVLGXH HQHUJ\ KHXULVWLF WR ILQG SDUWLFXODUO\
WDPSHULQJDQGPDOLFLRXVSURJUDPVWKDWKDYHDOPRVWUXLQHG
WKH ,QWHUQHW ZRXOG LPSRVH JUHDW WKUHDW RQ VHFXULW\ DQG
VWDELOLW\ RI VPDUW JULGV 6LQFH WKH GDWD LQ VPDUW JULG FDQ
HDVLO\ EH PRQHWL]HG WKH EDG GDWD LQMHFWLRQ LV H[WUHPHO\
DWWUDFWLYH IRU KDFNHUV HJ WKH KDFNHUV FDQ PDQLSXODWH WKHLU
GDPDJLQJ DWWDFNV LQ WKLV UHJLPH ;LH HW DO >@ SUHVHQWHG D
SRWHQWLDOFODVVRIF\EHUDWWDFNDJDLQVWWKHVWDWHHVWLPDWLRQLQ
GHUHJXODWHGHOHFWULFLW\PDUNHWVDQGVKRZHGWKDWVXFKDWWDFNV
ZLOO OHDG WR SURILWDEOH ILQDQFLDO PLVFRQGXFW 3DVTXDOHWWL >@
SURSRVHGDQDWWDFNPRGHOZKLFKJHQHUDOL]HVWKHSURWRW\SLFDO
HQHUJ\ FRVWV E\ PRGLI\LQJ WKH VPDUW PHWHU UHDGLQJV
)XUWKHUPRUH WKH EDGGDWD ZRXOG PLVOHDG WKH FRQWUROFHQWHU
WRWDNHHUURQHRXVDFWLRQVWKXVLWLVH[WUHPHO\GDQJHURXVIRU
VPDUW JULGV )RU H[DPSOH LQ  ,UDQ
V %XVKHKU QXFOHDU FRQWURO WKHRU\ 7KH VWXGLHV DERYH KDYH VXFFHVVIXOO\ SURYHG
SODQWZDVFUDFNHGE\6WX[QHWZRUPZKLFKXSGDWHGWKHIDOVH
V\VWHP VWDWH WR 6&$'$ WR VWRS WKH V\VWHP SURWHFWLRQ
VWUDWHJLHV
3RZHU V\VWHP VWDWHHVWLPDWLRQ DQG EDG GDWD GHWHFWLRQ LV
FRQVLGHUHGDVDJRRGVROXWLRQWRGHWHFWLQMHFWLRQGDWDZKLFK
LV DSSOLHG WR UHGXFH WKH REVHUYDWLRQ HUURUV DQG GHWHFW IDOVH
GDWD LQ WKH 6&$'$ 6FKZHSSH >@ ILUVWO\ LQWURGXFHG WKH
VWHDOWK IDOVHGDWD LQMHFWLRQ DQG UHSOD\ DWWDFNV 7R VROYH WKH
SUREOHP WKH\ GHVLJQHG SURYDEO\FRUUHFW GHWHFWLRQ DQG
LGHQWLILFDWLRQ SURFHGXUHV EDVHG RQ WRROV IURP JHRPHWULF
WKHH[LVWHQFHRI³XQGHWHFWDEOH´DWWDFNVDQGWKHOLPLWDWLRQVRI
VWDWHHVWLPDWLRQLQVPDUWJULGV
,QWKLVSDSHUZHSUHVHQWVHYHUDOEDGGDWDLQMHFWLRQFDVHV
LQ ,((( EXV V\VWHP ZKLFK FDQQRW EH GHWHFWHG E\ WKH
WUDGLWLRQDO VWDWH HVWLPDWLRQ 7KH SRWHQWLDO ULVNV VXFK DV
HQHUJ\WKHIWDQGFUDFNLQJHFRQRPLFGLVSDWFKDUHDQDO\]HGWR
FDOFXODWHWKHSRVVLEOHKDUPV7KHGLVWULEXWHGVWDWHHVWLPDWLRQ
FRQFHSW RI SRZHU V\VWHP VWDWH HVWLPDWLRQ DQG HPSOR\HG '6(PHWKRGSURSRVHGLQWKLVSDSHULVDSSOLHGWRGHWHFWEDG
:HLJKWHG /HDVW 6TXDUHV :/6 PHWKRG WR VROYH WKLV GDWD:LWKWKHSURSRVHGPHWKRGWKHSRZHUV\VWHPLVGLYLGHG

&RUUHVSRQGLQJDXWKRU
7KLV ZRUN ZDV VXSSRUWHG E\ WKH 1DWLRQDO 1DWXUDO 6FLHQFH )RXQGDWLRQ
   'RFWRUDO )XQG RI 0LQLVWU\ RI
(GXFDWLRQRI&KLQDWKH)XQGDPHQWDO5HVHDUFK)XQGVIRU
WKH&HQWUDO8QLYHUVLWLHVDQGWKH&LVFR8QLYHUVLW\5HVHDUFK3URJUDP)XQGD
FRUSRUDWHDGYLVHGIXQGRI6LOLFRQ9DOOH\&RPPXQLW\)RXQGDWLRQ

978-1-4673-3122-7/13/$31.00 ©2013 IEEE 4483

LQWR VHYHUDO VXEV\VWHPV DFFRUGLQJ WR WKH SK\VLFDO WRSRORJ\ VWDWH YDULDEOHV DUH UHODWHGWR WKH PHDVXUHPHQWV DV VKRZQ LQ
RUXVLQJFOXVWHULQJDOJRULWKPV&KLVTXDUHVWHVWLVDSSOLHGWR 
GHWHFW ZKHWKHU WKHUH LV DQ\ EDG GDWD UHVSHFWLYHO\ LQ HDFK
VXEV\VWHP7KHWHVWLQJUHVXOWVGHPRQVWUDWH WKDW DOO EDGGDWD  ] K[  H 
FDQEHGHWHFWHGLQWKUHHDWWDFNFDVHVDQGORFDWHGLQDVSHFLILF
VXEV\VWHPZLWKWKHSURSRVHG'6(EDVHGEDGGDWDGHWHFWLRQ ZKHUH [ LV WKH VWDWH YDULDEOHV DQG ] LV WKH PHWHU
,QVXPPDU\WKHPDLQFRQWULEXWLRQRIWKLVSDSHULVWZRIROG PHDVXUHPHQWV K[ >K[ [« [Q « KP[ [« [Q@7
6HYHUDOEDGGDWDLQMHFWLRQFDVHVDUHVLPXODWHGLQ,(((
EXVV\VWHP,WLVWKHILUVWWLPHWRGHPRQVWUDWHVXFKDWWDFNV
DQGFDOFXODWHWKHSRWHQWLDOULVNVIRUVPDUWJULGV
 '6( LV ILUVWO\ DSSOLHG LQ F\EHUDWWDFN GHWHFWLRQ 7KH
H[SHULPHQWVSURYHWKDW'6(EDVHGEDGGDWDGHWHFWLRQSUHVHQW
KLJKHUGHWHFWLRQDFFXUDF\DQGPXFKORZHUFRPSXWDWLRQFRVW
WKDQWKHPZLWKWKHWUDGLWLRQDOPHWKRGV
7KH UHVW RI WKLV SDSHU LV RUJDQL]HG DV IROORZV 7KH
EDFNJURXQG RI VWDWH HVWLPDWLRQ DQG EDG GDWD GHWHFWLRQ DUH
LQWURGXFHG LQ 6HFWLRQ ,, ,Q 6HFWLRQ ,,, D F\EHUDWWDFN
ZKHUH K,[ [« [Q LV D IXQFWLRQ RI [ [« [Q H 
>HH«HP@7LVWKHPHDVXUHPHQWQRLVHZKLFKLVDVVXPHGWR
IROORZ*DXVVLDQGLVWULEXWLRQRI]HURPHDQ7KLVDVVXPSWLRQ
LV JHQHUDOO\ DFFHSWHG LQ SRZHU V\VWHP VWDWH HVWLPDWLRQ
IRUPXODWLRQ>@
(VVHQWLDOO\ SRZHU V\VWHP VWDWH HVWLPDWLRQ LV D SURFHVV
ZKLFK XVHV UHDOWLPH UHGXQGDQW PHDVXUHPHQWV WR LPSURYH
GDWD DFFXUDF\ DQG DXWRPDWLFDOO\ H[FOXGHG IURP WKH HUURU
PHVVDJHFDXVHGE\UDQGRPLQWHUIHUHQFH7KHREMHFWLYHLVWR
ILQGDQHVWLPDWH[A RI[WKDWLVWKHEHVWILWRIWKHPHDVXUHPHQW
VFHQDULRLVLOOXVWUDWHGWRVKRZWKH SRWHQWLDOPRWLYDWLRQVDQG ] DFFRUGLQJ WR  7KH SUREOHP LV XVXDOO\ VROYHG E\ WKH
ULVNV 7KH PHWKRGRORJ\ RI F\EHUDWWDFN GHWHFWLRQ EDVHG RQ :/6$OJRULWKP>@7KHVWDWHHVWLPDWLRQFDQEHIRUPXODWHG
GLVWULEXWHG VWDWH HVWLPDWLRQ LV LQWURGXFHG LQ 6HFWLRQ ,9 ,Q DVDTXDGUDWLFRSWLPL]DWLRQSUREOHP
6HFWLRQ9WKHSURSRVHGPHWKRGLVWHVWHGZLWK,(((EXV
V\VWHP 7KH UHVXOWV DQG DQDO\VLV DUH DOVR VKRZQ LQ WKLV  - [ = >] − K[@7 5 −>]  K[@  
VHFWLRQ7KHFRQFOXGLQJUHPDUNVWKHQIROORZ
,,35(/,0,1$5,(6 ZKHUH 5  LV WKH PHDVXUHPHQW LQYHUVH FRYDULDQFH PDWUL[
7KHQRWDWLRQVDUHOLVWHGLQ7$%/(, 1HZWRZQ¶V PHWKRG FDQ EH DSSOLHG WR VROYH WKH TXDGUDWLF
RSWLPL]DWLRQSUREOHP
7$%/(, 127$7,216,17+,63$3(5 
1RWDWLR %%DG'DWD'HWHFWLRQ
'HILQLWLRQ
Q
7KHUH DUH VHYHUDO FRPPRQ PHWKRGV IRU EDG GDWD
SVXE 7KHQXPEHURIVXEV\VWHPVDIWHUGHFRPSRVLWLRQ
VXEV\VBN 7KHODEHORIVXEV\VWHPDIWHUGHFRPSRVLWLRQ”N”SVXE GHWHFWLRQVXFKDV&KLVTXDUHVWHVWDQGQRUPDOL]HGUHVLGXDOV
1 7KHQXPEHURIEXVHVLQDSRZHUV\VWHP PHWKRG>@7KHPRVWFRPPRQ PHWKRG&KLVTXDUHVWHVWLV
0 7KHQXPEHURIWUDQVPLVVLRQOLQHVLQDSRZHUV\VWHP XVHG IRUGHWHFWLQJ EDGGDWD LQ WKLV SDSHU ,WLVDVVXPHGWKDW
07,( 7KHQXPEHURIWLHOLQHV DOOWKHVWDWHYDULDEOHVDUHPXWXDOO\LQGHSHQGHQWDQGWKHPHWHU
1N 7KHQXPEHURIEXVHVLQVXEV\VBN
HUURUV IROORZ WKH QRUPDO GLVWULEXWLRQ ,W FDQ EH VKRZQ WKDW
0S 7KHQXPEHURIWUDQVPLVVLRQOLQHVLQVXEV\VBN
/RDG%XV*HQHUDWRUVLQSRZHUV\VWHPODEHOHGDFFRUGLQJWR P
 ] − K [ 
EXVL
WKHGHILQLWLRQLQ,(((VWDQGDUGFDVH”L”1 ¦ L L
L = σL
IROORZV D χ P − Q  GLVWULEXWLRQ ZKHUH PQ LV
/LM 7KHWUDQVPLVVLRQOLQHFRQQHFWLQJEXVLDQGEXVM
3LM 7KHUHDOOLQHSRZHUIORZIURPEXVLWREXVMREVHUYHGRQEXVL WKHGHJUHHRIIUHHGRP
[ 6WDWHYDULDEOHVLQSRZHUV\VWHP [ = > [  [  [Q @7  7KHVWHSVRIWKH&KLVTXDUHVWHVWDUHJLYHQDVIROORZV
] 0HDVXUHPHQWVLQSRZHUV\VWHP ] = > ]  ]  ]P @7   6ROYH WKH :/6 HVWLPDWLRQ SUREOHP DQG FRPSXWH WKH
H 0HDVXUHPHQWVQRLVHH >HH«HP@7
REMHFWLYHIXQFWLRQ
5 7KHGLDJRQDOPHDVXUHPHQWFRYDULDQFHPDWUL[
K[
7KH QRQOLQHDU IXQFWLRQ UHODWLQJ PHDVXUHPHQWV ] WR VWDWH P
 ] − K [  
YDULDEOHV[ - [  = ¦ L L  
R 7KHGHJUHHRIIUHHGRPLQSRZHUV\VWHP L = σL
7KH WKUHVKROG RI R GHJUHH RI IUHHGRP FRUUHVSRQGLQJ WR D
7RS
GHWHFWLRQFRQILGHQFHZLWKSUREDELOLW\S
 /RRN XS WKH YDOXH IURP WKH &KLVTXDUHV GLVWULEXWLRQ
[N 6WDWHVYDULDEOHVLQVXEV\VBN [ N = > [N  [N  [QN @7 
WDEOH FRUUHVSRQGLQJ WR D GHWHFWLRQ FRQILGHQFH ZLWK
]N 0HDVXUHPHQWVLQVXEV\VBN ] N = > ]N  ] N  ]PN @7  SUREDELOLW\ S HJ DQG PQ GHJUHHVRI IUHHGRP /HW
KN[N
7KH QRQOLQHDU IXQFWLRQ UHODWLQJ PHDVXUHPHQWV ]N WR VWDWHV
WKLV YDOXH EH χ   +HUH S = 3U - [  ≤ χ 
 P − Q  S   7KH
 P − Q  S
YDULDEOHV[NLQVXEV\VBN
RN 7KHGHJUHHRIIUHHGRPLQVXEV\VBN WKUHVKROG LQ WKLV SDSHU LV D YDOXH ZKLFK FRUUHVSRQGV WR D
DN 7KHIDOVHGDWDLQMHFWLRQDWWDFNLIH[LVWVLQVXEV\VBN GHWHFWLRQFRQILGHQFHZLWKDQGPQGHJUHHVRIIUHHGRP
$:/66WDWH(VWLPDWLRQ 7HVW - [  ≥ χ   ,I \HV WKHQ EDG GDWD ZLOO EH
 P − Q  S
3RZHU V\VWHP VWDWH HVWLPDWLRQ LV ZLGHO\ XVHG WR HQVXUH VXVSHFWHG(OVHWKHPHDVXUHPHQWVZLOOEHDVVXPHGWREHIUHH
WKH VDIHW\ DQG HFRQRP\ RI RSHUDWLRQ RI SRZHU V\VWHP 7KH RIEDGGDWD

4484
 ,,,$77$&.&$6(
$QDWWDFNFDVHRQ,(((EXVV\VWHPLVVKRZQLQ)LJ
WRH[SODLQKRZWKHLQMHFWHGEDGGDWDDYRLGGHWHFWLRQDQGWKH
SRWHQWLDO ULVNV RI WKH DWWDFN 7KH RULJLQDO ORDG RQ EXV DQG
EXV DUH 0: DQG 0: UHSHFWLYHO\ 7KH SRZHU
IORZRQWKHWUDQVPLVVLRQOLQH/LV0:,QWKHDWWDFN
5LVN &UDFNLQJ(FRQRPLF'LVSDWFK
(FRQRPLF GLVSDWFK LV WKH VKRUWWHUP GHWHUPLQDWLRQ
DERXW WKH RSWLPDO RXWSXW RI D QXPEHU RI HOHFWULFLW\
JHQHUDWLRQ IDFLOLWLHV ZKLFK LV WR PLQLPL]H WKH RYHUDOO
RSHUDWLQJ FRVW ZKLOH VDWLVI\LQJ WKH SRZHU ORDG RI V\VWHP
DFFRUGLQJ WR D UREXVW DQG UHOLDEOH PDQQHU 7R DFKLHYH WKH
FDVHWKHKDFNHUVWU\WRPRYH0:RISRZHUORDGIURP
EXVWREXV7KXVWKHORDGRQEXVDQGEXVDQGWKHSRZHU
IORZRQWKHWUDQVPLVVLRQOLQH/DUHPRGLILHGWR0:
0: DQG 0: UHVSHFWLYHO\ WR NHHS WKH SRZHU
EDODQFHRIWKHVHEXVHV7KHUHYLVHGGDWDLVDQDO\]HGZLWKWKH
JRDORIHFRQRPLFGLVSDWFKWKH2SWLPDO3RZHU)ORZ23)LV
DSSOLHG WR VROYH WKH ORDG IORZ DQG GHWHUPLQH D QHZ VHW RI
YDOXHVIRUJHQHUDWRU¶VRXWSXWWKDWUHGXFHVWKHJHQHUDWLRQFRVW
>@ :H FDOFXODWH WKH WRWDO JHQHUDWLRQ FRVW RI WKLV UHJLRQ
ZLWK 0$732:(5 D WRROER[ ZKLFK LV GHYHORSHG E\ WKH
VWDWHHVWLPDWLRQDQG-[A LVHTXDOWRE\VROYLQJ
ZKLFK LVOHVV WKDQ WKH WKUHVKROG 6R WUDGLWLRQDO EDG
GDWD GHWHFWLRQ IDLOV WR GHWHFW WKLV DWWDFN DQG WKH
PHDVXUHPHQWVZLOOEHDVVXPHGWREHIUHHRIEDGGDWD
&RUQHOO8QLYHUVLW\>@,QWKHQRUPDOVLWXDWLRQWKHRSWLPDO
JHQHUDWLRQ FRVW LV  K ,I WKH DWWDFN LOOXVWUDWHG LQ
ILJXUH  LV ODXQFKHG WKH ORDGV RI EXV DQG EXV DQG WKH
SRZHUIORZRQWKHOLQHZLOOFKDQJH7KHFRQWUROFHQWHUZLOO
EHPLVOHGWRJHQHUDWHDQHZVWUDWHJ\IRUDOOJHQHUDWRUV>@
7KH QHZ RSWLPDO JHQHUDWLRQ FRVW FRQVLGHUHG WKH DWWDFN LV
K LQFUHDVLQJ ZLWK  KDERXWRI WKH
FRVWZLWKRXWWKHDWWDFN
0RUHRYHU WKHUH DUH RWKHU VHULRXV KDUPV FDXVHG E\ EDG
GDWD LQMHFWLRQ :LWK WKH EDG GDWD LQMHFWLRQ WKH HFRQRPLF
GLVSDWFK SURJUDP FDQQRW JHW IHDVLEOH VROXWLRQ DQG WKH
HUURQHRXVDFWLRQVPD\EHWDNHQE\WKHFRQWUROFHQWHUVXFKDV
FXWWLQJ RII WUDQVPLVVLRQ OLQH WR DYRLG WKH RYHUORDG DQG
EODFNRXW
,9'6(%$6('%$''$7$'(7(&7,21
 REVHUYDWLRQ HUURUV :KHQ WKH QXPEHU RI PHDVXUHPHQWV
)LJ ,(((EXVV\VWHP

$QG WKHQ WKH SRWHQWLDO ULVNV RI WKH DWWDFN ZLOO EH
SUHVHQWHGDVIROORZV
5LVN (QHUJ\7KHIW
(QHUJ\ WKHIW PD\ EH WKH PRVW FRPPRQ DQG DWWUDFWLYH
PRWLYDWLRQV IRU KDFNHUV WR GHVLJQ WKH EDG GDWD LQMHFWLRQ ,Q
FRPPRQ VHQVHV LW LV ULGLFXORXV WR FKDQJH WKH ORDG IURP D
SRVLWLYHYDOXHWRDQHJDWLYHYDOXH+RZHYHULQWKHSDUDGLJP
RIVPDUWJULGVLWEHFRPHVDQRUPDOWKLQJ'HPDQGUHVSRQVH
SOD\V DQ LQGLVSHQVDEOH UROH LQ WKH VPDUW JULG )RU VRPH
HQHUJ\LQWHQVLYH LQGXVWULHV VXFK DV LURQ  VWHHO HQWHUSULVH
$VVKRZQLQWKHWKUHVKROGRIPHDVXUHPHQWVLVVHWWR
WROHUDWH WKH XQSUHGLFWDEOH DQG LQHYLWDEOH HUURUV LQ VWDWH
HVWLPDWLRQ DQG EDG GDWD GHWHFWLRQ 7KH DWWDFNHUV FDQ
HODERUDWHO\ FRQVWUXFW DQ DWWDFN YHFWRU KLGGHQ LQ WKH QRUPDO
JURZV WKH WKUHVKROG KDV WR ULVH WR WROHUDWH WKH KLJKHU
DFFXPXODWHGHUURU7KXVLWLVGLIILFXOWWRGHWHFWWKHLQMHFWHG
GDWD LQ D ODUJH V\VWHP ,I WKH UHGXQGDQF\ RI PHDVXUHPHQWV
FDQEHUHDVRQDEO\UHGXFHGWKHWKUHVKROGZLOOEHGURSSHGDQG
WKHGHWHFWLRQZLOOEHPRUHVHQVLWLYH,Q'6(EDVHGEDGGDWD
GHWHFWLRQWKH'6(LVDSSOLHGWRGLYLGHWKHFRPSOH[DQGODUJH
V\VWHPLQWRVHYHUDOVXEV\VWHPVDQG&KLVTXDUHVWHVWLVXVHG
WRGHWHFWEDGGDWDLQHDFKVXEV\VWHP6LQFHWKHWKUHVKROGRI
HDFKVXEV\VWHPLVH[SHFWHGWREHORZHUWKDQWKDWRIWKHHQWLUH
V\VWHP LW LV PRUH VHQVLWLYH WR GHWHFW WKH EDG GDWD LQ HDFK
VXEV\VWHP 7KH PDLQ SURFHGXUH RI '6(EDVHG PHWKRG
FRQVLVWVRIIROORZLQJVWHSV
DQGFHPHQWHQWHUSULVHWKH\DOZD\VKDYHWKHLUFDSWLYHSRZHU 'LYLGHWKHSRZHUV\VWHPLQWRVHYHUDOVXEV\VWHPV
SODQWV DQG HQHUJ\ VWRUDJH GHYLFHV >@ :KHQ WKH HQHUJ\ LV 6WDWH(VWLPDWLRQLQHDFKVXEV\VWHP
HQRXJK IRU SURGXFWLRQ WKH\ FDQ SDUWLFLSDWH LQ WKH GHPDQG
VLGH ELGGLQJ DQG GHPDQG UHVSRQVH IRU WKHLU HFRQRPLF
EHQHILWV 6R WKHVH HQHUJ\LQWHQVLYH HQWHUSULVHV PD\ IDOVLI\
WKHYDOXHRIWKHVPDUWPHWHUWRPLVOHDGWKHSRZHUFRPSDQ\
WREHOLHYHWKDWWKHHOHFWULFDOHQHUJ\LVIHGLQWRWKHJULGIURP
WKHVH HQWHUSULVHV 7KH HQWHUSULVHV WKXV FDQ REWDLQ WKH
HFRQRPLF EHQHILWV RI WKH DIRUHPHQWLRQHG HOHFWULFDO HQHUJ\
$FFRUGLQJ WR WKH FXUUHQW WDULII SXEOLVKHG E\ 3* ( WKH
HOHFWULFLW\SULFHLVN:‡K,IWKLVDWWDFNODVWVIRURQH
KRXULWPD\EULQJWKHFXVWRPHURQEXVXQMXVWHQULFKPHQW
DERXW
4485
%DG'DWD'HWHFWLRQLQHDFKVXEV\VWHP
$3RZHU6\VWHP'HFRPSRVLWLRQ
7KH GHFRPSRVLWLRQ RI SRZHU V\VWHP LV VKRZQ LQ )LJ
7KH SRZHU V\VWHP LV GLYLGHGLQWRD VSHFLILF QXPEHUSVXE RI
QRQRYHUODSSLQJ VXEV\VWHPV FRQQHFWHG ZLWK HDFK RWKHU E\
WLH OLQHV 7KHUH LV D VODFN EXV LQ HDFK VXEV\VWHP IRU VWDWH
HVWLPDWLRQ/LQHVLQRULJLQDOSRZHUV\VWHPDUHGLYLGHGLQWR
FDWHJRULHV
 ,17
 ,QWHUQDO/LQHV /L  M WKHOLQHVFRQQHFWWKHEXVHVLQWKH
VDPH VXEV\VWHP 7KHLU SRZHU IORZ ZLOO EH XVHG IRU VWDWH
HVWLPDWLRQLQWKHFRUUHVSRQGLQJVXEV\VWHP
7,(
WKUHVKROGRI EDG GDWDFRXOG EHOHVV WKDQ LWLQHQWLUH V\VWHP
7KHUHIRUHLWLVHDV\WRGHWHFWWKHEDGGDWDIURPWKHHUURUV
9(;3(5,0(17$1'$1$/<6,6
 7LH /LQHV  /L  M  WKH OLQHV ZKLFK FRQQHFW WKH EXVHV LQ
GLIIHUHQWVXEV\VWHPV7KHSRZHUIORZRIWKHWLHOLQHVZLOOEH
DGGHGWRWKHORDGRIWKHLUQRGHEXV
7KURXJKGHFRPSRVLWLRQWKHSRZHUV\VWHPLVGLYLGHGLQWR
SVXEVXEV\VWHPVFRQQHFWHGE\07,(WLHOLQHV,QWKHVXEV\VWHP
,Q WKLV VHFWLRQ ,((( EXV V\VWHP LV VHOHFWHG WR
VLPXODWH WKH EDG GDWD LQMHFWLRQ DWWDFNV 7KH FRPSDULVRQ RI
GHWHFWLRQ UHVXOW EHWZHHQ WKH SURSRVHG PHWKRG DQG WKH
WUDGLWLRQDO&KLVTXDUHVWHVWLVDOVRVKRZQ
7KHGHFRPSRVLWLRQRI,(((EXVV\VWHPLVFDUULHGRXW
N GHVFULEHG E\ VXEV\VBN N   SVXE WKHUH DUH  1N EXVHV
DQG0NOLQHVOHWQNGHQRWHWKHQXPEHURIVWDWHYDULDEOHVDQG
PN GHQRWH WKH QXPEHU RI PHDVXUHPHQWV WKH\ VKRXOG VDWLVI\
WKHIROORZLQJHTXDWLRQV
E\FOXVWHULQJDOJRULWKPVLQFHWKHV\VWHPLVQRWODUJHVFDOHG
DQG GRHV QRW KDYH VSHFLDO JHRJUDSKLFDO FRQVWUDLQWV 7KH
WRSRORJLFDO ORFDWLRQ RI ,((( EXV V\VWHP DIWHU
GHFRPSRVLWLRQ LV VKRZQ LQ )LJ DQG WKH GHWDLOHG GDWD RI
HDFKVXEV\VWHPVLVOLVWHGLQ7$%/(,,


SVXE
1 = ¦ 1N
N =
ˈ0 = ¦ 0
SVXE

N =
N + 0 7,(  QN =  1 N −    ǂ
1RWHGWKDWWKHUHPXVWEHHQVXUHGDVXIILFLHQWUHGXQGDQF\
RI PHDVXUHPHQWV LQ HDFK VXEV\VWHP WR FDUU\ RXW WKH VWDWH
HVWLPDWLRQ LH PN ! QN 7KH PHWKRG RI GHFRPSRVLWLRQ ZLOO
EHDGMXVWHGLQDFFRUGDQFHZLWKGLIIHUHQWVLWXDWLRQV
• 6HOIIHDWXUH ZKHQ WKH EXV DQG OLQHV UHSUHVHQW VSHFLILF
IHDWXUHV VXFK DV JHRJUDSKLF ORFDWLRQ DQG V\VWHP
VWUXFWXUHLWLVUHFRPPHQGHGWRGLYLGHWKHSRZHUV\VWHP
DFFRUGLQJWRLWVSK\VLFDOLQIRUPDWLRQ
• &OXVWHULQJ DOJRULWKPV LQ JHQHUDO D SRZHU V\VWHP FDQ
EHGLYLGHGLQWRDQXPEHURIVXEV\VWHPVE\XVLQJVRPH
FOXVWHULQJDOJRULWKPVHJ.0HDQV
%6XEV\VWHP6WDWH(VWLPDWLRQ 
)LJ 'HFRPSRVLWLRQRI,(((EXVV\VWHP
(DFK VXEV\VWHP LV DVVLJQHGZLWK LWVRZQ HVWLPDWRU7KH
VWDWHHVWLPDWRURIHDFKVXEV\VWHPFDQEHUXQLQSDUDOOHODQG 7$%/(,, '(&20326,7,212)7+(,(((%866<67(0
VHSDUDWHO\ ZLWK WKH UHVSHFW RI WKHLU RZQ VODFN EXV
UHVSHFWLYHO\ 7KH VWDWH HVWLPDWLRQ RI VXEV\VBN FDQ EH  EXV QBVWDWHV QBPHDVXUHPHQWV ') WKUHVKROG
IRUPXODWHGDVIROORZV VXEV\VB     

 ] N = K N [ N  + H N + D N (5) ǂǂǂ
VXEV\VB


   

*OREDO $OO    

N
ZKHUH] LVWKHPHDVXUHPHQWYHFWRUZKLFKGHVFULEHVWKHOLQH 
SRZHUIORZLQHDFKVXEV\VWHP[NGHQRWHVVWDWHYDULDEOHVLQ $VVKRZQLQ7DEOH,,WKH,(((EXVV\VWHPLVGLYLGHG
N N LQWR WZR VXEV\VWHPV ³VXEV\VB´ DQG ³VXEV\VB´ ,Q
VXEV\VBN K  [  LV D QRQOLQHDU YHFWRU IXQFWLRQ LQGLFDWLQJ
WKH UHODWLRQVKLS EHWZHHQ WKH PHDVXUHPHQWV DQG WKH VWDWH
YDULDEOHV LQ WKH VXEV\VWHPV N HN LV WKH UDQGRP *DXVVLDQ
HUURU DQG DN LV D VSDUVH YHFWRU RI ZKLFK WKH QRQ]HUR
VXEV\VBWKHUHDUH EXVHVODEHOHGIURP 1RWR 1R7KH
QXPEHU RI VWDWH YDULDEOHV LV  DQG WKH QXPEHU RI
PHDVXUHPHQWVLV³')´UHSUHVHQWVWKHGHJUHHRIIUHHGRP
LQWKHVXEV\VWHP') PQ$FFRUGLQJWRWKHSURSHUW\RI
HOHPHQWV DUH LQMHFWHG DWWDFNLQJ YDOXHV RQ VSHFLILF χ2 GLVWULEXWLRQ LQ  WKH WKUHVKROG RI EDG GDWD VXVSLFLRQ LV
PHDVXUHPHQWV  LQ VXEV\VB ,Q VXEV\VB WKHUH DUH  EXVHV  VWDWH
&6XEV\VWHQ%DG'DWD'HWHFWLRQ YDULDEOHVDQGPHDVXUHPHQWVDQGWKHWKUHVKROGLV
7R WHVW WKH SHUIRUPDQFH RI '6(EDVHG EDG GDWD GHWHFWLRQ
7KH EDG GDWD GHWHFWLRQ LV FDUULHG RXW LQ HDFK VXEV\VWHP
WKUHHDWWDFNFDVHVDUHFRQVWUXFWHGDVVKRZQLQ7DEOH,,,/LM
E\ WUDGLWLRQDO &KLVTXDUHV WHVW 5HIHUULQJ WR  WKH ORFDO
N
WKUHVKROG 7R  S RI EDG GDWD LQ VXEV\VBN LV GHWHUPLQHG E\ WKH
ORFDOGHJUHHRIIUHHGRPRNIRUχ2GLVWULEXWLRQRIWKHREMHFWLYH
GHQRWHV WKH WUDQVPLVVLRQ OLQH ZKHUH WKH PRGLILHG
PHDVXUHPHQWV ORFDWHG 3LM GHQRWHV WKH UHDO OLQH SRZHU IORZ
IURPEXVLWREXVMREVHUYHGRQEXVL7KHUHDOSRZHUIORZ3LM
IXQFWLRQ -[A  DQG RN  PN  QN 6LQFH WKH ORFDO GHJUHH RI DQG 3ML DUH PRGLILHG DW WKH VDPH WLPH WR JXDUDQWHH WKH
IUHHGRP RN LV REYLRXVO\ ORZHU WKDQ WKH JOREDO RQH WKH EDODQFH RI OLQH SRZHU IORZ ,Q $WWDFN &DVH  EDG GDWD LV
RQO\ LQMHFWHG LQWR VXEV\VB 7KH 3 LV PRGLILHG IURP 

4486
0: WR 0: DQG 3 LV PRGLILHG IURP
0:WR0:,Q$WWDFN&DVHEDGGDWDLVRQO\
LQMHFWHGLQWRVXEV\VB7KH3LVPRGLILHGIURP0:
WR 0: DQG 3 LV PRGLILHG IURP 0: WR
0: ,Q $WWDFN &DVH  D SDLU RI EDG GDWD LV LQMHFWHG
7RVROYHWKHSUREOHPWKH'6(EDVHGPHWKRGLVSURSRVHG
WR GHWHFW EDG GDWD LQMHFWLRQ DWWDFN 7KH EDVLF LGHD RI WKLV
PHWKRGLVWRLPSURYHWKHVHQVLWLYLW\RIEDGGDWDGHWHFWLRQE\
GLYLGLQJ D FRPSOH[ V\VWHP LQWR VHYHUDO VXEV\VWHPV 7KH
PDLQSURFHGXUHRI'6(EDVHGPHWKRGFRQVLVWVRIIROORZLQJ
ERWKLQWRVXEV\VBDQGVXEV\VB7KH3LVPRGLILHGIURP VWHSV'LYLGH WKH SRZHU V\VWHP LQWR DFHUWDLQ QXPEHU RI
0: WR 0: DQG 3 LV PRGLILHG IURP VXEV\VWHPV DFFRUGLQJ WR WKH V\VWHP¶V SK\VLFDO WRSRORJ\ RU
0: WR 0: 7KH 3 LV PRGLILHG IURP XVLQJVRPHFOXVWHULQJDOJRULWKPV6WDWH(VWLPDWLRQLQHDFK
0:WR0:DQG3LVPRGLILHGIURP0:
WR0:
7$%/(,,, ,1-(&7,21$77$&.21,(((%866<67(0
VXEV\VWHPDQG%DG'DWD'HWHFWLRQLQHDFKVXEV\VWHP
,Q WKHH[SHULPHQWV WKH WUDGLWLRQDO PHWKRG FDQQRWGHWHFW
EDG GDWD LQ WKUHH DWWDFN FDVHV +RZHYHU WKH '6(EDVHG
PHWKRG FDQ GHWHFW DOO EDGGDWD DQG ORFDWH WKH EDGGDWD LQWR
WKHVXEV\VWHPFRUUHFWO\
3LM0: 3ML0:
0RGLILHG
 0HDVXUHPHQW 2ULJLQDO ,QMHFWHG 2ULJLQDO ,QMHFWHG 5()(5(1&(6
9DOXH 9DOXH 9DOXH 9DOXH >@ ) & 6FKZHSSH DQG - :LOGHV 3RZHU 6\VWHP 6WDWLF6WDWH
$WWDFN (VWLPDWLRQ 3DUW , ([DFW 0RGHO ,((( 7UDQVDFWLRQV RQ
/    
&DVH 3RZHU $SSDUDWXV DQG 6\VWHPV YRO 3$6 SS 
$WWDFN 
/    
&DVH
>@ 1 ;LDQJ 6 :DQJ DQG ( <X $ 1HZ $SSURDFK IRU
$WWDFN /    
'HWHFWLRQ DQG ,GHQWLILFDWLRQ RI 0XOWLSOH %DG 'DWD LQ 3RZHU
&DVH /     6\VWHP 6WDWH (VWLPDWLRQ ,((( 7UDQVDFWLRQV RQ 3RZHU
 $SSDUDWXVDQG6\VWHPVYRO3$6SS
$V VKRZQ LQ 7$%/( ,9 ³7´ GHQRWHV WKH WKUHVKROG RI >@ 79DQ&XWVHP05LEEHQV3DYHOODDQG/0LOL+\SRWKHVLV
VXVSLFLRQ RI EDG GDWD *OREDO YDOXHV RI -[A  DUH  7HVWLQJ,GHQWLILFDWLRQ$1HZ0HWKRG)RU%DG'DWD$QDO\VLV
 DQG  LQ WKUHH DWWDFN FDVHV UHVSHFWLYHO\ ,Q 3RZHU 6\VWHP 6WDWH (VWLPDWLRQ ,((( 7UDQVDFWLRQV RQ
2EYLRXVO\ WKH\ DUH ORZHU WKDQ WKH WKUHVKROG 7KXV WKH 3RZHU$SSDUDWXVDQG6\VWHPVYRO3$6SS
LQMHFWHG EDG GDWD FRXOG QRW EH GHWHFWHG :KHQ ZH DGRSW 
>@ < /LX 3 1LQJ DQG 0 . 5HLWHU ³)DOVH GDWD LQMHFWLRQ
'6(EDVHG PHWKRG WRGHDO ZLWK WKH$WWDFN &DVH  ZH ILQG
DWWDFNV DJDLQVW VWDWH HVWLPDWLRQ LQ HOHFWULF SRZHU JULGV´ LQ
WKDWLQVXEV\VBWKH-[A LVZKLFKLVKLJKHUWKDQWKH 3URF$&0&RQI&RPSXWHU&RPPXQLFDWLRQ6HFXULW\
ORFDO WKUHVKROG  LQ VXEV\VB WKH - [A  LV  SS±
ZKLFK LV EHORZ WKH ORFDO WKUHVKROG  ,W LPSOLHV WKDW >@ 2 .RVXW - /L\DQ 5 - 7KRPDV DQG 7 /DQJ 0DOLFLRXV
WKHUHLVEDGGDWDLQVXEV\VB6LPLODUO\LQ$WWDFN&DVHWKH 'DWD $WWDFNV RQ WKH 6PDUW *ULG ,((( 7UDQVDFWLRQV RQ
'6(EDVHG PHWKRG FDQ GHWHFW WKH EDG GDWD LQ VXEV\VB ,Q 6PDUW*ULGYROSS
$WWDFN&DVHWKHYDOXHVRI-[A LQWKHWZRVXEV\VWHPVDUH >@ /H ;LH<LOLQ 0R DQG%UXQR6LQRSROL )DOVH 'DWD ,QMHFWLRQ
ORZHU WKDQ WKHLU WKUHVKROG UHVSHFWLYHO\ ,W LV IRXQG WKDW WKH $WWDFNV LQ (OHFWULFLW\ 0DUNHWV LQ  )LUVW ,(((
'6(EDVHG EDG GDWD GHWHFWLRQ FRXOG EH DEOH WR GHWHFW WKH ,QWHUQDWLRQDO &RQIHUHQFH RQ 6PDUW *ULG &RPPXQLFDWLRQV
SS
PXOWLSOHEDGGDWD
>@ ) 3DVTXDOHWWL ) 'RUIOHU DQG ) %XOOR &\EHUSK\VLFDO
DWWDFNV LQ SRZHU QHWZRUNV 0RGHOV IXQGDPHQWDO OLPLWDWLRQV
7$%/(,9 '(7(&7,215(68/72)75$',7,21$/0(7+2'$1''6( DQG PRQLWRU GHVLJQ LQ WK ,((( &RQIHUHQFH RQ 'HFLVLRQ
%$6('0(7+2'
DQG &RQWURO DQG (XURSHDQ &RQWURO &RQIHUHQFH  SS
 *OREDO VXEV\VB VXEV\VB 
YDOXH 7 -[A  7 -[A  7 -[A  >@ $OL $EXU $QWRQLR *RPH] ([SRVLWR 3RZHU 6\VWHP 6WDWH
$WWDFN
   (VWLPDWLRQ7KHRU\DQG,PSOHPHQWDWLRQ&5&3UHVV
&DVH >@ =KDRMLH:DQJHWDO$JUDGLHQWLQIRUPDWLRQEDVHGUHDOWLPH
$WWDFN SULFLQJ PHFKDQLVP IRU PLFURJULG LQ HQHUJ\ LQWHQVLYH
     
&DVH
HQWHUSULVHLQWKHWK:RUOG&RQJUHVVRQ,QWHOOLJHQW&RQWURO
$WWDFN
   DQG$XWRPDWLRQ%HLMLQJ
&DVH
>@ *) 5HLG DQG / +DVGRUII (FRQRPLF 'LVSDWFK 8VLQJ

4XDGUDWLF 3URJUDPPLQJ ,((( 7UDQVDFWLRQV RQ 3RZHU
9,&21&/86,21
,Q WKLV SDSHU WKUHH EDG GDWD LQMHFWLRQ DWWDFN FDVHV DUH
GHVLJQHGWRH[SODLQKRZWKHKDFNHUVPRGLI\WKHGDWDLQVPDUW
JULGV DQG DYRLG WKH WUDGLWLRQDO EDG GDWD GHWHFWLRQ LQ SRZHU
V\VWHP:HGLVFXVVWKHSRWHQWLDOULVNVRI%DG'DWD,QMHFWLRQ
LQ VPDUW JULGV DQG HYDOXDWH WKH ORVV RI HQHUJ\ WKHIW DQG
$SSDUDWXVDQG6\VWHPV3$6S
>@ 5'=LPPHUPDQ&(0XULOOR6DQFKH]DQG5-7KRPDV
³0$732:(5¶V H[WHQVLEOH RSWLPDO SRZHU ÀRZ
DUFKLWHFWXUH´ LQ ,((( 3RZHU DQG (QHUJ\ 6RFLHW\ *HQHUDO
0HHWLQJ-XO\SS±
>@ +: 'RPPHO DQG :) 7LQQH\ 2SWLPDO 3RZHU )ORZ
6ROXWLRQV  ,((( 7UDQVDFWLRQV RQ 3RZHU $SSDUDWXV DQG
FUDFNLQJHFRQRPLFGLVSDWFKLQ,(((EXVV\VWHP 6\VWHPV3$6S

4487