Springer Series in Statistics AnesoBorgr/Gleling Sista Moe Basel on Coan Process ‘anronterer Dus A Caton of ren fo tty Pes ore Stet Anscombe: Campvang i Stasi See though APL ‘erer: tai Devon Teo) a Baja Anal, 2a on, {efecto Ther fer inte Popltions BorsGroeen: Madr Muliinensonal Seng Try nd Appts ‘ronan Pit Proce snd Queer Martingale Die. ‘Brocwel/Davis Tie Ses Theory and Metts, ein Deter Jones An intone They Pot roe Dehapaie: Pane srion and Hype Testing Special Anais of ‘Stony Tne Sees. fomorch: Neng Carve Exton: Math, Tay and Appcon. alma Mv Ses Mveling Based on Geer Linea 7750. 500, “Two Cops “Teor Moe Crp PenbergMoapv Ka Taar (Bd: Sie Motel Federiok Mostlies ‘Peron Te Colecte Wors of Waal Het. Good, Permutoa Tes: A Prac Gade o Reaping Methods fr Testing yp Contains: Meares of Asian for Cos Caton. ‘rand Asp of ik Thr Hermon: Abate! Stasi, Volune I: Derg of Popa. ‘ae Te Bens and Epon Expanion Hinde: Soong Tenge Wi Ingenio i. Hart Nonna Stn ant Laat FT. ‘arian: Byes Te. Hedaya SlaneSnfen: Otooca Ans: Theor and Applcions. “eae Qos Linen an Appin: A General Azpocho Opti Parmeter Eximatin eer Theory o Sats Expres ‘Gaiden PLUS Examples. Jie Pinca Component Anis ‘elereman Tet Eoin Methods a races on Tolasn (Es) reboup in Sexi Vane Dimitris N. Politis Joseph P. Romano Michael Wolf SubsamplingDi Pie sore P Romano Depurnent f Marais Depart of Sass (they f Caer San Deo Sted very 5:00 Giinan Dove Son CA 84806-4085 Tat CX 90095.012 os its@mthaces Dogue de Ea y Emme ‘sera Cat Il de Ma Sys Grate mote 2 EE een na sre ttm ant wa tose foeeticnesae, ieee Wi ata ‘ereggemiin fe ae prea ee Ne Yn e175 Fel Ave Now a ‘SCT Ma lng Meohnd Mads ema eg be oc 6 aoe ISON ONT SUSLA SpingerVetig New Yor Bates SPO LT ‘To our families and friendsPreface ‘Sine Bfton's (1979) profound paper othe bootstrap, an enormous aenount of effort as been spent on the development of bootstrap, jackkife, and ther resampling methods, The primary goal of thse comspiter tensive ‘methods has bom boon to provide statistical ols hat work is complex i ‘tls without imposing unrealistic or unverifiable assumptions sbout the data generating mechanism. Bootstrap methods ote cas aleve thi oa, but lke any statistical method, they cannot be applied without chook While the bootstrap will undoubtedly serve as 8 widely used toot well Into the 2Ist century, it sometimes can fall Moreover, the asymptotic {istication of the bootstrap often pecliar tothe problem at hand. Tews ried in Plead Rotana (1002, 1094) tata very general sopronch to constructing saymptotialy valid inference procedures existe Dy appeopeate use of subsampling. That, the statistic of invrest, such ‘5 an estimntor or test ait blunted at rubeaaple of the data, ‘nd these subeumpled value are used to build up en estimated sampling ‘isintion, Historically the roots of subeumpling ean be traced to Quer woulle’s (1949) and Toke’ (1958) jackin. I fact, Matanobis (1946) oggested the use of subsamples to estimate standaed eros in studying {zop yields, though he ned the term interpenctrating samples. Hartigan (69,1978) exploited th use of subsamples to construct confidence incr ‘als snd to approximate standard eon; he obtained ome Bite sample resus in the sytmetle Teuton proble, a wel as some asynpeate o- Sul for asymptotially normal statisti. Hartigan's constructions valve recalculating @ statistic over all subsamples and all subsample sas, but Iiealo considers tng only randomly chosen subsamples or what he easIrdancad subsamples. OF courte, the use of subsamples is quite related to the delved jackrife which was developed roninly inthe content of va see estimation; soe Wa (1986) and Shao and Wi (180) Tn these seis ontext, Carini (186) considered the we of subsamples of what he ell subeeres, o approximate te variance of «statist, Later, Wu (1090) real. laed how Jackie pucdo ales ean be used for distribution estition as ‘ell, bat ony in tho content af independent, identically distebted (Hi. ‘observations when the statistic is approximately linear so thal asymptotic formality bods is arguments were specif to this onto. "The mala cntibution presented here can be summarized as lows the ‘onsateney proper of ctimated sampling dstibution based on a> ‘sampling hold undor extremely weak tssumption, and even in situations where the bootstrap fib. In contrast to the aforementioned results, we ‘an obtain quite general rosuls by comidering subample of xed sie (but depending on the sample ic and possibly date-dependnt) I fet, 1s bard to cancsve of theory forthe construction af conidesce tater ‘als that works (in a fit onder sense) inder weaker conditions. Moreover, the mathematical argunents to justify such a claim are based on fly simple ides, thas allowing for case of generalaaion. Therefore, we will ‘onsdersubeampling not just inthe usual eontextof independent and en cally disibuted observations, but also in the eatent of dapendent data ‘Staations, such as time serie, random lds, o marked point process. "Tholdea of using subsumes asa diagnostic tol to describe the sampling tition of an eatiator was ao considered in Sherman and Carita (2996). tn the id, context, subsampling is closely related to the usual hootstap, the man dflrence being resampling the data without replace- ment vers with replacement. Ik follows Ut the bootatrap also exjoys general fst ordor consistency propetics, but only if on i wing tore Simple at aes much smaller than the orginal sample sie, as noted In Polis and Romano (192, 1985); the benefits of ting a smaller ees ple size when bootstrapping are further exploited in Bick, tae, td van ‘Det (1997) Ofcourse, the usual bootstrap eas work peeectly wel, and ‘enjoys good higher order properties in some nice problems. Subsampling, fn the other hand, is ore generally apliable. Utimately, it may be used fs robust starting poit toward even more refined procedures (some of ‘whic we presi) “The primary goal of this bok to lay someof the foundation for susan pling methodology and related methods The book i ld out in two parts, (Chapters 1-7 provide the basic theory of the bootstrap and subeamplig (Chapter 1s devoted to developing some of the basic consteney propetis ofthe bootstrap. Obviously, one can write values about the bootstrap, ‘and there are now several books wnse main subjee isthe thoreial de ‘elopment of the bootstrap. Our purpose iso provide some mathomatial tools need in studying consistency properties of the bootstrap, thereby seitng the stage for studying sulsampllng. Thus, thc ehaper serves ab Preace ‘tutorial onthe bootstrap that can be included in graduate course om ‘mpotie methods The chapter i largely base on Beran (1984) and one aeauthorsleeure notes ina cass onthe bootsrsp taught by Beran at Ue desley in the El of 19RD. Chapter 2 consiers the use of subsam- Jin in the cae of independent and idenealy distributed obverations ‘ime compartons are made with the bootstrap, Chapler 3 considers sb- SRtling stationary tine srk, and generalizations tthe nonstationary or TRurskedestie case are presented In Chapter 4; the moving blogs boot inp abo considered. Generliations to random feds are developed ie Chapter § and marked poine process are discussed in Chaper 6. tn {Chapter 7 general theorem on subsampling is developed that allows the Sruorser pace of interest to be quite abstract, This concludes the frst rtwo pats of the book. The second part of the hooks concerned with ‘heen of the basi theory, at well as practical sues in implementa- ‘Ton and applications Chapter 8 addreses comaplox situations where the Convergence rate ofthe estimator is unknown since it depends on para ‘rs that are ot suid Known subaumpling is shown to be generally (Teta ere as well The sue of cholo of block size I discus in Chap tor 8 Chapter 10 i devoted to higher-order accuracy and extrapolation. ‘Two important cases where the convergence rata is unknown are detailed in Chapters Tf and 12, Chapter 1 considers the ease of tho mean with boy tall (infinite vazlnce), while Chaptor 12 eosiders inference for an ‘toregreive parameter inthe posbly integrated ess. In Chapter 13, wo sider financial appintion by using eubsampling to discus wheter ‘Mock relarascan be predicted from dividend yalds, Te appendices contain “ite rela on micing sequences thet are oie througout the book, "This book i intended for graduate students and researchers; it can be vd for an edvanced graduate cour im statistics that assumes a base Loowiedge of theoretic! statistics usually aught atthe ist-year Ph.D. level Even if one dvs aot want to devote an entre course to sutsamping, {Chuplers {are designed that they ean be included in any course on ‘symptote methods in statistics, We have benefited from the support of many fends and ellengus Special thanks are de to Patrice Bortll for his instrumental help toward ‘he origin! development of the ideas io Chapsers 8 and 10. Some of the Tok was presented in graduate courses taught at Stanford University and UC, San Diogo in 1007 and 1995, and we thank our sbuents fr proving Tall comments, We also mould ike to thaak the Nationa Sience Foun dation for supporting such of this research and writing over the pest few earsContents Preface a I Basic Theory 1 1 Rootsirap Sampling Distributions LI ntredueton LLL Pivotal Method 112 Asymptotic Pivotal Method 11S Asymptotic Apprasimation LA Bootstrap Approximation 12 Consistency 13 Cane of the Nonparametric Mean TA Generalizations to Menke Statistics is Bootstrapping the Empirical Proce Dilleroiablty andthe Bootstrap 7 Rurther Examples 19 Conelusons 2 Sulampling inthe LLD. Case 0 21 Introduction 2 22° ‘The llc Theresa a 23 Comparison withthe Bootseap aContent 28 26 27 28 29 General Parameters and Other Chee of Rot 25:1 Stadt Roots 25.2 General Parameter Space Hypothosn Testing Data Dependent Choice of Block Size Variance Fstimation: The Deleted Jackie Concusions Subsampling for Stationary Time Series a1 22 33 a 35 36 37 38 a9 Tneroduetion Univariate Pararater Cae 321 Some Motivation: The Simpiest Example 322. Thaory and Methods forthe General Univariate Parameter Case 323 Seudomind Roots ‘Multivariate Parameter Case Bearplor ypethess Testing Date-Dependent Choice of Black Size Bias Resietion Variance Estination 58.1 General Statistic Cate 382 Care of sh Sample Mean Comparison withthe Moving Blocks Bootstrap 8:10 Coneivons Subsampling for Nomstationary Tine Serie 42 43 aa 45 46 a Subsampling for Rand 51 52 53 54 55 56 Tntrodetion Univarlate Pazaaeter Case ‘Maltiariate Parameter Case Examples yp Tig and Dao Depot Cie of lok ‘Variance Estimation (Conclusions Fields Introduction and Definitions Some Useful Notions of Szong Mixing for tcdom Fd Consistency of Sutsamping for Random Fields 531 Univariate Parameter Case 532 Mulivarinte Parameter Case Variance Estimation and Blas Reducion Masimum Overiap Subampling in Continuous Time Some Iustratve Exarpler 32 2 s o 6 6 00 101 101 102 106 108 ur no 120 2 1 ma a 129 1 1 57 Conclusions 6 Subsampling Marked Point Processes G1 Istroduction 2 _Delnitons and Some Diferent Notions on Mixing 63 Subeampling Stationary Marked Point Processes 631 Sampling Soup and Assumptions 632 Mala Consistency Result, 6.33 Nonstandard Anymptotics 64 Siocuste Approximation (G5 Variance Estimation vin Sobeampling 66 Examples 67 Conclusions 11 Confidence Sets for General Parameters FA Intodetion 12° A'Basie Theorem forthe Empiical Messe 7A A General Thoorer on Subsampling 74 Subsampling the Empirical Process 15 Sobsampling the Specteal Measure 76 Conclusions a a8 1 0. ua rm 16 19 181 185, 138 169 159 10 64 186 168 10 I Extensions, Practical Issues, and Applications 171 4 Subeampling with Unknown Convergence Rate fa 82 8s at Thtrodetion Batimation ofthe Convergence Rate 821 Convergence Rate Estimation: Univariate Parnater Case 8:22 Convergence Rate timation: Multivariate Parsinier Case Subsampling ith Extimated Convergence Rate (Conclusion 9 Choice ofthe Block Sire or 92 93 os Tatroduetion Variaiee Estimation 021 Case of te Sample Meat 9.22 General Case Entlmation of « Distribution Function 93.1. Caliration Method 932 Minimum Volality Method Hypothesis Testing 8.1 Calibration Method 1 rod Ao 182 184 187 188 188 189, 180 2 3, 194 im 200 20Part I Basic Theory1 Bootstrap Sampling Distributions 1.1 Introduction “The bootstrap was dscovere by Bron (1979), who cone the name. this hapter the bootstrap Is developed es a general method to approximate th sampling dstbution of «state, a plvt, oF root (defined below), In order #0 constrict confidence roions fora parameter of interest. The te ofthe bootstrap to proximate a nul distribution inthe construction tof hypothesis tet sao considered. Much of th Usoretial foundations of te bootstrap ate ai out fn Bick and Frowiman (1981), Singh (1981), ‘ud Beran (084), The development begins by focusing onthe independer, ‘entialycatibuted (Ki) owe, (ies) bea semple of m independent and identically (Ud) sandom variables taking Values in sample space az having unknown probability distribution P, where P is assed to belong to a certain olaton P of distributions, The ellection P may be “parse mete” “Semiparameti\" of nonparameti," inthe usual sense tized ‘roughout much of the staisicalltrature- OF eouse, any family Ps parecized bythe ealistion of probalias P in P, and theres no ncod to place any retiction on P at ths poi ‘The interest les in constructing a confidence region for some parame ler OP), whose range {0(P) : P'c Ph wil be denoted ©. Typically, © ‘subst ofthe real ite, but we aso conser more general parameters For cumple, the problem of estimating the entire combative distrbationfunction (of el-vald obvation wil be tet, x that © san spot futon spc. "hi est consdeing rot Ra(zn,8(P), 8 erm rst oie by Bo- ra (106i), wich jo ome Fnetional depending on bath and BP) ‘he Leni tat confidence tral or hypotbe tats could be com ruc i the don of the root were Known. For ample, an {Stinator 8, of el vald paamter0(P) mig be given so that a at weal noi (2 6(P)) = @p OP) oF aleratvely Raz 80) = [8, ~0(P)/aq, whee aq ise atimate of the andar deviation of Uae tari tate, the rot Ra (zy (P)) wil be mmumed 10 e's. reakled fein ofthe data rand Ue parameter OP). ‘When Pi sultably rg so hat the probe osparametsc in nature, antaral conto fr aneaiata By oP) dy = OC), where Py tho empire! deriation oth dat- Of coun, thi constrain imply ‘sums that 8() I defined on the empl reasues tba 9) lest el dene. Alters, ela prnctrc problems fr which is indexed ty 0 parameter eloging (oa subset ¥ of P50 tat = (P- 9. W) then d(P) can be dein aa fnctina 19) ene 44 on tala Lobe tS), we Gn wre dasabl timate of 9, ‘ch ar main Hlboo! einai, «ooestep mason Lathood {Stimaor, ora minima distance estimator, et Jn(P) be the datibution of ay, 0(P) when «sample of sie from Py and ea P} et coreapondng emai dstebtion faction died by Jnlt,P) = Probe fale (PD) < 0) In order to construc a confidence region for @(P) based on the root Ra(a0(P), te sampling dstibation Ja(P) or Rs spproprate quantiles ‘st be known orelimated. Sone standard methods, bed on pivots aad ‘somptotie approximations are now brify reviewed, Note hat in many of the examples when the obeortlons are real elu, It is more conve ‘sien! and customary to index the unknown fly of ditions by the ‘smlaivedstttion function rather than P, We wl fely use Both, Alpnding on the stustion 1.4.1 Pivotal Method In certain exceptional cass, Ja(P) does not depend on P, and hence an ‘exact confidence region for 0(P) can be constructed. In tls ease the root Ha(2q,6(P)) scaled e peot. Tn general, if Fy (2,8(P)) isa pivot, ts 0, Lat y~¢ and y + be contimity points of G fr some pees 6, Then, Galy-0) +O < 1a od Gly +) = Gly) > 1-0 ence, forall ecently large m, yes G;'(l-a) E12), with BZ) <2. Then, (Zq) satisfies i, tn sup B1Z1(Za > 8 (4210) Proof. Since the distribution of Zy converges weakly to the distribution ‘of Z, we can sou, by Skorobod's Almost Sure Representation hore {soo Billingsley [1086), Theorem 25.6), that the Zy ar all defined en the Probability span (0,F,), wheres Use uit inzerval, Fis the Borel sets, find jis Logue measure furthermore, Ze converges almost surely 0 Z ‘Resume tho lena mere fae by monotonicity in of BYZal(2y > 3) frees then have i ims Fe 1a > 2 = whore © > 0. Fix 8 = of ie = 00; take 6 = 1. Choose © such that Elza <)> B(Z)~6. Coen, BZ) = Beal ln <0) + Elza > oh ay By dominated convergence, E\Z1(25 < e] + BIZI(Z <)]. Thus, the limaup over ofthe Fie side of equation (1.11) & greater than or equal ta E(zZ) "6 r4> E(Z); on th other band, the let side of (11) tens to E(Z), sielding » contradiction @ Remark 1.3.2. In fact, we can actually concur the lemma that {Zn} i formal ftagrables tat lms» 8p, E21 Zq > 3] ~ 0 Proof of Proposition 1.2.1. For purposes of the proof of!) construct Knap Xp which ace independent with denial dstebution Fy, and x ‘me must show that the law of w/#(Xy — ul) te to verify the Lindeberg contin for Yan wete Yas = Xay~()- Thi entalls showing that, fo each > 0, lin, BIVE YE, > nd) <0 (2) Fix any 6 > 0, We must show the expeetl value on tho tet sie of ‘equation (1-12) is Jos tha or equal to for ll saintly ange m. By {Umma 121, the Y2, sti: thee existe such tat fo all 3 > fe, Binsup B42 > A) = Bu, soon a6 me? > i, BING LYE, > mePD) < BNE LOPE, > Al 18 Cas of tho Nonparametic Moan 18, Which is ess than 6 forall sufcently lange n, implying Lindeberg's onan has. ‘In order to prov () by applying Thuorom 12.1 inthis example, en sppropeate resampling dstetution Py mast be apesfied, Let Fy be the fmpricaldstebution of th data. Thea, [Fy} € Cp with probably one: By the Gliverko Caatell theorem, sup, |[Fn(2)— P(z)| 0 with probe tality one; alo, by the stong law of lege umbers, E,) —@(F) with probability ope and a2(F,) = 2%(F) with probabalty one. Ths, boot Strap confidence intervals forthe man sed on the ote OCF) = 8/4, 0(F)} are acymptotically ald inthe seas of the Useaeem. Remark 1.8.8. The following extension of Propasition 18: bol. Let De be the set of wequenes {F) such that Fy converges weakly toa dis tutution G and o#( Fy) ~ 0(G) = 2°(F). Then, Proposition 1.3.1 holds with Cy replaad by Dp. (Actually, one relly only nerds to deine Dp 30 that and sequence (Fq) is tight and any weakly convergent subsequence of {UF} has the above propery) Thus the pase chains for the resampling ‘szibution are quite large in the ease thatthe bootstrap appraximation ‘I(Gz) ean be valid even if Gy fe note all ence to F. Por example, the ‘bole where Gy is nommal with mean X, and variance equal 1 a eons tent eatin of the mumple variance rm i conisteney. Therefore, the ‘oem approximation can i fact be views! ast hoostap procedure with a perverse choice of sampling distribution. Romark 1.84. Let F and G be two distribution functions on the ral line and define (FG) to be the ifn of (EV. — ¥ PIP" ove all pin of random variables X and ¥ wich that X has diteibution F and Y fine dition G. can be shown that te infimum sata wod that dy ina metscon the space of tributions having a rth tnoment. Further, if F has a ite vaiane 0°(F) then dy(FyyF)— 0 i equivalent to Fy converging wey to F and o*(Fa) ~» 2°(F) Hence, Proportion 12.1 may bo resatad a ollows I F hase finite variance o2(F) ad ds(Fo,F) =O, then Ja(Fa) converges weakiy to J(E). The metric dz is known as the [allows metre. For deals, so Bickel and Freedinan (198). Coatinsng the example ofthe nonparametric mean, it sof interes to ‘onsder the studetiznd root Balt, (FY) = n'y — FD /OCFa)e (133) ‘where o2(F,) 16 the usual bootstrap estimate of variance. To obtain ‘consistency of the bootstrap method, we appeal tothe following result Proposition 1.3.2. Suppose F is « distribution on the kine with f- nite nonsero variance o2(F), Let J4(P) be the diriution of the oot (4.13) based on a sample of size from F. Let Gp be defined 08 Proposition 1.3.1ML Bootstrap Sampling Dtebutons ‘Then, Jor any sequen (Fy) € Cry Ja(Fa) consees way (0 te andar normal triton Bete proving his propotin, we Sst eed weak nw of lage mrs for gular ary Pe folowing nwa serena sla vrion fr our paps Tennis 18:2 Suppose Yay Yau i a tongear ara of indepen tent radar rie, he nth rou hating df Gn Ase Gy comserge Sn dtribaton fo G and FIA EIVI Prabys{|2 Dos — Ral? o%(F)) >} +0. ‘Ba, ths ellows fom the previous lemma onthe weak law of large numbers for a triangular area. "The eoristency of the bootstrap method be on the rot (1-18) now follows ely. 11s interentng to conser how the bootstrap behaves when the under: lying distribution has en infinite vasaneo (but wel-deined meen). The short arswer is tht the Bootstrap proceare considered thus far wl ail In the sease that the convergence in expression (1) docs not ha. Tbe {allure ofthe bootstrap forthe meas inthe iflt variance cave was Brat noted Bab (1984); farther ehucidation i given im Athreye (1987) ane ‘Knight (1080) la fat, che following striking theorem smarts that the sim ple bootstrap ste thas far wil work for the mesm if and oniy if Use ‘rane finite The thoorom, which we sate But do not prove Is duo to (Gin wad Zinn (1980), LBA, Let Xp Ny bed with bf F, and let Py be the fed of X---, a. The X, variable are defined on a probably ‘space wth ouzomes 2. Conditional on Xie oy 1X} 1.3 Cae of the Nonparametre Mean 18 according to Fy. Let 9 he any metric metrising weak convergence, Suppose there exist random variables jig = jn(Xy.-- Xn, and erasing sepurice {oq} with ay > oo, anda dstsbution Ju, ) (depending pohly ono or cepuoalenty the sine soquence XX.) auch tha AEE plhnnkdtory—0 9 ‘th probability one. Here, £() denote the aw ofa random variable Then, o%(F) < co, ned —»€ for tome © > 0, Iee,F) = N(0,co(F)), and ig eam be taken 10 be WX ‘Ths rite variance isan essential assumption forthe strong consistency ofthe bootstrap. OF cour, for eoreetaxyrmptote level of eonfdence ects {asin the convergence (1.6) it Is ely ony necasary to have the ever ‘gener (114) bold in probability; a related result forthe wel coniteney Of the bootstrap ix alo provided in Gin’ and Zinn (1089). ‘The resale thatthe simple bootstrap stud so far works i and only if the variance {3 falte. Fora ice expoton of these rls, ae Cin (1997). Related resltsfor the studeatiaed bootstrap based on approximating the Aistibution ofthe root (1-13) wore considered Wy Cag an Maso (1989) ‘and Hall 1000) "The conclusion is that the bootateap i stongly or almost ‘arly comistent sand only the variance ie Snite the boost i weakly onsistont if and only PX; ein the domain of attraction of the normal ‘atria, Ts fat, it ea lied Atheoya (195) that the bootstrap ea be mod Sd otha cnsisteney en ith infin varianes. The modiieation colts in reducing the botstap sample ie. odd, the following tr. Thoorem 1.3.2. Let Xin. %u be bid wth ea. P belonging tothe domain ofairaction of a a-sabe law J(F) for 1< <2: ape, HRCASIOX: - 09) JU) {for some constants oy increasing to 2, where p metriss weak conserenc. ‘Suppose mg —> 00 aan» o6- Conditional om Nays Kes et Xiu es Xo be tia. acconing fo Pa, the empirical diatriution of X, Then, eB Ia, 0) 0 and only main 0. ‘The consisteney ofthe bootstrap sssuing a reduced sample ize ma with ssuming ma /n — 0 is div to Athreya (1885); the converse i due to Ar ‘ones and Ciné (1980) A velated result on the strong consistency of the16 1. Bootstrap Sampling Distributions bootstrap via reducing the bootstrap somplo sie is as given in Arcones and Gin (1989). In order to apply these resus, one must know the noe malizing sequence ay. However, Arcones and Gi (1001) show that on ry be estimated oo that sell-normalizing rots are considered, That is JF) denotes the distbation of the oot EM =e) on a under. Then, the bootstrap epprosination Jn (F) aise ASP) dE) “+0 in probably 0 and ha ~ co in fat hese resus on the nonparametric mean foreshadow our ger results on canneney develope In this retin AS wil be ace in oer iastaners whore te simple oottre als, conmtency enn often be ec ered by reducing the bootstrap sale ste. The bene of reducing the bootstrap sample size was recog! Rt In Bretagne (1989) An ven more gneal approach bse on steaming wil ofcourse, be cosiered Ralem QF) = i 1.4 Generalizations to Mean-like Statistics Example 1.41 (Linear functional). To generalize Example 1.1.3 let F bea datribtion onthe line and conse the probes of estimating a fanetional of F, given by ae [toares tt Re ily Rota, O(F)) = 0/7(0(F) ~ OF), (1.45) 16)= [eer 007) 1 onder to verify the conditions in Theorem 1.2.1 for the validity of the ‘otetrap, the folowing I ed, ‘roposition 1a.t. Let F lea ditribution on the line such that r(P) se nite and nonzero, Let Gp be the st of sapunacs of disrbutions{F) st. {ring r{Ey) > (F), and the ditribtion of o(¥) converges weal tothe latrbution of (Y) wen Ys, hos stration Fy and Y hag dsrbuton F. At In(F) be the law ofthe rot (1.15) under B 1 I GF} © Gr, then Ja( Fn) converges weakly to J(E), where JCF) és 14 Goverlntion o Moanike Statistics 17 1 Hence, the bootstrap is consistent inthe sense that AAda(E) Sn(F)) 10 wih probably one Moreover, the bootstrap eanfidence interval By(l — a) given by equation (13) 48 anynpotcally wad the aenae of the cover gence (1.8), Proof. The proof of 0 immediate from Trem 1.1. To prove (i) of the proposition, appeal to Theorem 121. Lat Fy, be tho emptieal distrib ‘on ofthe data Then, {Fy} € Cp with probability one ad comsiteney ofthe bootstrap follows. Remark 1.41, Noto that, iis assuned continuous, then Cp reduces to the set of sequences (F,} such that Fy converges weakly to F and 4s(Ey) —7(F), Without the cotinlty asumption, ths eed not Imply the setibation of (%) converge weakly to the distribution of oY). This ‘example illustrates one of many special propertics ofthe empincal dit ‘ution fonction. Tht ie if Yq us dstributon F, and Y har distribution then Ys conversing in distribution to Y does nt imply) conver in dseribution to (¥) without further assumptions on and/or F. Hom cer, suppase Fy isthe empirical dittbution based on @ sample Xy.-™ Ky {bom F. Then conditional on Fy iY, ha distribation fy, then lV) 6 ‘verges in distribution toy) for alma all ample requences Xi, Xay ‘with oo assumptions at all on yar F (except for tmeasuabilty ses” tons). To wee why, conitional on Fy, Y¥q) has the ditibation asigning mess n~* to ouch of V(X), 1-< 1 = mut this is jus the empial Alstribution of » obmevations frm the ditibution of YY) when Yas ‘Then, construet Xqis.--1Xnn bid. Fy via Xna = Qn(Ui), and let A Atos the empiric cbse on Xs. na. Note tht Ft) > il fd only fQy(a) < Now, the empleeal proces 5) under Fy satisfies 5,0)» By( Felt) because Salt) = nF (8 ~ FAO] = WAS Xne SI ~ FalO] mA S10) $8 FO) =n < FO} Fld BIO) ‘Henee, Ea(F), the distribution of $4() under F, is that of By(F,() But, by te alot sare representation Uasren (sce Pola, 1984, p71), there exist versions By of By such that ‘sp |B (6) ~ Bs) ~0 with probability one, ‘whore B() bi standard Brownian Bridge proces. Hone, Ba( Fx) has ‘he sume distribution a (F())- Buty sup BE (F0) ~ BLE maple Fst) — BCFA) +-s9p1B(FA(O) ~ BIE) "Tho fst vem onthe right-hand sie tends to ero with probability one because ofthe contruction of 2), The second tern ted to aec0 with Drobability ope becuse the Bromnian Brice proces BI) hae ifm ‘continuous sample paths with probably one. Taus, the dstebution of 2B, (F4() converges weakly to Ut of B(F(}), which the ee lit law ofthe empileal proceas under a fxed P. Teen follows (hy he contin ‘ous mapping theorem) that, whenevet do( Fa, F) —~ Jn(Fy) converges ‘weakly to J(E), which is the law of sup, [5(F(0)). Note that, i order to daim the comeet asymptotic coverage probability, i must be known pga ge preemie yg Saphire22-1 Bootstap Sampling Distibatons ‘Schmid (1958) or Tire'on (175). ene, part (i) follows because {Fy converges usiformly to F with probability be. “The botstrap ofthe empitienl proces has ben gerard greatly thanks to the Vapnik-Ceryonenkis Dudley theory of empireal proseses, Speci Fell, let Kinsey be tid, variables according Loa probally lw P taking vals in a general space S. The empiiel measure P, is the law satisfying 28) = 255104 € 5), for a sot En $. The emplrical procs Sy indexed by «elas of Fnetons Fis given by Sil) =nP SPF hore f € F, and the notation PY means f,f(2)4P(z). Clady, some TDessueablitysequiementa onthe functions / tn F are requited I we lok, tthe Binite dimensional dieribations of $\(), the malivarate Cental Limit theorem apis, so that (S4( fi), Sule) converges weak under Ptoamultvarate normal datibuton with mean nro and covariance that OFZ (yarns ey where oul 24] = PULL) ~ (PMP HY: ‘ofcourse this esses Pf? < oo. Hence, by the multivariate mean exam pl, the bootstrap of tho empircal proces indosed by a Bate famly of functions is immediate. Needle oy, the techniealdficalisspprecate comiderably once the fay F i large. Suen conditions forthe com ‘vergence of tho general empirical process canbe ven In tarms of entropy Conditions or covering numbers see Chapter VIE of Pulled (1954). Islet, Giné and Zinn (199) proved a remariable ret that esentally says the following: the empirical proces converges toa Gaussian lilt if and only the bootstrap procs converges (with probability ove) tothe same iets se thee paper fra precne statement. The acer theory of bootstxpping fenplnieal proces ls developed in van dor Vaart and Wallner (1906) ad Jn Chapter 2 of Ging (1097) In the special cise when $ i k-dimensional Buclidan space, the & ph Assume F has a density f. Then, one can check by simple eaeulus that the inBoence faction i (a) = PMESMO sete) =" 70RD) ‘Thon, the desired conclusion is that n*/(0F,) - O(F)] is esymptotclly ‘ormal with vasa seo and vince pp) FOR) Again, tho differentiable approach ead dicetly othe right answer (thous ‘diol there areoven sper approach this example), ad rigorous ‘elestion of asymptotic normality is discussed in Serfing (1980, p. 236), ‘Note, however, tha the qt unetionl snot Pct iferentable with Varelge(X))= 16 Diferentiaiity nd the Bootstrap — 29 respect to the supremun norm, However, weaker notions of dferetiabiity ood inthis example; so yan der Vaart and Welle (100, Section 39} ‘Tho folowing result isthe main esl ofthis ection, Under a iffecatobi ity hypothesis fora nonlinear exit, the bootstrap is wally conisca The theorem assumes Frchetdiferentiaility with respec to the mup oem, but the proof canbe trivially adapted to knclude other seminoma as al sch asthe one in Example 16. ‘Theorem 1.6.3. Assume F contains dstiutions with finite supe ort Also, assume 8) is Fréchet diferentible at Fath repect to the supremum norm js Vy and whose derivate, ge et Psagfee 0< Varrlgr(X)] - Again, Theorrn 1.63 oes act apply besa the median factional not Fret Aiferetabl, Tut ab expected and elaied, the bootstrap is will wld. TD appreciate 17 Further amples 31 veh, Jal) must be smooth in F in whieh case we can appeal to out ba 1: sonny here, To thie, tC bn ta et Fe) setitying Bi wR) +n) 1) = 2p(00R)) fore el Than, tea bestown ta 8} € GJ come ito J(F). Father ty oping tease soe pegs tb theorem can te sbown tet ex Venlo of eed tat fal'in Gy wth pally one. Weak constancy of Ue boos flnws Det se pita i Ba 6), Hiss crap wars a phrase th al dete jee nate franc a th sol ein inset rave, the tnt nae ove er ee a (Femi ser ewes mom putin On oer FP So ifemiade st (Foti bata done ee ‘bootstrap fails (see Huang, Sen, and Shao, 1996). ei Example 1.72 (Linar mgresion). re, we pres our et example that tras slightly outside of he Lid setup Suppo the datas calamity) oFaboeatons ‘Sthiying the near egsson model Bam Adem where de = (oisa} ithe mx p matrix of nonrandom regesors w ‘ankle. 0 ea p 1 vector aad eed Go Baya) ‘8a cshomn vector ofm Ld andor viable having. Fy F& awed to tave mean sr and fit vance (F). Soe ye fed, what thnowa about the data generating mhanio ie the dtu the (rot F and the regression cooicints@. The leat square esteaton of 9 2 SAAN Man fd te usual ectinate of (Fi then Sa (0 p)han — AdaD et Jn(F) denota the sampling distribution of 851A; Aq)™(3, — 6) ‘under Fj note that thie distribution ge not depan og 8 ‘The lest squares residual are éy = 25 dy: the rth component of 2 donoted By Lat Fz be the empirical cl of the eenared idle (Bases Bia) bovng th component defend by oS Be ei82 1, Bootstrap Sampling Ditrbutione a an Tel eo aac fe cee i) tes Fem may Fon EEL Sala eelrcl pa tr dere ln ef en dn om Sam ee where By = An(A,An)!A, Ie a symmetec projection (idempotent) tmatrix of rank 0, 1 = epillel—t By the law of large numbers, Yel#/(n 9) —» oF) with probability one, Moreover, Bae] = * er atnt Ba] = * tra Ba) EE ty = Car) = MEP 0 ene 6) npr dr FTF) ome way {otis dsb af wlene () Gr It remains to check (F7} falls in Cr with probability one. By Glivenko- cael cones wll te Fo yb ae By i a Sgt tn cg ry PUB) cc ahC)oin ily on Wot can dat ye Sith ove Theater eat coe ae ‘Sma dite nal is wh oboe el Frotina 181 Leane 3) By he nena on ence St {ES} lin Cr wit pray oe oe Boren, Ts exams fvtontdrdy Freeman (181 splat ht hor ty witha he proper earn the ene See feo ncGramin et amie St eee ea WOlbao tenga aioe 18 Hypothed Ting — 83 TOF) whore OF) = VR ECE) Let Fy be the empitieal ed.6 of Xiyoo:)Xq- Consider the estimator (Fa) of CF). Note that this estar isthe tame aa the one discus fn the previous linear regression example; however, tte diferent taxi have a diferent distribution theory. Let Jy(F) denote the dstbtion of lx) ~ OP) uate P. The bootstrap ettnator i ten Ja( a) We outline consistency of the bootstrap (Tor deals, see Bera, 1984). First, deine AUP) = Bel —TaLYPTET, which seamed ite as is (171). Let Cp be the et af waquenc (F,) satisfying F converges weakly to F, a(F,) ~* (P), A(Pa) = AC) nd V(F.) — VCE). teean be shown by the Lindeberg Central Lilt Sheveat tnd the law of large mumbers that Jy(F) converges weak to J(F), the law ofthe multvariace normal estibuton with mean tro and covetance anise V-IA(FWV-1(F) whenever (Fy) € Cr. Furthermore, (Fo) tals ‘in Gr with probabitiy one, and so the bootstrap ig consent 18 Hypothesis Testing In this ection, me conser the ws of the bootstrap forthe construction of trpothess tests, Asse the dita X,.-.,2y are Ld from an unknown law P. The observations are assed to ale valu in a gatple space 8 ‘The null hypothesis fi aserts that P belong ta certain fal of ‘wibutions Pa, while the altenatve hypetsieH; maser that P belongs fo family P1. Of course, we assume the intersection of Py anid Py i the ‘ll set, and the unknown lw P beongs to P, the union of Py an ‘There are several approaches one can fale to cousteuct byptieis test. First, consider the cas when the null hypothenis enn be stores 2% 4 hypothesis about a ral o vector-valued parameter (), Thea oe ‘an exploit the familar duality betwee confidence regions and hypotes Yess to make inferences about 6(P). Th, a asyptotcally Yall est of ‘the null hypothesis that (P) = 8 can be constructed by an aaynptans ‘ally valid confidence region fr (Pty he following rl! accept the ll ‘pothesis i and only ifthe confidence region incluso, Therefore, all the mods we have thus fr dscussd for coercing conidence eens tay be tized: methods bed on a pivot, an ayemptotie pt, en asym {oti approximation, or the bootstrap. Becase of ts failing dle, tho ‘deals of this approach ar lett the reader However, not all hypothe esting problems freely lato the ene ori of testing parameters. For example, consider the probe of sting ‘for normality. For another example, when X, is vector valoed, eeniteStL Bootstrap Samping Dstrbutions the peoblm of testing whether X, has distibution that Ss spherically "The approach taken inthis scton isto estimate the nll distribution ‘of some tnt statisti hy resampling rom a diteibiton ayn the com rants of the nul hypothesis. Indced, in odor to construct rte valu, the distribution of some tat statue must be known, atleast under the nul Hypothesis "Tobe expliel,ateume te gol isto construct an asyrmpttiealy valid test bsed on a ral-valnd txt waite Py = Tx(X eres %q)- Lange tus of Ty reer tho nul hypothesis. Thus, having picked sultable test statistic Tour goal so construct rita vale, sy, e(I—a), 30 at the test eh rej i and only iT exceeds ey (1—a) slits PUT (Xiyos-yXq) > enl —a)} 4.08 96 when P € Pg. Furthermore, we requle this rection probity to tend to one when P € Py. The question i ow to contract a etal value to achive these goals. Unlike in tho elgscal cass, the eriticalvaluo can end ‘ill be onstrated to be date dependent. The bootstrap approach ofers Solutlon to the question of how large the rte value should be, To this i, let the tr dinebtion of Tb ete hy (2. P) = Probe (Ty(Xiy--. Xe) Sth Note that we have introduced Gq(P) instead of utilizing Ja P) to Aiznguish from th cae of confidence inervle whee Jq(, P) represents ‘the disributin of «root which my depend bot on the data an on P. Inthe hypothesis testing context, GP) represents the dstrbation of atic under P. Hocus P fe unknown, Gy(,P) i nkiown, Also it au(1 ~a,P) =inf(2: Gs(2,P) 21a. “The bootstrap approtch isto estimate the nul sampling dstribution ty Ga(Q,), where Q, an eaimate of P in Py 0 that Q, salutes the conszaints ofthe null hypothe. A bootstrap critical value ean then be ‘defined by u(t 0,Q,) = laf la :Gul2,Qq) 2 1a}, ‘Then, the nomial level test rejects Ho and only Ty > galt 22a) Notice that we would not want to replace a Qy stistyng the mul hy pothass constraits by tho empeeal dstibtion function By, the usual Fesampling mechanism of rampling mechanixm of renpling the dale ‘vith replacerpent. One might say that the bootseap Is so adapt at st Thaling the datelbution of a watstic thal Gy(,P,) a good eatinate of Gal-P) whether or not P satises the nll hypothesis constraints. Hence, the test that rejects when Ty exeadsgy(1~a, Py) wil (under sultble con: Ations) behave asymptotically Ike the test that rejets whon Ty exeaods ‘as 4,P), ad this tse has an axymptote probability ofa of ejecting 18 Hypothesis Ting 96 the mull hypothesis. Obvonly, when P € Py, we want our tes to reject ‘with probability that we approuching one ‘Tho the choice of resampling distribution Qy shoul satiny the fallow ing. IP € Po, Qe should be near P 0 that Gy(~P) = OatOq), the ‘n(1 04 P) = gn(1 ~0,Q,) and Use saympttic rejection probs ap, pwosches a onthe other han, P'€ Py, Qu should not approcch P but some Fin Pp In this wa, the eltical value shuld sa In(1 ~04Q) ® gal 4, P)) +9 ~ a Ph) < 06 ‘5m ~ 0, Then, asuming tho ts statist ls constructed that Ty + a0 under P when P € Py, we wll have Py > all ~0,Q9)} = P(Dy > g(t ~0, Py} +4 8 ne, by Slutsky’ theorem, Pxample 1.8.1 (Testing the mean). Lat Xyyoc..Xy be ral valued with five sean and variance. The probit to st wht the mean ofthe X; i zero. So, Py is the st of dstibtions wih mean sero nad Anke vance. int poly for dice of tet sate kT, = na where Xs i the sample moun. To construct an appropiate resample ‘mechani that obeys the constraints of the mall hypethiaiy er Oe be ‘he dstrbation in Py cess to the empiren dstibtion Py One way to derive lsenes the folowing. or astbatons P at @ on the el ine et Br (PQ) be the (forward) Kullbick-Leilerdfvergebee betwee Prantl Q dined by .a)= [tba Note that 3x 1(%Q) may be 20, fx, 5 no 8 ttre, and tf not even sgtumezric im is arguments. To ogmeract the resenplng ist utios le se the Q that minimizes (Pa) over Q in Py. (Another pees 5 to minimize the (backward) Kulick Leber divergence bet, ,)) This choice for Qu ean be shows tobe wall dened and cose oo foding the nonparamstsie meximum lelhood ettimator of P seating P's constrained to have mean 22. The optinnation problem ch be tole fairly eaily. By Elron (1961), Q, asigos masa to Xe where 's proportional to pe OER ‘Sa reer ‘ed is chosen so that SPX; <0. * ‘Now; we addres che|consstney i level oF type Tero of the tests As 4m the construction of confidence interval, C,(, P) runt be uot ia P 'n order forthe bootstrap to suceted In ie thotem bow rath tha ‘specifying ast of sequences Cp as was done in Theorem 1 T,smcotins1. Bolaeap Sampling Distributions is alternatively desribed in terms of «meted: For the sae of stating a thre, ether approach could be raed ls oe Remark 122, The proot fs analogous tothe proof of Theorem 1.2. ‘Theorem 8A, Let KigonsXq Bo ttd. according to P ¢ Pox Assume The follwing triangular arty comergence: of d(Py,P) 0 and P< Po seples GusPs) comeciys weakly GCP) with GCP) a continuous weft Montyer astme Oy team estimator of P Bos om [Link] tha satis dle, P) 0 im probity whenever Pe Po, ‘Then, P(T > gy(t ~a,Qu)} —t 981 0, ‘Example 18.2 (Goodess af ft), ‘The probnto test wheter he eae ably itibtion P belong ox pare fay of dis Autos Py = {Ta,@¢ Oo), muro Oy an open stat of dines Cea tte fs be te ere amae bd 00 Xion a Feet be on ear of: Comider te ts tte Ty = mH PP) cor 6 some measure (yay a ete) betwen Ped. (la at Fe eee nymmerte wiih weal somtimes fF exam, tthe Ceunce won fis tts) Beran (188) conser the case Stare in minimum dsance eta, wile Romano (1088) aaunes Teds hace mympttcly near estate (ike typical maxima Heliod car. For the ramping medians, tale Qa = P,- Doth Beni) andere (GBR) give erent st of conton 0 that thal tore appa fc nuh ft rary tons ae aoa gn Seton 18 when the bottap ofthe mpi proces st Bera, Note, when Pe Py, oud converge th ale Staging AP. Pa) = ff, Po Of course this sues problans of uniqueness and existence ae taken end Fa ke wet ce 18 ypothos Testing 57 Infact, we claim ony testing probes can be pt in this framework [A that is requied to construct i the following! for P€ Po, define 1(P) = Fy for Pe Dy, deine 7(P) to be any member in Po. For the bootstrap to scone, honeer,7() should be smooth. One posibity the olowing minim data consireton. Let ge a met or poi ‘ preudometie, and passbly distinct from 8) on the space of probably Aisnibutions. IP for Pe Py, infgce, ol P,Q) ix oman ack, then ‘define (Ps that HPAP) = jaf P,Q) No, given @ particular choice of x(), a test statistic Ty can be conerocted by T= blot Pa) whore P isthe empirical measure. Her, 7m is at spproprinte sealing 80 {ha the limiting dsebution af Ty nondegenerate (in hope of sisting, the conditions ofthe Ubere). Notice that when p and 6 are the se, the test atin reins 10 dif 6Ps.0) In this setup, tbe resampling mechanism i now bul into the cnstrvetion af Use at im take Qu =r (Ps) Example 1.8. (continued) Roooisder Example 1.81, the problem of vesting the sncan. Ta Ue light of the above discwsion, istead of wing ‘Zana choice of test statistic, one ean diectly tee Ty = nb Pe, Qu) there Gy isthe @ miniming the Kulback-Labler divergence 8x. ver @ in Bp, This valet the test static uel by Owen (1088) in is consiicton of empirical Hetihood. However rather than sing the ‘sympttie ch squared dsebution ogo rte vas, a drcetbootseap spproach resample frm Qy Example 1.8.3 (Testing Independence). Lat Xi,» Xn bebe valued rand variable and suppose fiat 3X, = (Sayers Nea) mpc fd components The prblen so test the join indepence te compoents, To gt sted, suppose thet thet component thes vals ins space 5, and the produt space 8 = [e-,S-Il Piva Drobabiiy om 8 et P denote the marginal probability on ft th component. For any P, deine (P) tothe product iw TL, 2 Tn, Se Pitand oy iP ra tema rt {esate we nto specify 6.) Raman (988) takes 6F,Q) to be the spre dstaner vcr an apropriteVapik Ceronenk ase f sel, and shows Thorn 11 ta. Node that eng from Jn = (Pa) i eile to resampling om the arial te emp ‘al stibtion indepen: Rona (18) sow ttt sie Cleexample, such as testing to a randomization or prmtaion Us. Other exampl for spherical symmetry, testing for exchangeabity, or vesting fr a change in ae considered a wel 1.9 Conclusions In this chaper, the bos theory ofthe oot was preted the enter of ndopendere data Sone fara lege sume convergence bwopertin were devlped, mally inthe context of etic hat ae a Bona nt Aton nota i traci re tral and te costrcton of ypothess tests. The gs! Was to pov Three the ear ole estan oe a her al prop tte of te besarap, in stpton of analogs rere soba ing preered Ta the banice ofthe book. For more Uoroagh expose of the outa the rede i fered tothe works of ow (182), ran fn Ducharme (1001, Hal (192), Fon and Tithe (1080), Shao and {To (Jon), Ct (10) and ion at Hey (LT) A lating oe feet nme hes nr ante ch as Berchews im one chapter) ant wtih = preset in Hl (1992) The we retin perce methods suc as Eto BC. mato, loomed ‘Stevi ca be fond in Dies Roan (408. 2 Subsampling in the LI.D. Case 2.1 Introduction Jn thischapter, a general theory forthe construction of confidence intervals ‘oF rgions is presented, Much of whet is presenta is extent om Polis ‘nd Romano (192, 1046). The basic ide to approximate the satin, Aistrbation of sais basd onthe values f the stants compat ore ‘aller subsets of the data. For exazaple, in the eave where the data are ‘observations that are independant and itty dstebuted a sats 4 computed based othe entre dita set and i computed veal (2) taste of ie b mpi he notion ofa statisti sequen so that he atic is defined for samples of size mand 8. Those romped aloes of the statistic are suitably normalized to approximate the tras sargling istration, ‘The approach presented here is perhaps the sat general theory forthe fmitruction of fs-order aeyaptoicaly valid eouldence regioee That ‘it wil be soon that, under very weak assutaptions on bythe mth is valid whenever sho orignal statistic, suitay normalize, tas Hit ‘itsibusion under the true model. Other methods, such a the boost, ‘oauir that the dstsbution of tho statistic is somehow locally smooth op ‘function ofthe unknown model In frt nny paps have bes cevetad le showing the convergence of a suitably tema static ks Haag bution is appropriately uniform ass function of the wokowa model ‘especie situations. a contrast 0 auch acgumptinn or wieceon ofc ‘oothnes is required in the thory for witenplng Inet metho402, Ssbeanpling i the LD. Cae tere is applicable een in the several knows situations wich represent coveretpiet the bots. To appreciate why seaming Daves wil under suc weal atvuptions, oe tat each suet eb (len ‘eho replacement fom the rgial daa) i Indo asamp of sin 5 from tho tree model, Hence; it bol be itil ln Ut ene can ok ‘rast approximate the sampling distribution ofthe (somali) atic bod ona stpl of oe b Ba, oder the weak corvergene hyotb {te suming dtrtons base om spl a ee band shold be close ‘The buna on the oer baad, bans on rcrmplng atti cnet smile of se from some ented mae that i hop owe to the feo model Fateh ho ie ti he ccf any te series of tore generally,» homogentoss random Bld. Th ony ier Seer a te Stic computed over asa ma fet ef ‘he data that rain the depennce eracire ofthe erations. Pre mle, X,.- Xx oprenntn obsvations om some satinary ne tera the ates ecompted only ver the ab ste bot She form [Xi Nasu rent) Th extensions to tine sere, borage ‘eos rando Beli a mated pol proces wil doce alter heotes “Theme of subsample ars to approximate the wrinnce of» statistic is well now. The Quenoale Tule Joie etimate of asa a fe bse on computing a state vr al auuanpes of ae n= bs te wells end cst relate! to the meat nd variance of our cstimated sampling dstibtion with bn 1. Mabalanabis (1040) sg dented the eof samples to ctimate varity in tying eop held, {hove rd te ane terpeneteting sump Halal ethos haw been well sti nthe content of asmping theory (re MeCarthy, 106), Hatgns (196) Ba ltradund what on (182) calls aon ‘bearing meted, which bed on the cman a aii ore a2" nonempey abs of the data i method ls wen to produce tinct coublence lime Inthe special context ofthe symantec location bt Hat 7) ae a in ot mp ee eral context oferta cat of etinator whieh have appt na {tal dnsibacn But, even he content, aynptic rls saa the mmber of tsa se to recompstethe ate rman xed mt ‘n> oy which ete ela of eine “Therion random subarpting methods sini nth they both as subnets ofthe dato appesate sna errr of ati, ‘or pethpe erento appt mpling dsibtion, The method Bre ‘ented hee retains the conceptual simpy of thew methods ad ea {olbe applicable under very iia sumptions. Eins (1979) Bote, wil varng sone snr proper tothe sfoeestione methods aa core we defini in the jack, ‘nd hs taco th more abou gal of approximating a entire a 22 The Basie Theorun 41 pling distribution. Shao apd Wu (1960) have shown that, by basing a ekki estimate of variance on the satstie computed ever submarles With d observations deleted, sia of the decenceg ofthe wound cide emtimateofvarlane canbe removed, Later, Wi (1980) und these ‘utsample values to approximate an ene sapling dintbution by what he calls ache histogram, but only in segular Ld. stmatioas where the statistics appropriately near so that asymptotic normality ensue, ta ‘more broad generality, Sherman and Carlin (1096) colder the ase of subsamples ss a dgnostc tool to describe the shape ofthe taming «steibutio of genral statisti, though rma aferenee procedure, sh ' th construction of confidence Interval are not delivered. Her, we sok ‘ow thse subsample values ean acoratly estimate m sampllug tide tion without any assumptions of asymptote normality, by onl ssnuring the exstonce ofa limiting distribution, Moreover the taytaptoie vay af confidence sstements follows. In muttary, wile the method devlored ln this works quite related to several wellstudied technique the sighs, ‘ty of our arguments has lead to asympotie justifcation under the rst ‘eral conditions, Jn Section 2.2 the method is dscibed inthe contest of id, observe "ons. ‘The maln (hore is preted and sever! examples are ge, Sots ‘comparisons with the bootstrap are drawn in Sctlon 2.3, A theorem on ‘ochustic approximation to the subsampling dstribtion is presented Sction 24 In Section 25, the theory is extended i a staple way to ub ‘sampling studentinad roots and to roots designed to construe ceationce ‘ands for parameters that are function. The use of subsampling in the ontext of hypoteis testing is dselbed in Setion 26, A general fren, Proving consistency of subsampling wing Tandon or data deve cholees the boc size is presented a Seton 27, Suteampling sete to (ideted ackknle i the context of estimating the varinnee ofa seta Section 28, 22° The Basi [Droughost ths srton, Xi,....Xq 1 « sample of independent and ‘enally distributed random variables taking vue in an aitety sat Ble space S. "Tho common probally measure geneating the obervations bs denoted "The goal isto constrict conidence rion for some pe ‘antee (P). Por tow, asuie 8 is real vad but this ca andl os siderably generalized to allow fr the onstruction of condones eons ‘or makvasate parameters or coufidenc bane for histone Let By = By(XyonusXq) be an eatimator of OP). Te desired to et {Bate or approximace te truc sampling datibutio uf dy ln ordes too 'nfrenoes about 6(P) Nothing is arumed about the form ofthe cient ‘Theorem{2 2 Submampling inte LLD. Coe though its naturl In the Ld. comtext to assume i symmetric in ts ‘ngumeuts (bat even this snot neces). i Define Jy(P) tobe the sampling dstebution of f(y ~8(P)) based on ‘sample of si m fom P, whee my 4 normalring evstant. Alo deine the corresponding cumalaive distribution funeton le P) = Probe ralb(Xyo--s Xa) ~ (P)] <3 Essentially, the only assumption that eo will ned to constrict symptotielly wld conSidence Intervals for O(P) i the following. ‘Assumption. 2.2.1. There exes @ tiniting lw J(P) auch that J(P) converge wonky 10 JP) 08 = 0. ‘This assumption will be required to hold for some soquenc 7 twill be recesary, however, that, such atthe init law J(P) i wondegenerate "To deseribe the method stated in thi section, let Yi, ¥yy he el to the Na = (3) subscts of eae bof (yy), ordre In ty fashion, Of cous, the depend on band n, but this notation has ben suppressed Only a wry weak asamption on b wll be rogue. In typical stations, {e willbe assumed that n=O and 5» o0 a8 00. Now, let Ona coal tothe saint y evaluated at the dataset Yi. The apraximtion to Ja(2,P) wo stuty is dined by Ep) = NSS Gane 84) 2 ey “The motivation bend the metho isthe flowing or any anor sample of san orn P enc, the sac dsotion of G2) in IP). The empire stration of the Ny vale of mos — {&(P)) shoal then serve sa good approximation to P). OF eae, BP) le unknow, so we replace 6(P) by by whic is asyinpttieally permis because r(, ~ HP) is of order nro 0 ‘Now, specialize tothe cas when 5, = Xe isthe sample mean of cteratonn hiving man 6(P). Asin the vrtanes of Ky Bit, 80 “nl One can comp the variance of he dstbation La) and EV lven ty [fn )/n| 8, where 52— 350%, — Ny} rhs case tis valance depends on defo the dependence Ba with tach or aod with So, 8 acme desirable to hn replae 7 (21) torn lo/(nB)! 20 Un the rerling apposiating diebion bas ‘rinse, independent of nour theorems (n/m 0 tat the St order nynpote pope are the sane nth cane By makings change, however, ob eimator more cewly resembles that of Ws (10), tnd the rane of th approximating dirbutoncoreapond to Sho ad ‘Wits (198) dete d= bakin tinue of warinee. Heaecoth we do not ody nin (21) beause the above modiation i only fused 22 The Basic Theron 43 folie satis. Keep ia ind, omer, ta ch «mediation ay inpeoethe approximation in soe rations os Chapter My The hore example abo prs explin te ck of comratin of sing jttite pci ves to appre samping aren totale exception i Wu (1900), tte ample went ewe fb et and if ¥, is the original sample with X, deleted, then (7 — 1)(6n,44 ~ dq] = (Xa ~ X,). The empirical distribution ofthese values converges wniforaly tole dso of (7) Xt pay oom Hes ca our techrique (th Wisin ama iessney eee se Sic i ht oat. A ne of ang Encore th patie. Teams othe aro ew ‘atonal ue cade oly nt leon ed ‘in So Brae fra campmor tg ta er terest in icone a thooppnte creme be then nai ~ dy =X, ~ rand he cmpl dbs of tt vas onverg unre to the dition of XP) ws oobi oe Seti, nensstory flown Aside fom ihtcw caronn ees a he ‘hers pie spebl when f+ ade ‘Theorem 2.2.1. Assume Assumption 2.0.1. Also assume rota Doe, and b/n +0 am sce a LW x is « continuity point of 1(,P), then Lao) = JesP) in awe ty point of JP), then Lys(a) + Je, P) 4H. 17IG,P) 8 continuous, then sup lbms(e)—Ju(2,P)|-+0 in probably, 22) Mh tee 1 =a} =n: Engle) 2 1a}, Correspondingly, define i ~a,P) =inlte: He,P) > 1-0} WI(P) 8 continuous at ~0,P), then Probe rally —O(P)) 0, Lewt-anyn) < oe ‘Then, the convergence i () and (i) hol with probability one