38 ASYMPTOTIC EVALUATIONS Bection 103 The left hand side is 0 because Eoe(X —#) = 0, sa we have | [se-9] Fla —0) dx fue ~a [pre-e] ae = [0-9 [Foe He a] seo), where we use the fact that Zy(ai/o(s) = $ log.g(y).‘Ans last expression can be written Ea[(X ~O(@LNY), whore (61) is the log tfkelibood, yielding the identity Boul ~8) = 0 Sux - pf = Bole ~ayr@yn {whieh when wechoose j= yields yee hops faiar equation ~Eolt"(@1}) = EI (GLX): see Lemma AD 1.18 now a stople matter to compare the asymptote variance of an M-estimalar to that of the MLE, Recall that the agyinptotie variance of ele MLE, 4, ven by 1/Bt (@]X)2, 50 we have.” (aw(X — OAK — Epo — OER ~ toy virtue of the’ Cauchy Swartz Inequality. Thus, an M-estimator is always less ef ficient than tie MLE, anc! matches its effcieney only 1 y 8 proportional to {” (see Hsercise 19.20) mn this section we did not try to ehusify all types of robust estimators, but rather we were content with some exaraples. There are many good books that treat robustness a Aptis the interested reader might try Standte and Sheather (199) or Hettmansperger ie MeKean (1998). (20.2.9) AREG ye.) 10.2 Hypothesis Testing. As in Section 10.1, this eoction describes a. few methods for deriving some tests in coniplicated problems. We are thinking of problems im which no optimal zest, as defined in earlier sections, exists [for example, uo UMP unbinsed test exists) of is known. In such situations, the derivation of any reasonable test might be of use. It two subsections, we will discuss large-sample properties of likelibood ratio tests and colle: approximate large-snanple tes. 1UI.1 Asymptotic Distribution of LTS One of the most useful methods for complicated models is the Hkolihood ratio wethod ‘of test coustruction becanse it gives an explicit definition of the test statistic, sup L(@Ix) Seen 103 HYPOTHESIS TusTING “s and an explicit form for the rejection region, [x Ab} < ch. After the data. X x ate observed, the likelihood function, L(élx), is a eompletely defined function of the variable 0. ven if the two suprema ot L{Gpx), over the seta Oo and @, cannot be analytically obtained, they can usually be computed numerically, ‘Thus, the test statistic Aix) can be obtained for the observed data. point even if no couvenient formuls delining M(x) is available. “Te define a evel a test, the constant e must be chosen so that (in3.) sup P(X Se) Sa ve Tf we cannot derive a simple formula for A(x), it might seem that it is hopeless te derive the sampling distribution of A(X} and thus know how to pick e 10 ensure (10.3.1), However, if we appeal to asymptoties, we can get an appraximate answer Analogous to Theorem 10,1.12, we have the following result ‘Theorem 10.3.1. (Asymptotic distribution of the LRT—sinple Hy) For test- ing Hy <0 = Oy versus Hs = 04 Og, suppose X1,...,Xy are iid f(x8}, 0 is the MLE fF, and f{x|0) satisfies the regularity conditions Then under Ho, as = co, ~2log NO) — xf tn distribution, where xj is a x7 random variable with 1 degree of freedom. Proof: First expand log £(#}x} = i(9|x| in a ‘Taylor series around 9, giving oO HO) = Hla) + '(Glx\(6 — 8) + CAlxi. Now substitute the expansion for {(@olx) in ~2log Mx) = —2U(Bolx} + 20s), and get ~2log Ab) = (Bx) — 6Y. ‘where -we use the fact that U'(6}x) = 0. Since ~!"(|x) is the olwerved information Jn(@) and 27416} > 1(59) It follows from Theorem, 101.12 aud Slutsiy’s Theore (Thoorom 8.5.17) that ~3lox MX) — yf o Example 10.8.2 (Potsson LRT) Por testing io = \ ‘ase oa observing iy... Xs it Poison), we have 0 versus Hy day ~ Bog Ab) (8) Aa 3) ~ Mog(20/%)] where & = Szy/n iy the MLE of A. Applying Theorem 103.1, we would reject Hp st level a if 2log Mix) > ad.0 Figure 10.3.1, Histogram of 10,000 ealves of ~2log A(x} along withthe rdf of a x syn, bre ¢ anal ton eae and = a Fie 10.31 she stone of 10.00 te Fh pf oe, Tech en wo be nena Moreover. & comparison of the simulated (exact) and xf (epproximate) cutoff poin SO us tla be coos we emit nl Simulated (exact) and approximate percentiles of the Poisson LRT statistic. 1.630 nnlated tented to the cases wher nl ypetbess cone discussion tts topic may be Goud in Stunt, Ord, and Arnold (1989, Chapter 22) Th nb nts aC by en ent ejection for lar? Rejection of Hp: @ € Bp for small values of MCX) ie equivalent to rejertion f valtes of ~2log ACK). Thus, Hly is rejected if and only if — 2log A} > Vi oto i the dees of vodom speci fa Thenwems 1029. ‘The Type | oe el ae and the sample size is large. In probability will be approximately a it 0 € Be le Sax 0031) wi be appowmtely sts fr lresample ses and on. vce: fert hoo beon defined, Note that the theorem will actually lim Pylreject Hy) =e for each @ = Qo Nok that the sUpyeiy Fa(tejectHy) converges to a. ‘Tis is usually she ease for asymp totic size a tests ‘The computation of the degrees of freedou for the test statis: forward, Most often, © can be represented as a spaco that contains an open subset in RY, and @p can be represented as a gubsot of ‘dimensional Euclidian space that contains an open subse i 38”, where p< g. Then 4~ p= vis the degrees of froedorn for the test statistic. c is usuolly straight- subset of g-diinerssional Buclidian Example 10.3.4 (Multinomial LR?) are nonnegative and sum to 1. Suppose and P(X, = 3) = Daj = \kelinood funetton is Lat = (pips rss paspo), where the 9s Xn a id dseveee eandom variables 5. Thus the pmt of X, ie j(5}9) = py and the Lid [1 10210) = prepping, whore yy = mumnber of ze,...,2q equal to j. Consider testing Ho: Pi = pa =a and py versus Hy ‘The: full parameter space, ®, is really a four-dimensional set. Since ps = 1—p, Ps~ pe, there ate ouly four free parameters. ‘The parameter set is defined by Hy 6 wot true =P Syst and yo, p= 4 subset of containing an open subset of 4. Thus q = 4. These is only one free Parameter in the set specified hy Hy because, once p,0 S jy st equa p; and ps ys d—ta3, ‘To calenlate A(x), the MLE of 0 under both Oy and @ must be determined. By Setting 1 is fixed, pa = py ps woust equal 38, Thus p= 1, and the degrees of freedom a log L(@[x) = 0. for each of j = 1y... 4, im ad nsing the facts that py = 1 Wwe can verify thae the MII finetion reduces to, Pi~Pr~ pa py al yy =~ De — Ys— tay Df py under @ is p; = y,/ni. Under Hy, the tkelihood any Agtin, che usual method of setting the derivative e Pr nde a- Lis) = prem (4 -qual to O shows that the MLB of * Ho Ws fo = (i + a+ al/(Bn). Then Pro = Pro = Pay and Bay = Bra LL Shio)/2. Substituting these wolues and the jy values into Lld)x) and combining ns with the same exponent, yield Nx) = ay (msgeny (age) (4gTIONS Section 10 490 ASYMPTOTIC BVA Thus the tent statistic is (193.2) 2log Mos) = 2 aloe (4) EE vrs my =m = my = a+ +49)/3 and ng ~ may ~ (ya )/2 The asymp size a test rejects Hy if —2log A(x) > x¥,,. This example is one of a large class of testing problems fer which the asymptotic theory of the likelihood ratio text i= estonsively sed 1 10.3.2 Other Lary Sample Tests Another common method of constructing a large-sample test. statistic is basen ‘un estimator that has an asymptotic norma) distribution. Suppose we wish} test hypothesis about a real-valied parameter 8, and Wy = W(X,,....Xy} js" point estimator of @, based on a sample of size n, that has been derived by sgeie method For example, Wr might be the MLE of 9. An approximate test, based on a. normal approximation, can be justified in the following way. If 2 denotes the variance of We aad if we can ase some form of the Central Limit Theoreni to show that, a¢ 100, (Wy ~8)/or converges in distribution to a standard norinal random variable, then (1, ~9}/om can be compared to a n(0,1) distribution” We therefore have the basis for an approximate test. 7 There are, of course, many details to be verified in,the argument of the previous paragraph, but this idea does have application in giany situations. Por example, if W,, is an MLB, ‘Theorem 10.1.12 can be used to yallidate the above arguments. Note that the distribution of W¥, and, perhaps, the yalue of a, depend on the value of The convergence, therefore, more formally gas that for each fixed value of 9 € sf we nse the corresponding distribution fo¥ 17, and the corresponding value for 5 (Wy ~0) on converges toa stondard nopfial: If, for each nas is calculable constaat (which my depend on # but aoe anger iminown parameters), then a fst based on (Wy —8)/on, might be derived.,/ ako depevids on unknown parametoes. Jn such a caso, we look that 94/55 converges in probability to L 18) we ean deuce that (1, 8)/Se large-sarmple tet Tn some instances, for an estimate Sp of cm withythe property Then, using Slutsky's Theopém (as in Example 5. also converges in disteibution bo a standard norinal distribution. A may be based on his, et. Suppose ne wishita test the two-sided hypothesis Hy: 8 = 0 versus Hy: 0 #9, An appreximate,fest can be based on the statistic Zy — (Iq ~ 6)/S%, and would reject Hy if and only if Zq < —2q/ OF Zp > 2qya- If Ho is true, then @ = dy and Z» converges igistribntion to Z ~ 0(0, 1}. Thus, the Type I Bexor probability, Poa (Bu © ~taia 08 By > tapa) > PLE < and isis an asymptotically sie a test. Plow consider an alernative pacameter value 04 fo. We can write A Wan Wao , 6~M% 2) yor Z> ’ (10.8.3) Section 10.3 HYPOTHESIS TESTING 95 No mattor what the value of 9, the term (W, ~0}/S, — (0,1). Typically, it is algo the case that c, — 0 as n + 20. (Recall, 3 ~ VarW,, aad estimators typiéally become more precise as n — oo.) Thus, S, will converge in probability to 0 aiid the term (0 ~ 8))/S, will converge to +a or —oo in probability, depending, on/vhether (@~ ) is positive or negative, Thus, Z, will converge to +20 or —20 in probability and Pylscject Hp) = PelZy < ~Z9/2 OF By > Faia) 1 asm Z In this way, a test with asymptotic size «and asymptotic power L edn be constructed, if we wish to test the one-sided hypothesis Ho: < fg versus M1: > Bo, a similar test: might be constructed. Again, the test statistie Z, = (Wy ~u)/Sn would be used and the test would reject Ho if and only if 2, > 2. Using teasoning similar to the Above, we could conctude that the power function of this test converges to 0, 0, or 1 according as 9 < 9,0 = 8, or 0 > bo. Thus this test tog’has reasonable asymptotic power properties, In general, 4 Wald test is a test based on a statistic’ of the form Wa % where Mp is hypothesized value of the paramnéter 8, W, is an estimator of 8, and Sy ‘sa standard extor for W,, an estimate of tio standard deviation of Wa. IF Wy is the MLE of @, then, as discussed in Section 0.1.8, 1/ /Ta(W) is @ reasonable standard, : fh errr for Wa, Alternatively 1/5/7098, where & we iW) og (0) ste ors infomaion er, ofan wed (101.7) Example 10.8.5 (Large-Sample binomial tests) Let Xy,..., Xa be a random sample froma & Bernowitp) population. Consider tasting Hla’ p< py weraus TI, P> pu, where 0 < ny’ Lis 8 specified value, The MLE of p, base! on a sample at size n, is fn = Feta, Nu/n. Since py, is just a sommple moan, the Central Limi Theorem applicn pf states that for any p. 0

2. As an alternative estionata nin it ie oaths sunple Wald