The Jackknife--A Review Rupert G. Miller Biometrika, Volume 61, Issue 1 (Apr., 1974), 1-15. Stable URL: http//links jstor.org/sici?sici=0006-3444% 28 197404%2961%3A1%3C1%3ATIRG3E2.0,COW3B2-4 ‘Your use of the ISTOR archive indicates your acceptance of JSTOR’s Terms and Conditions of Use, available at hhup:/www.jstor org/about/terms.html. JSTOR’s Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the sereen or printed page of such transmission. Biometrika is published by Biometrika Trust. Please contact the publisher for further permissions regarding the use of this work. Publisher contact information may be obtained at hup:/www jstor.org/journals/bio. html. Biometrika (©1974 Biometrika Trust ISTOR and the ISTOR logo are trademarks of ISTOR, and are Registered in the U.S. Patent and Trademark Office For more information on JSTOR contact jstor-info@umich.edu, ©2003 JSTOR hupswww jstor.org/ ‘Sun Feb 16 02:10:29 2003Biometrita (1974), 61, 1, pst 1 Printd in Great Britain The jackknife - a review By RUPERT G. MILLER, Department of Statistics, Stanford University, California Svantany Research on the jackknifo technique sinee its introduction by Quenouille and Tukey reviewed, Both its role in bias reduction and in robust interval estimation are treated. Some speculations and suggestions about future research are made. The bibliography attempts to include all published work on jackknife methodology. ‘Some key words: Bias reduction; Interval eatimation; Jacklenife; Paoudo-values Robustness. 4, Intropucrion Quenouille (1949) introduced a technique for reducing the bias of a serial correlation estimator based on splitting the sample into two half-samples. In his 1956 paper he general- ized this idea into splitting the sample into g groups of size h each, n = gh, and explored its ‘general applicability. Let Y, .»,¥, bea sample of independent and identically distributed random variables. Let be an estimator of the parameter @ based on the sample of size n, Let 6; be the corres- sponding estimator based on thesample of size (g— 1)h, where the ith group of size & has been deleted. Define 8, = 98— (9-184 (6 9). (tt) The est ator 1g 1 7 en 8-0-5 S06 (12) has the property that it eliminates the order 1/n term from a bias of the form E(O) = 0+a,/n+ O(1/n). In an abstract Tukey (1958) proposed that the g values (1-1) could be treated as approxi- ‘mately independent and identically distributed random variables in many situations. The statistic i eo (1) 2a should then have an approximate ¢ distribution with g—1 degrees of freedom and constitute a pivotal statistic for robust interval estimation. In unpublished work Tukey subsequently called the g values (1-1) pseudo-values and created the name jackknifed estimator for (1-2) in the hope that it would be a rough-and-ready statistical tool. ‘The research which has substantiated, amplified, and built upon this original work is surveyed in this article. Since there are the two different aspeets of the jackknife technique, namely, bias reduction and interval estimation, the survey is divided along these lines into2 Rurert G. Miter separate sections, §§2 and 3, respectively. Section 4 describes some additional develop- ments, and §6 contains some speculations and suggestions about futureresearch on the jack- knife. The bibliography attempts to include all published work on jackknife methodology. Although it will be necessary to mention the ease of general g and h at certain points, the presentation in this article centres ong = nand/h = 1. Much of the research on the jacklenife hhas been devoted to this special case. Tt is the most appealing because it eliminates any arbitrariness in the formation of the groups, and itis probably the best form of the jackknife to use in any problem. For lange data bases, however, this may be computationally not feasible, In most if not all instances any result proved for the case h = 1 can be extended to hoa 2, Bras REDUCTION 241, Second-order jackknife In 1956 Quenouille also gave a way to eliminate the order 1/n? term from a bias by jack- knifing with weights n* the jackknifed estimator. The second-order jackknife estimator is _ n= (n= 1858 =I where @_, is (1-2) applied to the sample of size n—1 with the jth observation removed. In ‘terms of the original estimator 4 the second-order jackknife is expressible a om ey @ = (n= 1) [099 (nt — n—1) (2 (n= 1)(n—2){—2 = (n= 1) [9 (an8— 20-4 1)(n—1) (F20.,) + (n= M2) nz): (22) where 0_, denotes the original estimator applied to the sample of size n—2 with the ith and jth observations removed. Tf B(0) = 0-+a,)n+-a,)n4, then E(@®) = 0+0(1/n*), but 6 is not unbiased. Schucany, Gray & Owen (1971) suggested modifying the weights to achieve complete unbiasedness when the bias has only first- and second-order terms in 1/n. Their estimator, which has simpler weights than Quenouille's (2-2), is Gor at [ran 1 & 2.) +(n—2)4 facay Zed] : (2-3) ‘These ideas can be extended to eliminate even higher-order bias terms if one so desires. ‘The generalization of Quenouille's coefficients in (2-1) should be clear, and the extension of (2:3) is a special case in the next subsection. 22, Generalized jackknife In the same 1971 paper Schucany, Gray & Owen generalized the jackknife technique to handle more general forms of bias. Suppose there are two estimators 0, and 0, based on all or parts of the data for which the biases factorize in the following manner: E(O,) = 9+f,(n)(0), (Bq) = 8 +f,(n)b(0).. 7 ‘Then the estimator POD OFA(HO), BOO) = 0+ fn) 010. es) (25)The jackknife — a review 3 is exactly unbiased. The usual jackknife (1-2) for g = n fits into the form (25) with 0, = 0, 0, = 30_4fn, flr) = Am and flo) = 11) To eliminate k separate terms in the bias, each of which factorizes into distinct functions ‘of nand 0, £+1 estimators are required whose expectations are of the form EG) = 0+ & SuhdbjO) (i b+). (26) ‘The functions f,(n) are assumed to be known. The generalization of (2:5) is Oo Ooo fal") Seraal”) fuxln) Oem Susulr)| t Savas) (27) 1 fualm) Ful) o- Sosa,el) ‘The refined second-order estimator (2:8) is a special case of (2-7) with 0, = 0, 0, = E0_4/n, 8 = 23,58 - 00. With little extra effort the proof ean be extended to the case where 6 = f() and is the root of an equation analogous to (3-1) with greplacing 0. ‘The function f needs to have a continuous first derivative. ‘Note that the above limit was taken by holding the number of groups g fixed and letting ‘the size of each group become large. It has been shown in ratio estimation that smaller mean squared error is achieved by increasing the number of groups rather than the size of each group, and one might guess this is true generally whenever it is computationally feasible to do 80. Also, by letting g->0o one obtains Gaussian critical values, which are smaller than ¢ critical values, so that the intervals may be sharper for large g and small h. However, no one has succeeded in proving that (1-3) has a limiting normal distribution when g = n—>co under the usual regularity conditions on p(y; 6). The proof can be pushed through by putting ‘much heavier conditions on p(y; 8) such as bounds on derivatives higher than the third, but ‘the details are so unpleasant that no one has seen fit to publish them. In an abstract Fryer (1970) reports results on the moments of maximum likelihood jack- Iknifing in the multiparameter case. The maximum likelihood problem is perhaps not so significant an application for the jackicnife as the remaining subsections because the distribution of the random variables Y;is specified. This means that considerable other distributional machinery can be brought to bear to determine the exact or asymptotie distribution of 8. It would be quite interesting to know what the jackknife gives when the assumed parametric model does not hold, 3.8. Punetion of a mean Let d = fi) where = E(Y,). Miller (1964) proved that for 6 = f(Y) the limiting distribu- tion of 1-3) withg = n isa unitnormal distribution as n->co, provided that var (¥,) = 08 <0 and f has a bounded second derivative near p. ‘The class of statistics which can be putin the form f( V) israther limited, but extension to similar variables such as f(s?) where 6*is the sample variance (Bfiller, 1968) was immediate, ‘The class was considerably broadened by the generalization in the next subsection to statisties of the form f(U) where the argument is a U-statistic, A U-statistic is in a sense a fancy mean,6 Rurerr G. MinuEr 3.4, Function of a U-statistic Uy Yq) = Tae D Hiya (2) (c) here the kernel function (yj. Ym) issymmetricin its m arguments and the summation is overall the combinations of m variables Y,,....Yj, outofthen variables Vy, ....Yqistermeda U-statistic. Let = E((Y,....Yq)}. The parameter of interest is f(x), and the associated estimator to be jackknifed is = /(U). Then Arvesen (1969) proved that (1-3) with g = nhas a limiting unit normal distribution a8 n>co provided that B{l*(Y,....Yq)} co, it ean be proved that (1-3) with g = n has a limiting unit normal distribution.The jackknife ~ a review 7 This result extends the valid use of the jackknifo to the estimation of linear or nonlinear ‘J(6) in unbalanced analyses of variance or regression problems. Although the details have not been checked, the proofs seem to generalize to the case of nonlinear least squares DY — 9x5 A}. 3.6. Stochastic processes with stationary, independent increments Gaver & Hoel (1970) were interested in estimating the reliability parameter 0 = e~* for fixed 7 > 0 where A is the intensity parameter of a Poisson process {¥,}. If the process is observed over the interval [0, T], then = ¥,/T' is everyone's estimator for A. A jackknife method of estimating 6 = e~*' is to divide the time interval (0, 7'] into m equal-length sub- intervals and to jackknife the ad hoc estimator 6 =~, If AY, = Yig—Yie_ya» where d=T/n, then the estimator with the ith subinterval removed is 6_, =e, where A= Wr—AY) (7-4). ‘Unlike the previous examples it is possible to pass to the limit n->0o, This gives the limiting estimator tin Bo eh fi -v(o-1-3)) oo) This ia not, as one might naively suppose, an unbiased estimator of 8, Gaver and Hoel com- pared this estimator with the unjeckknifed 8, the minimum variance unbiased estimator, ‘and several Bayesian estimators. Although (3°) is not dominated by any other estimator, it can be considerably improved upon by some of the other estimators for certain values of @. However, which estimator and which values depend on the ratio of r to 7. Gray, Watkins & Adams (1972) generalized this idea to stochastic processes {¥,} with stationary, independent increments. Since they need to restrict {Y;} to processes whose path functions are piecewise continuous and of bounded variation, the Wiener process component is eliminated and (¥,) reduces essentially to a sum of independent Polason processes with different jump sizes 7 and intensity parameters A,. Let 0 = f(A), where E(Y,) = ¥ and =Y,/P. Then, by dividing the interval into equal-length subintervals, jackknifing, ‘and pasting to the limit +00, one obtains the estimator lim = 70)-E4,[s(A-F) ~10 FO), es) where N, is the number of jumps fina ‘yin [0,7] and f" is the derivative off. Under the conditions that I’ = {7} is a bounded set, f has a bounded second derivative near A, and ieee DEYN ot = var(h) <0 (33) in probability as 700, the estimator (3-4) is asymptotically normally distributed with mean @ and variance o*{f'(A)}#/P, a8 To. The limit as n—+co of the jackknife variance estimate #/n = Z0,—9} {nn — » is tim F-za, {r(-#) -say (0) Under the conditions (3-5), ' bounded, and f’ continuous near A, (3-6) multiplied by 7’ converges in probability to o%{f(A)}? as 1". Thus, 7¥(im —0)/(im /n)# has a limiting unit normal distribution as 7'->co under the stated conditions,8 Rurerr G. Miner 3.7. Counterexamples ‘The reader should not acquire the impression that the jackknife always works. This is far from true. For the jackknife to operate correctly the estimator 6 has to have a locally linear quality. Miller (1964) demonstrated that for 8 = Yq), the largest order statistic, the limiting behaviour of (1-8) with g = n can be degenerate or nonnormal. Similarly, in unpub- lished notes L. E, Mosesshowed that when 0isthesample median the asymptotic distribution of (1-8) with g = n is normal but with variance 4 when nis even. For a truncation point problem with 8 = Yq) Robson & Whitlock (1964) modify the definition of the jackknife because of the particular bias expansion and derive 2¥q)— Yn as an estimator. Gray & Schucany (1972, pp. 18-8, 44-1) show how this and the higher order Robson-Whitlock truncation point estimators can be obtained from the generalized jackknife (2:5) and (2-7). In obtaining approximate confidence intervals Robson and Whit- lock abandon the ratio (1-8) and give arguments for the approximate confidence statement 1 Pe {fin +42 iy —Ya-n) > = ta 7) [An area in which the jackknife has had little or no succes is timo serie analysis ‘This x ‘an ironic twist because it was for a time series problem that Quenouille originally proposed the ida, Exeopt for the eae g = 2, the removal of data segmente from a serially corelated sequence of observations couse diffculty for the jackin For example, no one haa successfully found a way to make it provide valid estimates of the variability of smoothed estimates of the spectral density. 3:8. Main application: inference on variances ‘Standard textbook methodology uses the chi-squared distribution for setting a confidence interval on o? from s%, the sample variance. Similarly, the F distribution is employed to make inferences on o¥/o} from s/s}, the ratio of variances from independent samples. Pearson (1931) and Box (1953) sounded the alarm on the use of these procedures as well as their &-sample cousins, Bartlett's test, Hartley's test, and Cochran's test. The distribution theory for these techniques is precisely correct when the observations are normally distributed, but the error probabilities can be grossly inaccurate for nonnormal distributions. Mosteller & Tukey (1968) and Miller (1968) studied the application of the jackknife to logs? or log?—loga}. The jackknife variance does correctly estimate the variability of logs or logt—log ¢ for all underlying distributions, whereas the previously mentioned techniques rely upon a theoretical variability which is valid only when fo = 3, For two- sample hypothesis testing the jackknife seems to perform about the same as the Box Andersen technique, in which a beta approximation uses the fourth-sample moments to adjust the degrees of freedom. Both outperform Levene’s test and the variant of the Box technique where the sample is divided into subsamples and comparison is made between the logs? computed for each subsample. In the Box-type tests power is lost in the arbitrary division into subsamples. Shorack (1969) makes additional comparisons among competing ‘variance tests which are relevant to the preceding brief discussion. Layard (1973) examined the jackknife’s performance in the &-sample hypothesis testing problem. The jackknife and an asymptotic * test, which, like Box and Andersen’s test,The jackknife ~ a review 9 estimates the fourth moment directly, perform about the same, and Box’s test lags farther behind than in the two-sample probiem. Inference on variance components in problems such as Yy = tact ey, SP Lew, with a; ~ (0,02) and ¢j ~ (0,02), all random variables being independent, is also extremely sensitive to normality assumptions. Arvesen (1969) considers application of the jackknife to inference on 3, and Arvesen & Schmitz (1970) to inference on a3/a3. In the latter paper their Monte Carlo results reaffirm the unsuitability of normal theory techniques for general distributions, and demonstrate the robustness of the jackknife when applied to the log of the variance ratio, The results are not as good without the log transformation ‘More will be said in §6-3 about the need for transformations in conjunction with the jack- knife. In a technical report Arvesen and Layard give the theoretical machinery needed to justify the use of the jackknife in unbalanced variance component problems. 4, OTHER DEVELOPMENTS 41. Multivariate analysis ‘Normal theory tests on covariance matrices are also sensitive to nonnormality. Layard (1972) asymptotically described the nonrobustness of the usual tests for the equality of two covariance matrices and the effectiveness of the jackknife in dealing with this problem. In a recent paper Duncan & Layard (1978) investigate by Monte Carlo sampling the special case of the correlation coefficient in a bivariate population. It is dangerous to use Fisher’s transformation Z = tanh™1r for testing whether a correlation coefficient equals a specified nonzero value or for testing the equality of two correlation coefficients because of the extreme sensitivity of the variance of Z to a lack of normality. Jackknifing Z does, however, pro- duce a correct asymptotic variance. In an earlier paper Dempster (1966) proposed a modified jackknife when dealing with canonical correlations. His proposal is to delete single degrees of freedom rather than single- veetor observations, The procedure seems harder to follow than the ordinary jackknife, but this may be illusory. Another area of multivariate analysis in which the jackknife has found application is discriminant analysis. One can jackknife the discriminant coefficients to assess their variability, but the more interesting application is in the estimation of the error or mis- classification probabilities. The method of testing each vector observation in the samples, with the discriminant function computed from all the observations, in order to estimate the errorrateshas been known forsometimeto besubjecttoserious bias. Thereare variousnormal and nonparametric procedures alternative to this, and one is to test each observation with. ‘the discriminant function computed from the data. with that particular observation re- moved. This has been termed the U method by Lachenbruch & Mickey (1968), and in the same journal issue Cochran (198) referred to this method as an application of the jackknife principle although this is not precisely correct. Mosteller & Tukey (1968) examined the performance of the U method on the discrimination problem created by the Federalist: Papers. A synopsis of the Mosteller-‘Tukey work is given by Gray & Schueany (1972, pp. 115-36).10 Rovzrr G. MLLER £2, Infinitesimal jackknife Ina recent Bell Telephone Laboratories technical memorandum L. B. Jaeckel has intro- duced the concept of an infinitesimal jackknife. At the moment it does not appear to be as practically useful as the ordinary jackknife, but it establishes what could be an important ‘bridge between the jackknife and therecently developed theory for robust estimation of the location of a symmetric distribution (Andrews ef al. 1972; Huber, 1972) To understand the connexion itis necessary to briefly summarize some relevant aspects of the theory for robust estimation. Many estimators 6 are equal to, or are asymptotically equivalent to, functions 7(F) of the sample cumulative distribution function P, where T is defined over a wide class of distribution functions. In particular, the unknown parameter @ is T(P), the valueof the function at the true F. Under regularity conditions estimators ofthis type can be expressed in the form TP) = TUF) + [T'(P, ya -F) (y) +0904), (41) where 7"(F, y) is a von Mises (1947) derivative, defined by lim. Fr +66) T(F) ‘The term influence curve has been attached to 7"(F, y) by F.R. Hampel because it measures to what dogroo a change in the mass at y will change the estimate, If the function Tis assumed to be normalized so that T(cF) = T(F) for all F and c > 0, then [7"(F,y)aF(y) = 0, 90 that TB) = TE) +E EE, Y)+0,00-4). (43) = [Tr yaay. (#2) ‘The average of independent and identically distributed random variables in (4-3) is asymp- totically normally distributed with mean zero and variance Lye. yar). (44) If 7" is known, an empirical estimate of the asymptotic variance is SE remy. Co) To define the infinitesimal jackknife think of the estimator 8 as. function 7(Y;w) of the observations ¥ = (Hy,....¥q)’ and arbitrary weights w= (W,...,10,)'. If w,= 1/n, then 6 = 7(F). Also, suppose that the function of the observations and weights is self-normalizing in the weights 90 that 7(Y;ew) = 7(Y;w) for all ¢ > 0. For the ordinary jackknife 04-0 (%, but for the infinitesimal jackknife the weight of the ith observation is reduced only to 4m—e instead of to 0, namely, 8.40 =1 (Ky. (#6) (47)The jackknife — a review Ml By analogy with the ordinary jackknife the infinitesimal jackknife estimate of the asymp- totie variance of 8, or its infinitesimally jackknifed version, is defined to be FO C59 (0.4-i20 so} (8) When ¢ = 1/n, this coincides with the ordinary jackknife estimate, ‘Suppose that 7(y; 0) is differentiable with respect to the weights. Then, let ATiy;w) Dy = SEU) 40) foie (40) where the derivatives are evaluated at y= ¥, w= (1/n,...)1/n)'. The self-normalizing condition on the weights in 7" implies that 2D, = 0. From the expansion O46) = 0-cD, + 4eDy—..., (4-10) it follows that FO) tim 20 _ 1 $ py (411) me me But D, is precisely 7"(F, ¥,) 80 (4-11) equals (46). Thus, the infinitesimal jackknife variance estimate (4-11) provides an estimate of the asymptotic variance (44). For the ordinary jackknife 9-8 (n—1 (528.6) (412) estimates the bias of 0. Correspondingly, let ire ft ‘Expression (4-18) equals (4-12) when ¢ = 1/n, From the power series expansion (4:10) it follows that 5) 240-0}. (13) 50) = tim Be) = BD (414) ty be = as By ‘The infinitesimal jackknife estimate is defined to be 6(0) = 6—5(0). ‘Taeckel proves under general conditions, which will not be cited here, that for estimators of the form (4-3) 4%(0) and n5(0) converge to the correct asymptotic constants as n->co. Tt is not at present clear how easy it is to check these conditions in particular examples. Jacckel also shows that under the same conditions the ordinary jackknife estimates behave correctly asymptotically. £3, Miscellanea Salsburg (1971) gives an application of the jackknife to testing in quantal response bio- assay where the probabilities of success at the observed dose levels are all near t (or 0). This application is also described in a review paper by Arvesen & Salsburg (1973), In a recent paper Gray, Watkins & Schucany (1973) explore the connexion between the jackknife and uniform minimum variance unbiased estimation of f(x), f(o?) and f(u, 04) for special f, where and o? are the parameters of a normal distribution. In the discussion of a paper by P.Sprent on linear functional relationships Brillinger12 Rurerr G, MILLER (19665) proposes the use of the jackknifo to assess the variability of the estimators. In another forthcoming paper P.O. Anderson has studied the jackknife's effect on regression estimators which minimize the orthogonal distance. 5, New pingcrroxs 5.1. General remarks Where should research on the jackknifo go from here? What worthwhile questions on the jackknife still remain unanswered? Listed below are some thoughts on this which are by zno means all-inclusive, 5.2. Linear combinations of order statistics Estimators based on single order statistics such as the median or maximum (§3'7) donot behave properly under jackknifing. What about smooth functions of the order statistics such as 8 = EJ(i/n)¥,o/n, where J is a continuous function? Estimators like this arise in robust estimation of the location of a symmetric distribution and overlap with the class of estimators (4-3) where the jackknife is known to work under regularity conditions; see §42 ‘How broad is the class of estimators for which (4-3) and the needed regularity conditions are satisfied? Will the jackknife accurately assess the variability of the more complicated adaptive estimators? Obtaining good estimators of the variability of robustestimatorsseems to be an open and important question. 5:8. Transformations ‘Most advocates of the jackknife would suggest using a variance stabilizing transforma- tion on the estimator in conjunction with the jackknife. Examples are jackknifing log st and tanh-r instead of s# and r. However, the connexion between transformations and the jackknife is more than just a nicety. Transformations are needed to keep the jackknife on scale and thus prevent distortion of the results. Consider inference ono, Without the log transformation the pseudo-values ‘ns*—(n—1)st, can be negative. The jackknife does not know variances must be posi- tive. Sizeable or frequent negative psoudo-values can produce distorted point and interval estimates. With the log transformation a negative pscudo-value corresponds to ‘a small variance, and this will not pull the aggregate so far to the left. Table 1. Control blood flow data Dye 11517014338 38O TO BOL 8.00 zr 13872189 AT 88345 BT LBL 8-7, ‘The following data illustrate the analogous point with correlation coefficients. A medical investigator at the Stanford Medical School wanted a measure of association between two techniques for measuring blood flow. A standard method (py®) is to inject dye into the pulmonary artery and sample it from the aorta. A computer integrates the experimentally determined curve of dye concentration to obtain a blood-flow measurement, The electro- magnetic flow probe method (RFF) is newer. In it a cuff placed around the aorta creates an lectrical field to measure the blood flow. The Dy method is extremely variable, and the vr has serious calibration problems. ‘To assess the amount of agreement between the two ‘methods, essentially simultaneous measurements were made on nine dogs. ‘The data in appropriate units are displayed in Table 1.‘The jackknife ~ a review 13 ‘The number of observations may be rather small to be trying the jackknife, but it is interesting to seo what it gives. The correlation cooificients and pscudo-values with and without the transformation tanh~ appear in Table 2. ‘The correlation coefficientestimate ; is quite remarkable ! The untransformed jackknifed estimateis = 0-9452 andits estimated standard erroris 00408, The jackknife has increased tthe estimated value of p slightly, which is probably in the wrong direction. Because of the size of the standard error, a confidence interval for p will extend well beyond the upper limit. 1-0. The transformed values give taih-'r = 1-593 which corresponds to a p value of 0-9206. The estimated standard error of tdtih~*r is 0-471 so that a 95 % confidence interval for p is (0-4672, 0-9908). The peculiar pseudo-value — 1-724 is what makes the interval so broad. Table 2. Correlation coefficients and pseudo-values for control blood ‘flow data with and without transformation be taming taht, Eee tambien, tatty 0 09s 80 50-9766 o-6004 2218 10-0408 9784143 2.078 8 0.9396 0-986 1-735 2 094k 0-950 17681876 70-9205 11302 1-598 30-0434 —0-9560 17081-8768 80-9398 9881-735 4 09408 0.9768 1-745 2.060 90-9583 08368 1-025 ‘Table 3. Propranolol blood flow data bye 077 087 10521380055 B88 mre 1030-95125 168 3:30 O80 B12 ‘The connexion between transformations and the jackknife is worth more exploration than a single numerical example and a few published Monte Carlo results (Arvesen & Schmitz, 1970). Is there an optimal way to select a transformation for use with the jack- knife Bt, Outliers ‘The jackknife is not a device for correcting outliers. The following numerical example illustrates this. On seven of the dogs in the experiment described in §5°3, first a stimulant, isuprel, and then a depressant, propranolol, were administered after the data listed in ‘Table 1 had been obtained under control conditions. Dual measurements on DYE and FP ‘were taken under these induced states. The propranolol data are displayed in Table 3. The fifth point could be considered an outlier because in all other pairs DYE < BFP. ‘The transformed correlation coefficients and pseudo-values are presented in Table 4. ‘The jackknifed estimate is tfthr = 0-375 which corresponds to a value of p of 0-359. The jackdenife has pulled the estimate down from 0-948 as one might hope, but it seems to have ‘gone beyond the bounds of reason. The estimated standard error for tatth"1is 1-077 60 that a 95 % confidence interval for p is (— 0-978, 0-995). Tt would be interesting to study theoretically how outliers perturb the jackkmnife esti- mator. Is the examination and correction of pseudo-values a good way of handling outliers? 5:5, Higher-order variance estimate ‘At the moment the variance estimate for the jackknife involves only the pseudo-values from the first-order jackknife. Can the higher-order jackknife expressions be used in any way to improve the variance estimate?14 Rurerr G, Minier, Table 4. Transformed correlation coefficients and pseudo-values ‘for propranolol blood flow data ‘ te Gahty, ° 0-9480 — 1 0-9436 2.008 2 0-9457 1-950 3 0-9463 014 4 0.9519 1872 5 0-9950 — 5-286 6 0.9423, 24136 1 0.9837 -1m2 56, Multisample jackknives ‘To illustrate this question consider a two-sample problem. One way to jackknife is to compute an estimate of the unknown parameter from the two samples and then jackknife by successively deleting each observation in the first sample with the second sample intact and then deleting observations in the second sample with the first intact, An alternative ‘method of jackknifing is to jackknife each sample separately and combine the results, Both methods are valid asymptotically, but is either one better than the other? Similar questions arise wheni there are three or more samples and when the observations come from an experimental design of some sort. ‘This work was partially supported by a fellowship from the John 8. Guggenheim Memorial ‘Foundation and by a grant from the National Science Foundation. The paper was written while the author was visiting Imperial College and London School of Hygiene and Tropical Medicine. D.R. Cox and D. V. Hinkley contributed a number of the suggestions for future research in §6, and the author thanks them for their assistance in the preparation of the paper. RevERexces ‘The numbers in bracketa at the end of each reference give the sections of the paper in which the reference is mentioned. Apans, J.E., Gray, H. L. & Wamains, T. A. (1971). An asymptotie characterization of bias reduction by jackknifing. Ann. Math. Statit. 42, 1608-12, [52-2] Axpanws, D.F, Bioxes, P.J., Hawret, F. R., Hosen, P. J.,Roosns, W. H. & Toxey, J. W. (1972). ‘Robust Estimates of Location: Survey and Advances. Prinooton University Pros, (§42] Anvusnx, J.N. (1969), Jackknifing U-statiatios, Ann, Math. Statist. 40, 2070-100, [§§34, 3-8). ‘AnvrsEx, J.N. & Sarap0na, D. 8. (1973). Approximato tests and confidence intervals using the jack- ‘knife, In Perapoctives in Biometrics, Ed. R. M. Elashoff. New York: Acadomic Proas, (§4°3). Anvasex, J.N. & Somurz, 'T. H. (1970). Robust procedures for variance component problems using ‘the juckdnife. Biometrics 26, 677-86, [§§3°8, 53] Box, GE. P. (058). Non-normality and tests on variances. Biometrika 40, 318-35, (89-8). Burcunae, D. R. (1964). The aaymptotie behaviour of Tukey's general method of eotting approxi- ‘mato confidence limits (the jackknife) when applied to maximum likelihood estimates. Rev. Ina. Int. Statist. 32, 202-6, (§3-2 BuiuNaER, D.R. (19660). The application of the jackknife to the analysis of sample surveys. Commentary 8, 74-80, (§2:3). Baruiame, D.R. (19666). Discussion of paper by P. Spront. J. R. Statist, So. B28, 294, (943) ‘Cuaxnananry, R.P. & Rao, JN. K. (1008), Tho bias and stability of the jack-lnife variance eeti ‘mator in ratio catimation. J. Am. Stati. Assce. 63, 748-9, [523]. Coounax, W. G. (1968). Commentary on “Estimation of error rates in discriminant analysis.’ Techno. ‘metrica 10, 208-5, [541].The jackknife - a review 16 Denisa, W. B. (1963). On the correction of mathematical bias by use of replicated designs. Metrita 6, 37-42, (92°. ‘Dexesmm, A.B. (1966). Estimation in multivariate analys ‘Krishnaiah, pp. 315-34. New York: Academic Pres, (§41. Doxeax, G.P. & Lavan, M. W.J. (1973). A Monte-Carlo study of asymptotically robust teta for ‘correlation cooficienta. Biometrika 60, 561-8, [54-1] ‘Donny, J. (1989). A note on the application of Quenouille’s method of bias reduction to the estimation of ratios, Biometrika 46, 477-80, [523] Paver, J. (1970). Jackkniing maximum likelthood estimates in the multiparameter case (Abstract). "Ann. Math. Statist. 41, 1392, (53-2) Gaven, D.P., Ja, & Hort, D.G. (1970). Comparison of certain small-sample Poisson probability ‘estimates. Pechnometri 12, 835-80, (63-6) Gnay, HL. & Sonvcany, W.R. (1972). The Generalized Jackknife Static. New York: Marcel ‘Dekker, (962-2, 37, 41]. Gray, HL, Warxivs, T. A, & Apants, J. E,(1972). On the jackknife statisti, its extensions, and its rolation to gy-transformations. Ann. Math, Statist. 3, 1-30, ($22, 36] Gray, H. L., Warxmss,'T. A, & Somucany, W. R. (1978). On the jackknife statistic and its relation to ‘umnvu estimators in the normal case. Commun. Statist. 2, 286-808, [§43} Honan, P.J. (1972). Robust statisti: A review. Ann, Math. Statist, 43, 1041-67, (54-2). ‘Horomsox, M, C. (1971). A Monte Carlo comparison of some ratio estimators. Bizmetrika 58, 313-21, (23). LAcHENDRUGH, P.A. & MicKEY, M.. (1068). Ratimation of error ratos in discriminant analysis. ‘Pechnomedrice 10, 1-11, ($41) Lavann, M.W.J. (1972). Large sample tests for the equality of two covariance matrices. nn ‘Math. Statist. 43, 128-41, ($41) Lavan, M.W.J. (1973). Robust large-sample teste for homogencity of variances. J. Am. Statist. “Assoe. 68, 195-8, ($38). ‘Maxrer, N. (1967), Assumption-freo estimators using U statistics and a relationship to the jackie ‘method, Biometrics 23, 867-11, (83-4) Matus, R. G. (1964), A trustworthy jackknife. Ann. Math, Statist, 38, 1504-605, [§65°, 8-7). Maran, R. G., (1968). Jackknifing variances. ann. Mach. Statist. 38, 567-82, (83:3, 38). Matumn, RG. (1974). An unbalanced jackknife. Ann. Statist. 2. To appear. (§3°5} vox Mists, R. (1047). On the agymptotic distribution of differentiable statistical functions. Ann. ‘Math. Statit. 18, 309-48, (§42}. Mosruiizs, F. & Toxzy, J. W. (1908). Data analysis, including statistes. Handbook of Social Paycho- logy, Eds. G. Lindzoy and E. Aronson, Reading, Mass: Addison-Wesley, (§§3°8, 41]. Pransox, E.8. (1931), The analysis of variance in cases of non-normal variation. Biometrika 21, 114-85, (58:8) Qupxovrize, MH. (1949). Approximate testa of correlation in time-series. J.R. Statist. Soc. B 11, 68-84, (51) Qupwourx, M. HF. (1956). Notes on bins in estimation. Biometrika Rao, J. N. K, (1965). A note on the estimation of ratios by Quenouil (523) Rao, J.N.K. & Breaux, L. D. (1967). A Monte Carlo study of some 47-56, ($23, Rao, J.N.K. & Wansra, J. (1960). On two methods of bias reduction in the estimation of ratios. Biometrika 83, 571-7, (823). Rao, B.S. R. 8, (1969). Comparison of four ratio-type estimates under a model. J. Am. Statist. Assoc. 64, 574-80, [§2°3) Rao, P.S.R.S. & Rao, J.N. K. (1970). Some small sample results for ratio estimators (Abstract). ‘Ann. Math, Statist. 41, 1141-2, [§2°3) Ronsox, D. 8. & WarrLoox, J. H. (1964). Estimation ofa truncation point. Biometrika 81, 38-9. (58-7). Sausnono, D. (1971), Testing doso responses on proportions near zero or one with the jackkei ‘Biometrics 27, 1035-61, (643) Somveany, W.R., Gnax, H. L. & Own, D. B. (1971). On bias reduction in estimation. J. Am. Stats “Assoc. 66, 524-83, [H3-1, 22}. Snonack, G. R. (1968). Testing and estimating ratios of soale parameters. J. Am. Statist. Asse. 64, 999-1013, (§5-8). ‘Tax, M. (1965). Comparison of some ratio estimators. J. dm. Statist. Assoc. 60, 204-307, [52:3]. ‘Poxsy, J. W. (1958). Bias and confidence in not-quite large samples (Abstract). Ann, Math, Statist. 29, 614, (811 In Multicariate Analysis, Bd. P.R. 358-60, [661, 2]. ‘method. Biometrika 82, 647-9, i estimators. Sankhya B29, [Received February 1973. Revised May 1973]