STATISTICS IN MEDICINE, VOL.

8,323-329 (1989)

DETECTION OF ABERRATIONS IN THE OCCURRENCE

OF NOTIFIABLE DISEASES SURVEILLANCE DATA

DONNA F. STROUP, G. DAVID WILLIAMSON AND JOY L. HERNDON

Diuision of Surveillance and Epidemiologic Studies, Epidemiology Program Office, Centers for Disease Control,
I600 Clifton Road, N.E., Mail Stop CO8,Atlanta, Georgia 30333, U.S.A.

AND

JOHN M. KARON
AIDS Program Office, Centers for Disease Control, I600 Clifton Road, N . E., Mail Stop G22, Atlanta, Georgia 30333,
U.S.A.

SUMMARY
The detection of unusual patterns in the occurrence of diseases and other health events presents an important
challenge to public health surveillance. This paper discusses three analytic methods for identifying
aberrations in underlying distributions. The methods are illustrated on selected infectious diseases included
in the National Notifiable Diseases Surveillance System of the Centers for Disease Control. Results suggest
the utility of such an analytic approach. Further work will determine the sensitivity of such methods to
variations in the occurrence of disease. These methods are useful for evaluating and monitoring public health
surveillance data.
KEY WORDS Surveillance Aberrations Clustering Bootstrap Scan

INTRODUCTION
The detection of unusual patterns in the occurrence of diseases and other health events presents an
important challenge to public health surveillance. Aberrations in usual distributions of disease
incidence may provide an early signal of an epidemic or may provide clues to important risk
factors associated with the occurrence of a disease in time or space. The concern with the detection
of aberrations or ‘clusters’is not a new idea, particularly in the area of chronic diseases.’. In this
report, we differentiate between ‘clusters’and ‘epidemics’, using the former term for aberrations in
usual patterns of disease occurrence without requisite epidemiologic verification of excess.
The purpose of this investigation is to describe an analytic framework for detecting aberrations
in disease occurrence, using an existing data set. The methods illustrated here could also be used
over time to develop baseline measures and identify areas that might need further attention as part
of a surveillance system.
This work first describes the surveillance data and the statistical problems inherent in these
data. We discuss three methods for detecting aberrations in these data and the advantages and
drawbacks to each method. We conclude with recommendations for further work upon which
recommendations for use will be based.
Since 1944, the Public Health Service has collected national surveillance data each week on
cases of up to 50 notifiable diseases. Data are submitted to the Centers for Disease Control (CDC)
0277-67 15/89/030323-07$05.00 Received September 1988
0 1989 by John Wiley & Sons, Ltd.
324 D. F. STROUP E T AL.

by state health departments and are disseminated in the Morbidity and Mortality Weekly Report
( M M WR). A particular strength of this surveillance system is its timeliness. Reports of cases
received in a state health department are forwarded to CDC and disseminated to epidemiologists,
clinicians, and other public health professionals within a week of receipt.
The impetus for this investigation was a proposal to increase the usefulness of the tables that
currently appear in the M M W R (Figure 1) with a graph (Figure 2). In Figure 2, the vertical axis at
1 indicates that the current month’s report is equal to the baseline. Diseases with elevated reports
are shown with bars to the right; diseases with lowered reports are bars to the left. The shading on
bars indicates the amount of the report in excess of a 95 per cent limit (note that the ratios are
plotted on a semi-log scale). The problem then is how to determine this limit.

METHODS
Reporting of cases to CDC has been shown to vary by factors unrelated to the disease process -
such as reporting practices or time of month.3 Thus, to reduce the variability from these causes, we
will aggregate disease reports over a four-week period. We will refer to this four-week period as a
‘month’ for simplicity of expression. Let xo be the number of cases of a given disease reported to
CDC in the four-week period ending with the current week. We compare this current value with a
baseline report consisting of 15 previous totals, denoted xl, . . . ,x15,taken from the correspond-
ing month and the surrounding months in each of five previous years (Figure 3). This choice of the
baseline implies that the resulting measure addresses the question: is the number of cases this year
different from that last year. Thus, expected seasonal fluctuation should not be apparent.
Several aspects of our method were determined by constraints imposed by the surveillance
system. The constraints imposed by weekly dissemination do not allow different assumptions to
be checked or different models to be fit for different diseases or time periods. The choice of 15as the
number of baseline measurements enables all of the notifiable diseases under study to be analysed
with the same algorithm, regardless of the date the disease became notifiable.
We assume that, even if the disease exhibits seasonal variation, in the absence of any aberration,
xl, . . . , x15and xo are independent random variables with the same distribution function. For
most diseases, the three-month window produces data satisfying this assumption. Thus, we
calculate a two-sided confidence interval for the ‘expected’number of cases for a four-week period
for a given disease, then use this confidence interval and the ‘observed’ current value xo to
conclude whether the disease process is ‘out of range’ for the current month. For the purposes of
this investigation,we chose March 1987 as the current month and used data on national incidence
of eleven reportable diseases.
For example, CDC received 2080 reports of hepatitis A during the month of March 1987. For
the baseline period (February, March, and April 1982-1986), we received an average of 1778 case
reports per month. The epidemiologic and statistical question is: in what sense should the expected
number of cases of hepatitis A for March 1987 be 1778, and is the number of cases reported for
March 1987 excessive? After all, the years chosen could be highly atypical of hepatitis A reporting
in general, or a trend in hepatitis reporting could be present during those years. The law oflarge
numbers guarantees that in large samples the estimate of a mean or median calculated from the
sample is very likely to approach the true value of the parameter for the entire population. A
sample of only 15 months, however, is not a large sample.
We have chosen to present our results as a ratio y of the current value xo divided by some
measure of central tendency of the 15 baseline values. For the measure of central tendency, we
evaluated the mean and the median. The choice between these measures is affected by one’s desire
to control the sensitivity of the method with regard to aberrations (either high or low) in the
Table 111. Cases of specified notifiable diseases, United States, weeks ending July 16, 1988 and July 18, 1987
(28th Week)

Encephalitis Hepatitis (Viral), by type

Aseptic
Menin- Post in- Gonorrhea Unspen- Legionel-
AIDS gitis Primary fectious (Civilian) A B NA, NB fied losis Leprosy
Cum. Cum. Cum. Cum. Cum. Cum. Cum. Cum. Cum. Cum. Cum. Cum.
Reporting Area 1988 I988 1988 1988 1988 1987 1988 1988 1988 1988 1988 1988

UNITED STATES 16,652 2,358 366 84 358,613 420.662 12.878 11.618 1,359 1,144 454 94
NEW ENGLAND 71 1 102 12 1 10,741 13,088 470 708 85 68 16 11
Maine 23 7 1 22 1 382 14 29 3 1 2
N.H. 17 13 1 137 213 30 35 5 3 1
vt. 8 5 3 77 111 5 19 5 2 1
Mau. 397 42 6 1 3,673 4.756 220 444 57 50 11 10
R.I. 37 28 !?93 1,108 55 61 9 3 1
Conn. 229 7 1 5,640 6,518 146 120 6 12
MID. ATLANTIC 5,565 219 38 4 54,178 69.044 790 1,519 83 130 114 8
Upstate N.Y. 757 124 26 1 7,452 8.967 430 399 38 13 50
N.Y. City
N.J.
2.966
1,334
48
47
7
5
3 22,853
7,856
38,859 170
129
700
347
9
27
91 17 ;
1
8.668 26 20
PO. 508 16.017 14,550 61 73 9 27
EN. CENTRAL 1.238 314 92 7 58.193 m959 852 1,270 117 61 100 1
Ohio 276 106 28 2 13,246 13,419 189 308 18 10 42
Ind. 80 38 11 4.489 4.821 77 180 11 15 8
111. 552 49 19 5 16,188 18,829 240 233 43 13
Mich. 261 108 23 16.033 18.309 206 407 27 20 40
Wia. 68 15 11 4.237 5,578 140 142 18 3 10 1
W.N. CENTRAL 410 102 25 5 14.572 17,030 765 558 65 19 52 1
Minn. 88 19 2 2 1.987 2,654 58 80 11 3 2
Iowa 21 19 8 1.091 1,835 33 50 11 1 13
Mo. 21 1 33 1 8.233 8.797 432 331 30 9 10
N. Dak. 2 4 84 159 3 3 2 4 1
S. Dak. 5 3 1 1 288 310 6 3 2 14
Nebr. 25 3 4 2 880 1,101 29 32 5
Kana. 58 19 5 2,029 2.374 204 59 9 2 7 1
S. ATLANTIC 2.838 561 49 25 106,000 110,507 1,088 2.382 204 165 81 1
Del. 30 11 2 1,502 1,685 20 70 6 1 7
Md. 327 62 4 3 10.415 12,547 147 374 20 10 11 1
D.C. 275 11 1 7,524 7.41 1 11 27 3 1
Va. 182 57 19 3 7,132 8,055 225 168 47 107 6
w. Va. 8 10 2 743 804 8 31 2 3
N.C. 154 73 14 16,593 16,528 181 41 7 44 25
S.C. 79 10 1 9.179 9,161 26 306 8 3 12
GI. 432 70 1 20,220 18.947 203 369 8 3 11
Fla. 1.351 257 7 17 32,692 35,369 265 620 66 37 9
E.S. CENTRAL 404 173 29 6 27,651 31,268 390 705 92 6 18 1
Kv . 50 52 10 1 2.684 3,153 333 123 36 2 6
Tann. 177 14 6 9,320 10,996 34 349 25 6
Ala. 108 86 13 2 8,857 9,972 8 183 25 4 2 1
Mirs. 69 21 3 6,990 7.147 15 50 6 2
W.S. CENTRAL 1A1 5 273 37 3 40,362 47,272 1,446 949 105 283 11 19
Ark. 49 5 2 3.895 4,927 172 56 1 8 2
La. 205 48 12 8,329 8,663 76 189 16 9 4 1
Okla. 68 21 4 3.675 5,210 251 96 24 19 5
Tex. 1,093 199 19 2 24,463 28.472 947 608 64 247 16
MOUNTAIN 551 92 19 2 7.81 1 11,049 1,637 913 147 97 25 1
Mont. 8 2 239 290 23 32 8 3
Idaho 6 1 212 404 90 60 4 3
wvo 3 1 129 255 4 6 3 2
Cob 210 34 3 1,795 2,384 123 116
137
42
11
46
7
5
1
1
N. Mex. 26 5 2 723 1,205 35 1
Ariz. 169 27 5 1 2.751 3,614 916 350 44 27 12
Utah 42 13 4 1 312 339 208 84 26 13 2
Nev. 87 9 5 1,650 2.358 122 126 9 4 3
PACIFIC 3,720 522 65 12 41,105 60,445 5,240 2.614 461 315 35 51
Wash. 234 3 4 3,362 4.582 1.140 395 87 30 10 3
Oreg.
Cab!.
Alaska
95
3.326
13
460
11
59
2
e 1,680
35.100
603
2,260
52,202
91 1
81 1
3.112
171
325
1.831
24
47
322
4
13
264
4
22
1
39
1
Hawaii 52 51 1 360 490 6 29 1 4 3 7
Guam 1 86 116 5 7 2 1 3
P.R. 769 23 2 1 778 1.173 25 152 25 27 3
V.I. 24 216 139 1 5 2
Amer. Samoa 45 45 2 4 2
C.N M.I. 27 1 2 4

N: Not notifiable U Unavailable C.N.M I Commonwealth of the Northern Mariana Islands

Figure 1. Tabular presentation reproduced from the Morbidity and Mortality Weekly Report 37(28),433 (1988)
326 D. F. STROUP ET AL.

DISEASE

AIDS
Aseptic Meningitis
Encephalitis
Gonorrhea
Hepatitis A
Hepatitis B
Hepatitis NonA NonB
Hepatitis Unspec.
Legionellosis
Malaria
Measles
6J
Beyond 95% CL

Ratio I
Meningococcal Inf.
Mumps
Pertussis
Rabies, animal
Rubella
Syphilis
Tuberculosis
.25 .5 1.o 2 4 8
Ratio of Current Report to Five-year Average

Figure 2. Proposed summary-Notifiable Disease Reports, United States, comparison of four weeks ending
28 March 1987, with same quarter of previous five years

Feb Mar ADr

1987 Current
Period
86

85

84 Confidence
Interval
83

82

Figure 3. Baseline for comparison cases reported for March 1987

baseline. Use of the mean in the denominator would lower the ratio when an epidemic is present in
the baseline, making the system less likely to detect an aberration in the current period.
We investigated three analytic approaches to the problem of the excess of the value xo
compared to the baseline. Results from each approach were evaluated by contacting the CDC
programme responsible for the disease to discuss whether the statistical deviations were consistent
with an epidemiologic change, such as an outbreak requiring health department assistance, or an
increase in laboratory isolates.
 DETECTION OF ABERRATIONS 327

Method 1
Method 1 is a classical parametric method which uses the 15 baseline values to compute a normal
theory confidence interval, using the mean of the baseline period in the denominator. A resulting
characteristic of the method is that extreme fluctuations in the data from the baseline (such as
might result from an epidemic) may obscure elevations in the current period, since the mean is
sensitive to the extreme values. An estimator which is more robust to deviations in the baseline
would use the median of the past values. However, shifting from the mean to the median
invalidates the normal theory bounds of Method 1. Additionally, the assumptions of this method
would require that the data be normally distributed for each disease and each time period.

Method 2
Method 2 uses the b o o t ~ t r a pa, ~computer-intensive procedure. The motivation for applying the
bootstrap estimate relates to shifting from the mean to the median in the baseline to obtain an
estimator that is robust with regard to past epidemics. The bootstrap method gives asymptotically
correct confidence intervals for the median. Furthermore, the bootstrap is a non-parametric
method, and thus avoids the need to check assumptions for each application.
Repeated random samples (taken with replacement) from the observed sample simulate the
distribution from which the original sample was drawn. The bootstrap sampling is repeated, say,
1000-5000 times. The name ‘bootstrap’ reflects the fact that the one available sample may give rise
to many others. Confidence limits are the appropriate percentiles of the bootstrapped distribution;
for example, for a 90 per cent confidence interval, we use the 5th and the 95th percentiles of the
bootstrapped distribution.
The 15 baseline values used in Method 1 are not a sufficiently large sample for the asymptotic
results to hold for the bootstrap, and results are too sensitive to the particular characteristics of the
small sample. Thus, we applied the bootstrap methodology to a different set of data. For Method
2, we compute every available ratio (each based on a five-year baseline) for the three years
immediately preceding the current year. Thus, for any four-week period in 1987, each consecutive
four-week period in 1984-1986 is computed. This will yield 156 ratios over the three-year period,
using data back to 1979 (the five-year baseline for the 1984 ratios).
To motivate this method further, consider the cumulative empirical distribution functions for
these 156 ratios (Figure 4). The horizontal axis shows the range of values of the ratios, and the
vertical axis shows the cumulative percentage below a given ratio. For example, 40 per cent of all
ratios are below 0.90. The empirical distribution function of the median from the bootstrapped
samples has less variability than the empirical distribution function based on samples of size 15,
reflecting the larger sample size and yielding a more useful estimate of the underlying distribution
of the median.

Method 3
Method 3 is the scan stat is ti^,^, developed to monitor disease incidence. The scan statistic
assesses the significance of the number of cases in the current month, given the number in each
month of the baseline. The statistic can be computed by plotting the number of cases reported over
time, taking a ‘moving window’ of fixed length, and then finding the maximum number of
observations in the window as it scans the entire time period. For example, for data reported
monthly, the probability of at least 10 case reports in 1 month, given a total of 40 in the past 12
months, is computed (using a Poisson distribution) to be 0.06.
328 D. F. STROUP ET AL.

u
al
al
L
a 60

Ratio

Figure 4. Empirical distribution functions

RESULTS
Epidemiologic data showed elevations for mumps and hepatitis A during March 1987. Method 1,
the parametric approach, was applied to these data and excesses were seen for hepatitis A and
mumps (Figure 2). A significant decline also was noted for gonorrhea. The decline in gonorrhea,
however, was not associated with any epidemiologic confirmation.
Method 2, using the bootstrap on all the available ratios, gives results consistent with those
from Method 1. It also found significant increases in the number of cases of mumps and hepatitis
A, as well as in the number of cases of meningococcal infections. The decrease in gonorrhea was
not significant. Although numbers of cases of meningococcal infections generally are elevated in
the winter months immediately preceding this baseline, Los Angeles was the only location to
report a substantial increase during March of 1987 (personal communication, Dr. Clair Broome,
1988). Thus this increase is apparently due to a statistical fluctuation (that is, a false positive).
Method 3, the scan statistic, was consistent with Method 1 for mumps and hepatitis A and did
not detect any false positives (Figure 5). The results of all three methods are summarized in
Figure 5.

CONCLUSIONS
To summarize, Method 1 is based on normal theory and is an accepted methodology, conforming
to the type of quality control monitoring in reporting of laboratory results. In this example,
Method 1 detected two increases in disease occurrence deemed to be epidemiologic increases and
one decrease without epidemiologic confirmation, although the latter aberration was of small
magnitude. The problem here is one of robustness. To minimize the effect of past epidemics on the
ability of the system to detect current aberrations, one can replace a mean value by a median.
However, this shift in the estimator invalidates the normal theory estimates. To monitor disease
incidence based on the median of past values, we used Method 2, the bootstrap, because the
bootstrap is non-parametric and should be robust to different underlying distributions in disease
occurrence. This robustness is important in a method used for multiple diseases and multiple time
periods with resulting differences in distributions. A characteristic of this method is that it will not
detect a trend in a disease, but only an aberration that is different from the trend. An additional
advantage of the bootstrap is that confidence bounds can be developed once for application over a
year. Method 3, the scan statistic, gave no false aberrations among the diseases investigated here.
 DETECTION OF ABERRATIONS 329

Notifiable Disease Method Documented

FOidemic
1 2 3 Activity
Aseptic Meningitis
Hepatitis B
Hepatitis A * * * *
Hepatitis, Unspec.
Malaria
Measles
Meningococcal Infection *
Mumps * * * *
Pertussis
Rubella
Gonorrhea A
A : below lower limit *: above upper limit

Figure 5. Summary of methods, detection of aberrations in notifiable disease data

However, this method presumes no trend in the baseline data, and violation in this assumption
results in decrease in power of the method. Another drawback to the routine use of this method is
the lack of tabulated values of the statistic and the computational difficulty of obtaining exact
probabilities.
This investigation applied three statistical methods to surveillance data for cases of notifiable
diseases. The results from a bootstrap estimate are consistent with a common parametric
approach. Further applications of the methods are needed for different time periods to determine
whether these conclusions are specific to this investigation. Additibnally, a simulation study is
planned to check the sensitivity of the method to aberrations in a variety of distributional forms.
For diseases with small numbers of cases reported, the ratio of two Poisson variables will be
inve~tigated.~ The result of these investigations will be an easily implemented method that can be
used as part of an ongoing monitoring programme, the importance of which is to allow easy
detection of aberrations requiring further investigation in public health surveillance.

Note in pro05 In order to determine when a ratio falls outside a central confidence interval, the authors are
exploring a bootstrap approach to estimating confidence intervals on the appropriate upper and lower
percentiles, rather than a confidence interval on a median ratio.

REFERENCES
1. Mantel, N. ‘The detection of disease clustering and a generalized regression approach’, Cancer Research,
27, 209-220 (1967).
2. Ederer, F., Myers, M. H. and Mantel, N. ‘A statistical problem in space and time: do leukemia cases come
in clusters?, Biometrics, 20, 626-638 (1964).
3. Choi, K. and Thacker, S. B. ‘An evaluation of influenza mortality surveillance, 1962-1979’, American
Journal of Epidemiology, 113, 227-235 (198 1).
4. Efron, B. ‘The 1977 Reitz lecture: bootstrap methods: another look at the jackknife’, Annals ofStatistics, 7 ,
1-26 (1979).
5. Naus, J. L. ‘The distribution of the size of the maximum cluster of points on a line’, Journal ofthe American
Statistical Association, 60,532-538 (1965).
6. Wallenstein, S. ‘A test for detection of clustering over time’, American Journal of Epidemiology, 111,
367-372 (1980).
7. Johnson, N. L. and Kotz, S. Discrete Distributions, Houghton Mifflin 1969.