Research Article

(wileyonlinelibrary.com) DOI: 10.1002/qre.1919 Published online in Wiley Online Library

Detecting Periods of Significant Increased

Communication Levels for Subgroups of
Targeted Individuals
Ross Sparks*†
This paper investigates detecting significant increases in communication patterns and levels between small groups of individ-
uals within a moderate-size targeted group. Potential applications range from trying to establish emerging thought leaders
within an organisation to the detection of the planning stages of a crime. The scan statistic is a popular choice for monitoring
and detecting spatio-temporal outbreaks, but it is difficult to apply to large-scale target groups because of the computational
effort required. When monitoring communication levels between thousands of people, the number of combinations of peo-
ple whose communication may have increased is very high, and to scan through all of these to find which combinations have
increased communications significantly is an enormous task. A successful surveillance plan will have early communication
outbreak detection properties and good diagnostic capabilities for identifying individuals contributing to this outbreak. This
paper proposes a new computationally feasible approach for detecting communication outbreaks based on exponentially
weighted moving average smoothed communication counts between individuals within the network. We apply a cumulative
sum of ordered signal-to-noise (SN) ratios for communication counts to flag significant departures from their respective me-
dian values. This plan is demonstrated to be efficient at detecting changes in communication levels for a small part of the
network and diagnosing who is involved in the outbreak. Copyright © 2015 John Wiley & Sons, Ltd.

Keywords: cumulative sum; communication matrix; exponentially weighted moving averages; outbreak detection; poisson counts

1. Introduction
The importance of detecting an increase of communication within subgroups is clearly important in terms of security for preventing
terrorism events and identifying the planning stage of a crime. Such applications generally involve monitoring a matrix of daily commu-
nications between targeted individuals in terms of who contacts whom. However, its importance is not limited to security applications.
The following examples indicate the importance of applications involving data structure that have similarities with communication
matrices:
1. Monitoring market risk: The number of complaints by customers from 100 different locations around the world relating to an
international company’s 100 different products supplied by the company.
2. Marketing success: Following an advertising campaign, was there an increase in the number of visits for a subset of (potential)
customers to a range of product information websites compared with the period prior to the campaign?
3. Worldwide syndromic surveillance: The numbers of twitters (ignoring retweets) that mention senders have specific symptoms
(e.g. headaches, rashes, coughs, colds, flu symptoms, diarrhea, vomiting, nausea) per day from a hundred countries around the
world.
4. Worldwide monitoring invasive fauna or flora: monitoring a range of different invasive species monthly counts at 100 locations
within a country or across 100 countries.
5. World-wide travel monitoring: monitoring the number of people travelling from one country to the next per day.
6. Change management: When does the communication levels between people employed in a company increase for sub-groups
of employees (counting the number of meetings, phone calls and e-mails between pairs of employees called by one individual -
rows who called the meeting or made the connection and columns who attended the meeting to receive the call/message per
week)? This could assist a company after initiating a change in their business or business model to identify early the emerging
thought leaders within the company.

CSIRO Data61: PO Box 52, North Ryde NSW 1670, Australia

*Correspondence to: Ross Sparks, CSIRO Data61 : PO Box 52, North Ryde NSW 1670, Australia.
†
E-mail: ross.sparks@csiro.au

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

The focus on 100 is an arbitrary choice at this stage, but is used to make a start with this research. The aim is to later make the
methodology more generally applicable to variations in the number of targeted people or other entities.
Investigators in a variety of fields have indicated interest in detecting changes in the communications patterns and levels within
a targeted group of people. For example, research to date has focused on detecting changes in political networks,1 identifying com-
promised financial accounts,2 identifying potential price-changing activity associated with specific stocks,3 early recognizing of an
emerging crisis,4 characterising an earthquake event in near real-time5 and identifying terrorism networks.6, 7 In pursuit of an effec-
tive, computationally feasible and widely applicable methodology, this paper focuses on detecting when a subgroup of targeted
people increases their communication levels significantly. The technology outlined in this paper focuses on security matters or crime
including the following:
1. When is there an increase in the communication levels between a group of known/convicted criminals potentially indicating the
planning of a new crime? Maxson and Klein8 indicated that thieves with greater robbery experience were more likely to spend
longer planning their next robbery.
2. When is there an increase in the number of messages or phone calls between a subgroup of fund managers and company
directors in an industry sector that is reported as under financial stress in the media? Was there significantly larger number of
sales or purchases of shares from company associates/employees prior to a company announcement one could monitor share
purchases in the top 100 companies in a country for a target group of company directors and employees of these companies.
3. When are groups of (potential) criminals searching through many more Facebook pages than usual from particular postal codes,
for example, searching for residential addresses with occupants on vacation?
The scan statistic has been used in space–time monitoring of crime,9–12 but no work has been carried out with monitoring com-
munication counts for a targeted group of people when only a small subgroup of these showing elevated counts. The scan statistics
could be used for small targeted groups of people rather than the 100 or 1000 used in this paper, for example, if we are monitoring 20
peoples’ communication counts and we are looking for an unknown group of four having elevated counts then there are
 
20
D 4845
4

possibilities, but if we are monitoring 100 people communication counts and we are looking for an unknown group of four having
elevated counts, then there are
 
100
D 3921225
4

possibilities. In practice, we do not know the number of people involved in the communication outbreak, which makes the scan statistic
not feasible for a targeted group of 100 people.
This paper explores a cumulative sum control chart (CUSUM) approach based on the order statistics of the exponentially weighted
moving averages (EWMA) temporally smoothed communication counts. Traditionally, univariate analyses of departures of these EWMA
smoothed counts from expected counts have been explored in the literature.13–15 However, in this paper, the matrix of communication
counts is temporally smoothed using a multivariate EWMA. This involves smoothing say several thousand different communication
counts over time. After smoothing the matrix of counts, the CUSUM statistic is applied to find the subgroups of the target people that
has significantly elevated counts from ‘what is expected’. This CUSUM is not carried out in the usual way, that is, looking for temporal
change points. It is used to assess whether the communication counts have increased significantly for subgroups of targeted people.
In other words, this CUSUM looks for communication level change points rather than finding temporal change points. The high-end
EWMA temporal smoothed counts are compared with their median order statistics values using the cumulative sum statistic to iden-
tify communication outbreaks for sub-groups of the target people. The intent is to deliver a plan that is robust at detecting multiple
disconnected subgroups of the network with significantly increased communication levels. The methodology is applied to a commu-
nication count matrix with expected counts that can be space–time-dependent but whose in-control behaviour is assumed to change
predictably with time.
To assess the performance properties of plans, we use the average time to signal (ATS) in days as a criteria for assessing early de-
tection. Plans are explored for an in-control ATS of 100 (that is an in-control ATS of 100 days when monitoring daily counts), and their
respective out-of-control ATS properties are compared. All ATS values in this paper are calculated using 10 000 simulations.
Section 2 of this paper covers the multivariate EWMA temporal smoothing of communication counts. This is designed to smooth out
noise and reveal trends in communication counts. Section 3 introduces the CUSUM for continuous data16 and the CUSUM for Poisson
counts. Section 4 outlines the recommended plan that cumulatively sums order statistic departures from their respective median value.
The paper deals firstly (Section 4.1) with the case where communication counts are homogeneous. In this case, outbreak information
is more likely to be contained in the high-end ordered smoothed counts. Not all order statistics are examined; only the g largest order
statistics are included in the CUSUM. The paper considers departures of order statistics from their median values rather than their
mean count, and therefore, it differs from the work in Sparks and Patrick.17 When more of the communication counts are unusually
higher than expected, then the higher-ordered smoothed counts are generally larger than their respective median values. Therefore,
the CUSUM of these is likely to trend away from zero on the high side.
An alternative CUSUM suggested by Sparks18 for an ordered set of people performs worse than the CUSUM plan in this paper, al-
though the results are not reported here. Section 4.1 also investigates a simulation study designed to compare this CUSUM plan in the

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
R. SPARKS

homogeneous mean case. Section 4.2 describes the CUSUM for order statistics approach for application with non-homogeneous com-
munication counts. Section 5 looks at designing a computationally efficient scan statistic. Section 6 presents an example of application.
Section 7 concludes by summarising the findings and suggesting areas of future research. The paper aims to address the situation
where the in-control mean is moving with time (t ), rather than stationary with time t D , but starts with t D  for all t. In most
communication surveillance settings, it is usual for the mean counts to vary with time (i.e. t ¤ ) even when in-control.

2. Temporal exponentially weighted moving average smoothing of communication counts

The EWMA is a very popular univariate monitoring tool. There are several papers on monitoring Poisson counts using EWMA.13–15, 19, 20
Let the communication matrix21 be defined by yi,j,t for the total number of times the ith person calls the j person on day t and its
respective mean communication count be i,j,t . The EWMA in this paper is used to both temporally smooth the communication counts
and accumulate memory of past counts as follows:

yN i,j,t D ˛yi,j,t C .1  ˛/Nyi,j,t1

where yN i,j 0 D N i,j 1 and 0 < ˛ < 1. The expected value of these smoothed counts is given by

N i,j,t D ˛i,j,t C .1  ˛/N i,j,t1

where N i,j 0 D N i,j 1 . Smoothing of the counts serves to reduces the ‘noise’ and reveals trends in the counts. Moreover, it also helps
in retaining some memory of past communication counts. The value ˛ is a constant that determines how much temporal memory to
retain in the smoothing process. Larger values of ˛ retain less memory and therefore smooth less. To avoid worst-case scenarios, we
examine the reflective boundary EWMA (e.g. Gan, 1993) defined by

 
yNN i,j,t D max ˛yi,j,t C .1  ˛/Nyi,j,t1 , N i,j,t

The worst-case situation in the non-reflective boundary EWMA would occur if the counts wondered lower prior before the outbreak
occurred, which would delay its detection, because it would take longer to reach the threshold for outbreaks. Later, we use an adap-
tion of the traditional CUSUM statistic to define the level of communication matrix aggregation we need to detect communication
outbreaks, so we now introduce the CUSUM in its traditional form.

3. The cumulative sum statistic

Let yt be a general control variable, measured at time t, that we wish to monitor for changes in location. Let t be the expected value of yt
when in-control, and its variance is assumed to be 1 without loss of generality. Assume that we are only interested in flagging significant
changes on the high side, then the extension of the CUSUM statistic advocated by Page16 for monitoring time vary ‘in-control’ means
for high-side changes in the control variable is

St D max.0, St1 C yt  t  kt /

where kt is the reference value determining the amount of temporal memory to retain in the CUSUM statistic and S0 D 0. Larger
values of kt retain less temporal memory, because then St more readily forgets past observations by resetting St to zero whenever
St1 C yt is not larger than t C kt . In the zero state case and normally distributed data (i.e. S0 D 0 and outbreaks are started at t D 1),
Moustakides22 proved that kt D t =2 is optimal when t D  for all t. The steady-state optimal choice is more complicated as can be
seen in Sparks23 and Jiang et al.24 for the adaptive version of the CUSUM.
Lucas,25 Soresson,26 Han et al.,27 Sparks et al.15 and Schuh et al.28 showed that
the zero state optimal Poisson counts CUSUM is of the form15, 25–28

Spoisson,t D max.0, Spoisson,t1 C yt  kpoisson,t /

where Spoisson,0 D 0 and the reference value kpoisson,t for a fixed proportional increase in mean counts defined by ı > 0 was taken as

kpoisson,t D .t .1 C ı/  t /=.log.t .1 C ı//  log.t // D t ı=log.1 C ı/.

For a specific ı , this reference value is only optimal for the zero-state situation; it is not optimal for the steady=state situation (see Sparks
et al.15, 19 for the adaptive version of this CUSUM).

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

4. Cumulative sum plans for monitoring communication networks

Counting events in cells within a lattice (for example, Glaz et al.29 ) looks very similar to communication matrix of counts between
individuals.21 However, the spatial lattice dimensions have a natural ordering defined by spatial neighbours, whereas in the commu-
nication matrix, the people may not be ordered. Therefore, the scan statistic is not easy to apply in the communication matrix setting.
Searching all combinations of m sub-networks from n.> m/ people in the target network can involve excessive computational effort
(e.g., for the target group of 100 people, scanning all combinations of four different people from the 100 targeted involves nearly four
million combinations of unique groups of four people).
Now, we wish to lead up to developing a CUSUM plan for communication settings where the communication counts means are not
constant over time or across people, that is, the mean communication counts are equal to i,j,t . However, we first start with the more
unrealistic but easier situation where communication count means are constant across different communication counts and across
time, that is, i,j,t D  for all t, i and j. Later, we use the results from homogeneous communication networks to extend the technology
to apply to the more realistic non-homogeneous communication networks.

4.1. Monitoring homogeneous communication networks

In this section, we assume that the counts yi,j,t are independent Poisson counts with mean i,j,t D  for all i, j and t. Rather than compar-
ing these noisy counts with their expected values, we first temporally smooth these counts to reveal trends above expected. Now, we
need a measure of how different these smoothed counts are from their respective expected values. The traditional measure of depar-
tures (standardisation) of by subtracting the mean from Poisson counts and dividing the standard deviation can result in instabilities
for smoothed counts with small mean values (i.e. usually for poor communicators). Although this is not an issue for homogeneous com-
munication networks, it is for non-homogeneous networks, which we are aiming for in our methodology development. Instead, the
CUSUM is used to compare the high-end ordered smoothed communication counts with their respective median values. For homoge-
neous counts, the information on outbreak communications will be in the high-end order statistics for small communication outbreak
clusters and in all order statistics for communication outbreaks involving all target people in a network. Now, we outline how these
median values were estimated.
For a given time t D 1, 2, : : : , 25 simulate the communication matrix yi,j,t assuming E.yi,j,t / D  for all i, j, t for a burn-in period of 25
consecutive days to achieve a steady state. Calculate yNN i,j,25 for all i and j. Let the order statistic of yNN i,j,25 for i D 1, : : : , n, j D 1, : : : , n be
2
ytŒ1  ytŒ2  : : : ,  ytŒn  . Repeat this process for another 4000 times, and the results are used to estimate the median value for ytŒk .
Denote this median value by mŒk ./. Carry this out for all k D 1, 2, : : : , n2 . When in-control, we expect ytŒk to be either greater than or
less than mŒk ./ with probability 0.5 (that is, equally likely). The median is selected rather than the mean because it is more robust. The
R routine for establishing median values is reported in Appendix A.
The CUSUM of departures in ytŒ` from mŒ` ./ with some forgetting process may be a useful statistic for monitoring communication
outbreaks involving a small subset of the n targeted people. For example, in the spirit of Page’s (1954) CUSUM, we examine the statistic
 
SS`,ks ,t D max 0, SS`1,ks ,t C ytŒ`  mŒ` ./  ks

where ks .> 0/ is some reference value that determines the level of forgetting in the CUSUM, SS0 D 0, and ` D 1, 2, : : : , n2 . Note that
this differs from the usual CUSUM that accumulates temporal memory of significant departures from expected values. This CUSUM
looks for significant departures in the ordered communication counts from their respective in-control median values. Note that if n is
large and the communication outbreak involves m.<< n/ people, then SS`,ks ,t are zero or close to zero for most low ` values (particularly
for larger ks values). For the homogeneous spatial mean case, the information on the outbreaks with a large enough step change is
generally housed in the larger communication counts (that is, the high-end order statistics). Therefore it may be computationally more
efficient and sometimes sufficient to examine the CUSUM for only the g highest-order statistics, that is, examine
 
SS`,ks ,g,t D max 0, SS`1,ks ,g,t C ytŒ`  mŒ` ./  ks

SSn2 g,ks ,g,t D 0, and ` D n2  g C 1, n2  g C 2, : : : , n2 . If m is known and the increase in communication counts for the outbreak
group of m is high enough, then selecting g D m2 would seem to be appropriate; however, m is never known in advance, and sensible
choices for g are needed.
To keep things simple, the same reference value ks is used for all order statistics. The fact that the variance of ytŒ`  mŒ` ./ increases
as ` approaches closer to 10 000. This strategy has computational advantages, and it simplifies the plan by having a higher chance of
resetting the CUSUM to zero for smaller `, which helps for large step changes in a small subset of the communication counts in the
matrix. A spatio-temporal outbreak is flagged when

SS`,ks ,g,t > hSS .ks , g, /

for all ` D n2  g C 1, n2  g C 2, : : : , n2 , where hSS .ks , g, / is a threshold whose value is chosen to deliver an in-control ATS that is
acceptably high. In other words, the value of hSS .ks , g, / controls the false discovery rate.
Table B1 in Appendix B provides models for threshold values hSS .ks , g, / for the 100 by 100 communication matrix structure (i.e.
n D 100) when all communications in the matrix have their means equal to a fixed value. The dimension constraints in Table A1 are

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
R. SPARKS

˛ D 0.075:
1. ks D 0.005 and g D 200, 300, 400 or 500
2. ks D 0.006 for g D 400 or 500
3. ks D 0.007 for g D 500.

The hSS .ks , g, / values are available in Table A1 for a range of homogeneous communication count means spanning from 0.002 (i.e.
expected total of 20 communications per day) to 4 (i.e. expected total of 40 000 communications per day), and the designed in-
control ATS is taken as 100 days. The threshold values derived from Table A1 allow users to design plans for the rare cases where
the communication count means are constant over the entire communication matrix. These plans, for homogeneous communica-
tion counts, are used later to design plans for the 100 by 100 communication matrix structure with non-homogeneous expected
communication counts.
In this section, we explored simulated examples of counts on a 100 by 100 communication matrix with all communication count
means equal to a constant for all i, j and t. Outbreaks are generated by simulating communication counts with mean equal to i,j,t .1Cı/,
where ı > 0 for some i and j combinations and in-control counts are generated when ı D 0 for all other i and j combinations.
For fixed  and ˛ , this CUSUM statistic has two parameters that help in its efficient design. One is the magnitude of the reference
values ks and the other is the number of high-end order statistics g to include in the CUSUM plan.

4.1.1. Selecting ks and g for known homogeneous communication count means . This section examines the ks and g options that deliver
the early detection properties for one specific outbreak. A full factorial experiment with ˛ D 0.05, 0.075 or 0.1; ks D 0.003, 0.005, 0.007
and 0.009; and g D 200, 300 and 500 is carried out to decide on the ‘best’ choices for ks and g for communication outbreaks of a gang
of 11 people with homogeneous communication mean of  ranging between 0.1 and 0.8. In the following, we offer a summary of the
findings that resulted from this simulation study. No single (˛, ks , g) value is optimal for all communication outbreaks. The general rules
for better choices are as follows:
 For larger step changes in communication count, it was better to select larger values for ks (i.e. ks D 0.009=0.007 rather than taking
ks D 0.003). However, for in control means as low as 0.01, then ks D 0.003 was better for all step changes given ˛ D 0.05, 0.075 or
0.1 and g D 200, 300 or 500.
 Selecting ˛ D 0.1 was better than selecting ˛ D 0.05 or 0.075 for large step changes in communication mean counts; selecting
˛ D 0.05 was better than selecting ˛ D 0.075 or 0.1 for small step changes.
 Generally, when communication outbreaks covered a larger number of people, it was better to select larger values of g. The choice
of g depends on the number of communications that is involved in the outbreak and the magnitude of the step change.
 The various options considered for g and ks in this factorial experiment had similar ATS performance when the step changes were
large. In fact, the performance of the approach is reasonably robust to the selection of ks with their ATS values differing by no
more than 3% for large changes in location.
 Increasing the value of g has a similar influence to lowering the value of ks , and so there is not necessarily a uniquely best solution
for each particular outbreak. Larger values of g suit small communication level increases for a larger group of people, but so does
smaller values of ks . When the optimal solution is not unique, then we always give preference to plans with lower g values because
this involves less computational effort.
From now on, ˛ D 0.075 is selected without too much empirical evidence to support this choice other than it is the middle of the
two others tried, namely ˛ D 0.05 and 0.1.

4.1.2. Detecting communication outbreaks for small target networks with 100 people. The plan is designed for a fixed number (100) of
target individuals. When monitoring communications between gang members or employees of companies, targeting communica-
tion levels for 100 of its members is quite feasible. For example, bikie gangs in Australia involve about 200 people per chapter, with
members having both legal and illegal commercial business enterprises. Not all members would be under surveillance by the po-
lice. Increased communication levels between suspicious individuals could indicate the planning stage of an illegal activity. Detecting
such increased activity could give security organisations an advantage. Similarly, potential gangs of terrorists within a state of Australia
could involve only 100 individuals. The same would be true if we were monitoring the communications levels between workers of a
medium-size company with a few hundred employees. In such cases, it would be easy to target the top 100 employees, monitoring
their within-company communication levels for the purpose of either understanding why they change significantly or assessing the
consequences of organisational change has on social interactions within the company. This section will focus therefore on developing
plans for monitoring a target group of 100 individuals. The communication matrix is defined as outlined before, and the methodology
discussed is applied.
A plan with reasonably robust performance across all outbreaks considered in the experiment (i.e. those in Tables I and II to follow)
is the plan with g D 200 or 300 and ks D 0.005 for a communication outbreak involving 11 people. This looks at targeted people with
homogeneous communication counts ranging from 0.2 calls per day to 2.5 calls per day. The plan (g D 200 or 300) with the smallest
respective out-of-control ATS has their respective ATS values in bold text in each table. Tables I and II demonstrate that there is little
difference between the plan with g D 200 and the one with g D 300. Generally, the plan with g D 300 is slightly better for smaller shifts
because the outbreak communication counts are less likely to be in the higher-end ordered counts than larger shifts, and therefore, a
larger value of g works better. However, for large shifts involving a small group of people, all its information is housed in the largest 200
ordered counts, and therefore, g D 200 is generally slightly better.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
Copyright © 2015 John Wiley & Sons, Ltd.
Table I. Average time to signal performance for a range of k values. Comparing ATS performance of the plans with g D 200 & 300 when
ks D 0.005, the in-control ATS=100 and the communication outbreak spans a group of 11 people. For  D 0.2, 0.3, 0.4, we have
.a1 , a2 , : : : , a8 / D .0.5, 1, 2, : : : , 7/ and  D 0.5, 0.7, we have .a1 , a2 , a3 , a4 , : : : , a8 / D .0.25, 0.5, 1, 2, : : : , 6/
Communication outbreaks of six people by six people in a 100 by 100 target group

 0.2 0.7

k 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

ı ATS

0.5 70.60 75.33 76.30 74.83 74.11 72.61 53.71 43.36 55.29 57.67 58.23 55.59 47.71 33.97 23.70 17.41
1.0 50.25 56.20 57.65 48.34 45.86 34.02 20.27 16.08 27.69 33.63 37.48 34.24 24.05 14.30 9.19 6.59
2.0 28.15 33.89 34.26 28.32 19.64 12.62 8.05 6.16 15.57 18.32 19.46 16.79 11.60 6.60 4.29 3.09
3.0 19.12 23.48 23.51 19.35 12.67 7.87 5.20 3.95 10.32 12.23 12.96 11.01 7.77 4.50 2.89 2.12
4.0 14.70 15.10 17.46 14.37 9.91 5.78 3.74 2.94 8.23 9.57 10.18 8.22 5.69 3.52 2.24 1.80
5.0 12.06 14.30 13.88 11.30 7.56 4.68 3.13 2.39 6.70 7.89 8.17 6.60 4.69 2.91 1.88 1.41
6.0 10.31 12.15 12.14 9.35 6.29 3.96 2.60 2.11 5.72 6.68 7.03 5.68 3.89 2.46 1.69 1.07
7.0 8.97 10.55 10.33 7.77 5.43 3.41 2.28 1.89 5.22 5.84 5.96 4.83 3.37 2.13 1.46 1.00
8.0 8.05 9.41 9.30 7.25 4.81 3.01 2.01 1.76 4.40 5.26 5.40 4.34 3.02 1.94 1.21 1.00
ATS, average time to signal
R. SPARKS

Qual. Reliab. Engng. Int. 2015

R. SPARKS

Table II. Comparing ATS performance of the plans with g D 200 and 300 when ks D 0.005, the
in-control ATS=100 and the communication outbreak spans a group of 11 people
Communication outbreaks of 11 people by 11 people in a 100 by 100 target group
CUSUM order statistic with kc D 0.005

hSS ../ 3.721 4.613 4.499 5.623 5.00 6.302 5.40 6.90 6.03 7.84

 1.0 1.4 1.7 2.0 2.5

g 200 300 200 300 200 300 200 300 200 300

ı ATS

0.2 33.7 30.85 25.54 24.75 22.61 22.55 19.55 19.50 16.42 16.35
0.4 11.99 11.89 9.56 9.50 8.65 8.64 7.77 7.71 6.84 6.75
0.6 7.27 7.26 5.97 5.92 5.41 5.39 4.99 5.00 4.45 4.41
0.8 5.30 5.30 4.39 4.39 4.06 4.02 3.70 3.72 3.35 3.34
1 4.15 4.17 3.52 3.52 3.24 3.27 3.00 3.02 2.72 2.72
2 2.16 2.17 1.94 1.94 1.88 1.89 1.82 1.83 1.69 1.70
3 1.71 1.72 1.36 1.41 1.17 1.18 1.06 1.06 1.00 1.00
4 1.14 1.16 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00
ATS, average time to signal.

If the whole target network increases its communication level, then generally, this is detected earlier if we monitor aggregated
communications over the whole target group. In such cases, applying the plan with g D 300 would not be efficient because all or-
der statistics are likely to increase and only considering the high-end order statistics results in a loss of information. Therefore, the
appropriate EWMA plan for monitoring global increases in the daily communication counts is

X
100 X
100
TEWMAt D max.0.075  yi,j,t C 0.925TEWMAt1 , E.TEWMt //
iD1 jD1

P P100 P100 P100

where t indicates the day and E.TEWMAt / D 100 iD1 jD1 E.yi,j,t / D iD1 jD1 i,j,t D 10 000. TEWMAt statistic is substantially better
than using the CUSUM order statistic approach at detecting global communication outbreaks (Table III). For example, for  D 0.7,
the TEWMAt statistics flags a step change in the mean count of ı D 0.013 for all communications in 5.06 days on average, while the
CUSUM order statistic approach with g D 300 takes on average 17.49 days. Table III demonstrates that the CUSUM order statistic
with g D 300 approach should not be used for early detection of global communication outbreaks, and therefore, its usefulness is
evaluated for small subgroup communication outbreaks. How small the subgroup should be before the CUSUM order statistic with
g D 300 has early detection advantages over using TEWMAt is illustrated in Table IV and V for  D 0.2 to 0.7. Table IV demonstrates
that if the outbreak communication group is 11 of the target 100 people, then the CUSUM order statistic with g D 300 has small early
detection advantages over using TEWMAt . If the outbreak communication group involves six or less people, then its gains start being
substantial (Table V). Hence, if the communication outbreaks involves 11 or fewer people, then the extra computational effort in using
the CUSUM order statistic with g D 300 is warranted. In other words, if the changes are in a smaller subgroup of the target people, then
efficiencies can be achieved by the accumulation of the appropriate subgroup memory of these changes rather than using TEWMAt .
These cutoff relative efficiency judgments are based on one communication outbreak involving all 11 people, but two simultaneous
communication outbreaks involving mutually exclusive subsets of eight people would be roughly equivalent because 112  82 C 82 .
Similarly, three separate simultaneous communication outbreaks: one involving all seven people and the other two involving all six
each are equivalent, because 112 D 72 C 62 C 62 . So this judgment is not based on the number of people involved in the outbreak, but
on the number of communications in the matrix involved with the outbreak.

4.2. Design cumulative sum order statistics plans for non-homogeneous communication count means
Finding a 100-person target group worth monitoring that usually has all communication count means across all days equal is extremely
rare. To be practically useful, the methodology needs to be robust to variations in communication count means across the communi-
cation matrix. In this paper, we will always use a 100-person target, but allow the mean for communication counts to vary across the
communication matrix. Let yNN i,j,t be the spatio-temporal smoothed counts with the reflective boundary and its non-reflective version
corresponding mean is N i,j,t . In this general case, we find the rank position that the smoothed counts hold within its respectively ‘me-
dian upper tailed order statistics’ for each communication count. The ranking of the non-homogeneous smoothed communication
counts is found using the following steps:
1. Find the median high-end (9701 to 10 000) order statistics for yNN i,j,t with related expected communication counts all
equal to N i,j,t D 0.002, 0.003, : : : , 0.040, 0.042, 0.044, 0.046, : : : , 0.440, 0.4425, 0.445, 0.4475, : : : , 0.73, 0.735,0.74, 0.745, : : : , 1.3,
13.1, 1.32, 1.33, : : : , 4.2 and denote them by mŒ` .N i,j,t /, ` D 1, 2, : : : , 300.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
Copyright © 2015 John Wiley & Sons, Ltd.
Table III. Comparing ATS performance of two plans when the in-control ATS=100 and the communication outbreak involves the whole target group.
Communication outbreaks involving the whole target group

CUSUM order statistic with ˛ D 0.075, kc D 0.005 & g D 300, 1000, 2000 (denoted COS)
EWMA control chart - total communication count for the target(denoted TEWMA)
network with ˛ D 0.075 (denoted TEWMA)

hSS ../ 3.721 1.547 1.89 1.95 4.50 2.08 2.84 3.02 5.00 2.53 3.73 4.23 5.40 2.97 4.78 5.54 6.03 3.70 6.33 7.99

 0.2 0.3 0.4 0.5 0.7

TE- COS TE- COS TE- COS TE- COS TE- COS
g WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000

ı ATS

0.013 10.49 31.14 22.99 21.96 8.52 23.86 18.31 15.97 6.93 22.48 14.88 13.65 6.12 20.48 13.98 12.20 5.06 17.49 11.97 9.93
0.025 4.75 15.10 11.21 10.49 3.96 12.55 8.79 7.99 3.36 11.72 7.70 6.97 3.00 10.36 7.26 6.03 2.50 9.23 6.09 5.27
0.100 1.37 3.36 2.77 2.60 1.17 3.22 2.40 2.18 1.05 2.89 2.16 1.95 1.02 2.74 2.07 1.83 1.01 2.47 1.86 1.66
0.150 1.03 2.54 1.97 1.87 1.00 2.29 1.75 1.58 1.00 2.05 1.59 1.48 1.00 1.98 1.55 1.31 1.00 1.81 1.36 1.15
0.200 1.00 1.97 1.59 1.51 1.00 1.79 1.42 1.25 1.00 1.69 1.26 1.13 1.00 1.61 1.18 1.05 1.00 1.47 1.08 1.00
0.250 1.00 1.67 1.33 1.23 1.00 1.52 1.15 1.06 1.00 1.44 1.07 1.01 1.00 1.30 1.03 1.00 1.00 1.20 1.01 1.00
ATS, average time to signal; EWMA, exponentially weighted moving average.
R. SPARKS

Qual. Reliab. Engng. Int. 2015

R. SPARKS

Table IV. Comparing ATS performance of two plans when the in-control ATS=100 and the
communication outbreak involves 11 people from a gang. One plan monitors the total commu-
nication counts and the others uses CUSUM order statistic
Communication outbreaks involving a sub-network of 11 people

CUSUM order statistic with ˛ D 0.075, kc D 0.005 & g D 300 (denoted COS)
EWMA control chart - total communication count for the target
network with ˛ D 0.075 (denoted TEWMA)

hSS ../ 3.721 4.613 4.499 5.623 5.00 6.302 5.40 6.90 6.03 7.84

 0.2 0.3 0.4 0.5 0.7

TEWMA COS TEWMA COS TEWMA COS TEWMA COS TEWMA COS

ı ATS

a1 26.06 26.35 20.68 19.81 17.94 15.89 33.80 36.52 26.74 30.87
a2 10.77 9.95 9.42 7.95 7.55 6.63 15.86 13.84 12.35 11.15
a3 4.87 4.46 4.03 3.70 3.38 3.23 6.76 5.88 5.44 4.99
a4 3.11 2.94 2.70 2.51 2.32 2.23 3.08 2.91 2.60 2.54
a5 2.47 2.30 2.04 1.98 1.84 1.83 2.15 2.03 1.86 1.88
a6 1.93 1.92 1.76 1.74 1.54 1.69 1.74 1.75 1.48 1.48
a7 1.74 1.73 1.51 1.45 1.38 1.19 1.43 1.33 1.20 1.07
a8 1.54 1.29 1.35 1.18 1.17 1.02 1.23 1.05 1.08 1.00
ATS, average time to signal; CUSUM, cumulative sum; EWMA, exponentially weighted moving
average.

2. Using linear interpolation, find the estimated median values that correspond to the expected value of the non-reflective bound-
ary of yNN i,j,t . That is, find N i,j,t rank position amongst  D 0.002, 0.003, : : : , 4.2 and then interpolate mŒ` .N i,j,t / values from the
respective median values either side of this position.
3. Find the rank position of yNN i,j,t in amongst the median values mŒ` .N i,j,t /, ` D 10000  g C 1, : : : , 9999, 10 000. Denote these by Ri,j
and note that 1  Ri,j  300. Repeat for all i, j D 1, 2, : : : , 100.
4. Now, rank the Ri,j , i, j D 1, 2, : : : , n from smallest to largest, that is, find RŒ1  RŒ2      RŒ10 000 .
5. Find the smoothed counts yNN ` that correspond to RŒ` for ` D 1000  g C 1, 1000  g C 2, : : : , 10 000, and the corresponding
mean count N Œ` t and medians mŒ` .N Œ` t /.

Now, a similar adaptive approach as in Sparks23 and Shu and Jiang30 is applied. This cumulatively sums the division of the ‘smoothed
ordered count minus their median minus the reference value’ by the threshold hSS .ks , g, Œ`
t /, where the corresponding mean ordered
count is Œ` 15 18
t . A similar approach to this is also followed by Sparks et al. and Sparks. Therefore, the adaptive CUSUM statistic for the
non-homogeneous mean case is
      
ASS`,ks ,g,t D max 0, ASS`1,ks ,g,t C yNN tŒ`  m
Q Œ`
t N Œ`
t  ks =hSS ks , g, Œ`
t

where ` D n2  g C 1, : : : , n2  1, n2 and ASSm2 g,ks ,g,t D 0. This plan flags an outbreak when

ASS`,ks ,g,t > 1

for at least one ` D n2  g C 1, : : : , n2  1, n2 , delivering an in-control ATS of approximately 100.

This plan was tested for the several scenarios where the directional communication mean counts are assumed random uniform dis-
tributed on the interval between a and b with 0.01  a < b  4 (denoted U.a, b/) and the in-control ATS values. In 30 runs of different
choices of a and b gave ARL values with minimumD95.2, first quartileD97.6, medianD99.8, third quartileD101.2 and maximumD105.2.
Generally, the communication counts are not random uniformly distributed, but networking members are expected to have similar
communication count means. For example, if the reverse direction means were assumed to be correlated such that

i,j,t D i,j D j,i C .1  /ui,j

where ui,j is random U.0.01, 4/. Start with j,i U.0.01, 4/ for j > i and  D 0.5, 0.7 or 0.9, then by simulation, the plan was shown to give
the approximate false discovery rate of one in 100 days, that is, 96.6, 95.9 and 98.8 days, respectively. This approach works well as long
as i,j,t do not vary a great deal for increasing values of t for all i, j. If this is not true, then the adaptive adjustment needs to be made
before temporal smoothing.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 Table V. Comparing ATS performance of two plans when the in-control ATS=100 and the communication outbreak involves six people from a gang. One plan monitors the total
communication counts and the others uses CUSUM order statistic

Copyright © 2015 John Wiley & Sons, Ltd.

Communication outbreaks involving the whole target group

CUSUM order statistic with ˛ D 0.075, kc D 0.005 & g D 300, 1000, 2000 (denoted COS)
EWMA control chart - total communication count for the target(denoted TEWMA)
network with ˛ D 0.075 (denoted TEWMA)

hSS ../ 3.721 1.547 1.89 1.95 4.50 2.08 2.84 3.02 5.00 2.53 3.73 4.23 5.40 2.97 4.78 5.54 6.03 3.70 6.33 7.99

 0.2 0.3 0.4 0.5 0.7

TE- COS TE- COS TE- COS TE- COS TE- COS
g WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000 WMA 300 1000 2000

ı ATS

1 42.01 23.09 24.88 25.42 35.32 17.30 19.10 20.21 30.07 14.32 15.35 16.51 28.94 12.00 13.90 14.90 22.90 9.75 11.14 12.26
2 20.23 9.16 9.57 9.77 16.44 7.18 7.85 8.01 13.94 5.96 6.12 7.08 12.39 5.28 6.12 6.47 10.37 4.45 5.11 5.67
3 13.19 5.54 5.90 5.99 9.84 4.53 4.92 5.12 8.89 3.89 4.35 4.56 7.47 3.47 3.99 4.23 5.44 2.94 3.43 3.75
4 9.45 4.13 4.40 4.49 7.20 3.35 3.70 3.83 6.11 2.97 3.05 3.47 5.12 2.66 3.05 3.22 4.45 2.30 2.68 2.89
5 7.07 3.31 3.55 3.60 5.61 2.75 3.01 3.07 4.59 2.42 2.70 2.81 4.29 2.19 2.54 2.63 3.50 1.93 2.16 2.38
6 5.72 2.80 2.96 2.99 4.57 2.35 2.55 2.63 3.96 2.05 2.27 2.39 3.43 1.94 2.12 2.23 2.93 1.79 1.93 2.01
7 4.66 2.44 2.56 2.59 3.66 2.05 2.22 2.27 3.30 1.90 1.99 2.06 2.94 1.80 1.91 1.97 2.52 1.55 1.78 1.87
8 4.05 2.19 2.26 2.29 3.39 1.88 1.98 2.00 2.93 1.75 1.86 1.90 2.57 1.58 1.83 1.86 2.26 1.24 1.61 1.75
ATS, average time to signal; EWMA, exponentially weighted moving average.
R. SPARKS

Qual. Reliab. Engng. Int. 2015

R. SPARKS

Table VI. Comparing ATS performance for the EWMA scan plan Z1g when the in-control ATS=100 and
the communication outbreak involves 11 people from a gang (0.2    2.5)
Communication outbreaks involving a sub-network of 11 people
EWMA scan plan for statistic Z1g
hz1g 0.3895 0.3709 0.3680 0.3640 0.3565 0.3510 0.3455 0.3435 0.3420 0.3399
 0.2 0.3 0.4 0.5 0.7 1.0 1.4 1.7 2.0 2.5
ı ATS
a1 26.90 19.91 16.02 37.05 30.11 31.87 24.89 21.98 19.52 16.40
a2 10.01 7.86 6.65 14.80 11.10 11.86 9.51 8.54 7.74 6.82
a3 4.50 3.69 3.24 5.89 4.95 7.26 5.92 5.39 4.99 4.44
a4 3.03 2.56 2.23 2.92 2.55 5.30 4.39 4.02 3.72 3.35
a5 2.33 1.98 1.85 2.03 1.88 4.16 3.54 3.25 3.02 2.29
a6 1.94 1.76 1.55 1.75 1.49 2.18 1.96 1.89 1.83 1.70
a7 1.74 1.46 1.19 1.34 1.08 1.73 1.43 1.18 1.06 1.01
a8 1.54 1.18 1.03 1.05 1.00 1.15 1.05 1.00 1.00 1.00
ATS, average time to signal; EWMA, exponentially weighted moving average.

5. The scan statistic

The scan statistic is a popular monitoring plan for detecting spatio-temporal outbreaks that cluster geographically (e.g. Tango31 ), but
its application to detecting outbreaks in communications for a subgroup of target people is hampered by the excessive number of
subgroup combinations that need to be scanned to cover all options of increased communication counts (e.g. for 100 targeted people,
this involves scanning a total of 21 00  101 groups). To reduce this excessive computational burden, we use order statistics to decide
what communication counts to scan. Two options are considered. These are outlined next when in-control communication counts have
homogeneous means
1. Aggregate over the top g most the unusually high EWMA smoothed counts relating to the g highest ranked order statistics
RŒ1 < RŒ2 <    < RŒ10 000 in Section 4.2. That is, we find the smoothed counts yNN Œ` , ` D 10 000  g C 1, : : : , 9999, 10 000. A
standardised sum of departures in these counts from their respective median values mŒ` .N Œ` t / is calculated, that is, calculate

0 1 v
X
10 000 X
10 000 .u
u X
10 000
Z1g D @ yNN Œ`  mŒ` .N Œ`
t /
A t N Œ`
t .
`D10 000gC1 `D10 000gC1 `D10 000gC1

This standardisation is arbitrary because it does not divide by the square root of the sum of variances for yNN Œ`  mŒ` .N Œ` t /, but
this standardisation is selected to reduce the computational effort of the plan. Flag an outbreak when Z1g exceeds an appropriate
threshold (hz1g .Œ`t /) designed to deliver a specified false discovery rate of ATSD100 days. This is roughly equivalent to applying
the CUSUM plan with ks D 0, except this statistic does not reset to zero whenever the statistic goes below zero and therefore can
suffer in a worst-case situation. For example, if yNN Œ`  mŒ` .N Œ`
t / for all ` D 10 000  g C 1, 10 000  g C 2, : : : 10 000  m C 1
2

and yNN Œ` > mŒ` .N Œ`

t / for all ` > 10 000  m 2
C 1 . The SS `,ks ,t with k s D 0 will have SS `,ks ,t D 0 for `  10 000  m 2
C 1 and
SS`,ks ,t > 0 for ` > 10 000  m2 C 1, and therefore, SS`,ks ,t is not influenced by widespread lower counts than expected within the
non-outbreak region, whereas the group scan statistic with g > m2 is adversely influenced by such situations.
2. Aggregate over the g smoothed ordered counts yNN Œ` that correspond to RŒ` for ` D 1000  g C 1, 1000  g C 2, : : : , 10 000, and
compare them with the corresponding aggregation of the mean count N Œ` t using a traditional SN ratio, that is
0 1 ,v
10 u
X 000 X
10 000
u X
10 000
Z2g D @ yNN Œ`  N Œ`
t
A t N Œ`
t .
`D10 000gC1 `D10 000gC1 `D10 000gC1

Flag an outbreak when this exceeds an appropriate threshold (hz2g .Œ` t /) designed to deliver a specified false discovery rate of
ATS=100. This is similar to the traditional scan statistic, albeit an EWMA version (Sparks32 and Sparks and Patrick17 ).
These two scan plans are asymptotically equivalent and therefore are not differentiated in the remainder of the paper (the
equivalence of these plans was checked by simulation, and their performances were almost identical). Applications involving non-
homogeneous counts will use the adaptive version of these statistics. These are constructed by dividing their statistics by their
respective homogeneous thresholds, and then applying a threshold of one for the resulting adaptive statistic; for more information on
adaptive control statistics, see Sparks.23, 33

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

Table VII. Comparing ATS performance of two plans when the in-control ATS=100 and the
communication outbreak involves six people from a gang for 0.2    0.7. One plan is the
EWMA scan plan Z1g and the other uses CUSUM order statistic
Communication outbreaks involving a sub-network of six people

CUSUM order statistic with ˛ D 0.075, kc D 0.005 & g D 300 (denoted COS)
EWMA scan plan for statistic Z1g

hSS ../ 3.721 4.613 4.499 5.623 5.00 6.302 5.40 6.90 6.03 7.84

 0.2 0.3 0.4 0.5 0.7

Z1g COS Z1g COS Z1g COS Z1g COS Z1g COS

ı ATS

1 23.71 23.09 17.40 17.30 13.95 14.32 11.91 12.00 9.71 9.75
2 9.21 9.16 7.20 7.18 5.99 5.96 5.28 5.28 4.45 4.45
3 5.82 5.54 4.59 4.53 3.91 3.89 3.50 3.47 2.96 2.94
4 4.29 4.13 3.42 3.35 2.97 2.97 2.67 2.66 2.31 2.30
5 3.46 3.31 2.83 2.75 2.45 2.42 2.21 2.19 1.94 1.93
6 2.90 2.80 2.40 2.35 2.10 2.05 1.97 1.94 1.79 1.79
7 2.55 2.44 2.10 2.05 1.92 1.90 1.82 1.80 1.55 1.55
8 2.25 2.19 1.91 1.88 1.77 1.75 1.60 1.58 1.26 1.24
ATS, average time to signal; CUSUM, cumulative sum; EWMA, exponentially weighted moving
average.

Tables VI and VII report the results for plan Z1g for communication outbreaks involving 11 and 6 persons of the targeted 100 people,
respectively. Results in Table VI are compared with that in Tables I and II. If the respective out-of-control ATS values are smaller than in
Table I, then these are highlighted in bold text in Table VI. Similarly, results in Table VII are compared with that in Tables IV and V, and if
the respective out-of-control ATS values are smaller than in Table V, then these are highlighted in bold text in Table VII.
The results for  D 0.2, 0.3, 0.4, 0.5, 0.7 in Table VI can be directly compared with the results of the respective  values in Table 1a,
and results for  D 1.0, 1.4, 1.7, 2.0, 2.5 in Table VI can be directly compared with the results of respective  values in Table II. Notice that
the EWMA scan plan performance in Table VI is overall only slightly worse than the CUSUM order statistics plan (Table I). Generally, the
scan plan work better than the CUSUM order statistic plan if there is a smaller step change. In such cases, the CUSUM order statistic plan
could be improved by selecting a smaller ks value, but this would be at the expense of poorer performance for larger step changes.
The results for  D 0.2, 0.3, 0.4, 0.5, 0.7 in Table VII can be compared with the respectively CUSUM order statistic plan in Table IV
for communication outbreak only involving six people. Notice that the EWMA scan plan is generally not competitive with the CUSUM
order statistics plan for small mean communication counts, that is,   0.4. This illustrates that the smaller the outbreak subgroup
(m < 11) and the smaller the mean communication counts (  0.4) are, the less competitive becomes the EWMA scan plan with
a fixed value of g.
It should be noted that CUSUM plan protects against the scenario where all order statistics are below their median value other
than the few that relate to the outbreak. The CUSUM plan achieves this by resetting the sum to zero when communication counts are
persistently below their respective median order statistics. Therefore, the CUSUM statistic SS`,ks ,g,t is not deflated by most of the high-
end ordered statistics, besides those related to the outbreak, being lower than expected. The fact that the CUSUM order statistics offers
protection from this ‘worst case’ scenario and because its overall performance is robust to changes in mean communication counts, it
is now the only method explored in the application to follow.

6. Example of application
For reasons of protecting privacy, example data are hard to publish. Therefore, we will use a realistic simulated example. The internet
is likely to supersede the need for physical meetings, particularly with the increase of CCTV camera surveillance methods. Face-to-face
interactions are no longer necessary. Social media that facilitates easy communication such as twitter, e-mail, Facebook, chat rooms,
and mobile phone allows communications to occur between groups of people who never meet. Monitoring all internet interactions
that relate to all potential criminal activity is very difficult for law enforcement agencies. However, communication monitoring involving
groups of ex-convicts under suspicion or suspected criminals using the common communication mechanisms is potentially feasible.
Assume that in a city, there are a target group of 100 suspected criminals, each involving gangs of 10 people. These could be
people who either have been partners in past criminal activities or are suspected of being partners in a past crime. More information
on criminal activities can be seen in the literature.(34–40 ) In the example, we will assume that there is no contact across gangs, that
is, the probability of a contact between two people not in the same gang is assumed to be 0. Within a gang, the communication
counts are assumed Poisson, with mean counts taken as uniform of the interval 1/7 to 2. We will assume that the diagonal counts in
the communication matrix are the number of daily calls either to the home landline or to their partners’ cell phone with the same

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
R. SPARKS

distribution as the gang communication counts. We will assume that there are subgroups of four or five people (denoted p4 and p5)
from the 10-member gang who are planning a crime. Moreover, assume the following:
 Scenario 1: the gang starts planning its crime by elevating their mean communication counts by one call extra per day, and they
take 3 weeks to plan their crime (labelled 21d).
 Scenario 2: the gang starts planning its crime by elevating their mean communication counts by 1.5 calls extra per day, and they
take 2 weeks to plan their crime (labelled 14d).
 Scenario 3: the gang starts planning its crime by elevating their communication counts by two calls extra per day, and they take
10 days to plan their crime (labelled 10d).
 Scenario 4: the gang starts planning it’s crime by elevating their communication counts by three calls extra per day, and they take
7 days to plan their crime (labelled 7d).
If the target network is assumed to be made up of 100 gangs of 10 people, people within gangs are assumed to contact each other
at least once per week, but a maximum of twice daily. It is assumed that there is no communication between gangs, and therefore, we
are monitoring 100  10  10 D 10 000 communications (identical to the 100 by 100 communication matrix).
One thousand simulation runs of the application will be used to evaluate the performance of the plan. The performance evaluation
in this simulated application is made on the following:
1. How often is the crime detected before it occurred?
2. If flagged before the crime was committed, then was there enough time for the police to apprehend the criminals before or
during the crime?
3. What proportion of the members of the gang is identified by the high-end order statistics, that is, the top-order statistics that
depart significantly from their median value?

6.1. The proportion of planning events detected before the crime is committed
Table VIII indicates how early the communication outbreak was flagged and the proportion of times it flagged before the crime was
committed. It also records the average number of days from the start of the outbreak it took to detect the outbreak. Note that the
crime was detected before it occurred in almost all situations. For example, for the 21-day planning scenario involving a four-person
gang (denoted 21d.4p), it was detected 98.4% of the time before the crime was committed. In more than 70% of the cases (over all
scenarios), the increase in communication levels was detected 3 or more days before the crime was committed. The average time to
detection indicates that, on average across all scenarios, the increased communication is detected well before the crime date, allowing
sufficient time for detective work that may help pinpoint the date of the crime and its location.

6.2. The proportion of the members of the gang identified

Figure 1 indicates the proportion of times the gang members increased communications, relating to the planning of a the crime,
occurs in the top 300 order statistics. Figure 1 illustrates that although the four-person gang takes longer to detect than the five-person

Table VIII. ATS performance of the plans when the in-control ATS=100 and the
communication outbreak involves six people from a gang in 100 network groups of
10-person gangs
Communication outbreaks involving a sub-network of four or five people

Proportion of times flagged as suspicious

d days after crime planning started

d 2 3 7 10 14 21 ATS

Gang Five members

21d.5p 0.008 0.020 0.280 0.538 0.935 1.000 9.1

14d.5p 0.015 0.042 0.727 0.992 1.000 — 6.4
10d.5p 0.033 0.234 1.000 1.000 — — 4.1
7d.5p 0.260 0.925 1.000 — — — 2.8
Gang Six members
21d.4p 0.002 0.011 0.150 0.425 0.755 0.984 11.6
14d.4p 0.012 0.021 0.738 0.850 0.987 — 8.1
10d.4p 0.026 0.132 0.989 1.000 — — 4.9
7d.4p 0.118 0.604 1.000 — — — 3.3
ATS, average time to signal; EWMA, exponentially weighted moving average.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

Figure 1. The proportion proportion of members involved in the communication gang outbreak that are flagged by the 300 high-end order statistic

Figure 2. Diagnosing the members of a gang in the top 300 communication order statistics

gang, the proportion of members in the top-order statistics is generally higher. This indicates that this delay in detection improves
identification of the exact group planning the crime. This was an unexpected surprise.
Figure 2 uses a grayscale image to flag those communication counts that are in the top 300 most unusual order statistics in terms
of their rank position relative to its median order statistics, which is relative to their rank Ri,j . White space indicates that the communi-
cation between the two people in the gang is not in the top 300 ranked order statistics. A black space indicates it is the highest ranked
order statistic, that is, the most unusually high communication count. Spaces that are lower ranked in the top 300 ranked order statis-
tics are shaded gray – the lighter the gray shading, the lower the ranking of the order statistics. In summary, dark black shading in
Figure 2 indicates the communication count with the higher ranked order statistic, lighter shading indicates communication counts in
the top 300 ranked unusual communication counts and white space indicates communication counts that are not ranked in the top
300 most unusual order statistics. Figure 2 therefore provides a useful display for identifying person within gangs with significantly el-
evated communication counts. Each set of plots in rows in Figure 2 involves one simulation run for each of the 7 to 21 days planning
scenarios involving five of the 10 gang members. Figure 2 presents the flagged position of directional communication counts in the
ranked order statistics used in the CUSUM. The left-hand plot in each row in Figure 2 involves the outbreak gang planning the crime,

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
R. SPARKS

and the right-hand plot is the gang of 10 with the next highest SN ratio (the most unusually high communication counts after the
outbreak gang).
In each case in Figure 2, the graphical output clearly identifies the gang of five members with an outbreak in communication,
whereas for the non-outbreak gang of 10, there is no obvious indication of an outbreak cell. Only in the 21-day plan scenario is there
some doubt as whether to include person 6 into the crime-planning group, because person 1 contacting person 6 is ranked as very
unusual, but there is no indication that person 6 engaged with the others in the gang more than usual. In addition, there is no evidence
that others in the crime-planning group (p2 to p5) engaged more than usual with person 6. So it appears that this high-order statistic
is spuriously unusual and unrelated to the crime planning activities. The right-hand plots in Figure 2 indicates what is expected for the
most unusual communication gang when ‘no planning is going on’.
Figures 1 and 2 demonstrate that, on average, a high proportion of the gangs with elevated communication is captured by the high-
end order statistics. Certainly, Figure 2 helped select the five members of the gang involved in the outbreak, but most outbreaks are
going to involve a random five members of the gang of 10. In these cases, we can re-order the members in the network of 10 in terms
of their signal-to-noise ratio using their marginals communication counts (that is, how unusual their total number of calls they make
to all others), and then the same effect would be achieved in Figure 2. This is based on the assumption that people do not stop calling
others in their social network of 10 because they are planning a crime (a feasible scenario).
The five-member gangs are identified fairly accurately by the plan (see the .5p group for example in Figures 1 and 2). Firstly,
consider the crime-planning gang that increases their communication level by one call per day over a 21-day crime-planning period
(see the 21d.5p boxplot in Figure 1). When first flagged, 75% of the simulations identified more than 50% of the outbreak members, and
50% of the simulations identified more than 64% of the outbreak members. Secondly, consider the crime-planning gang that increased
their communication on average 1.5 calls per day over a 14-day crime planning period (see the 14d.5p boxplot in Figure 1). When first
flagged, 75% of the simulations identified more than 64% of the outbreak gang member communications, and 50% of the simulations
identified more than 72% of the outbreak member communications. Thirdly, consider the gang that increased their communication on
average three calls per day over a 7-day crime-planning period. When first flagged, 75% of the simulations identified more than 72%
of the member communications, and 50% of the simulations identified more than 80% of the member communications. Note that
if the plan continues to run after the communication outbreak has been flagged, then the identification of the members planning a
crime would be improved. This is less true for the 21-day crime-planning period (Scenario 1) than for the 7-day crime-planning period
(Scenario 4).
Provided that the crime-planning gang communication level has increased enough and that the appropriate level of detective work
is carried out after the outbreak has been detected, this simulation clearly demonstrates that the monitoring plan outlined in this paper
would help (Table VIII) the police to either avoid the crime being committed when people’s lives are at stake or apprehend the crime-
planning gang while the crime is being committed. It is not only useful for early detection of the planning phase but also helpful for
identifying the members of a gang involved in the planning.

7. Concluding remarks
This paper introduces a new surveillance algorithm that is computationally feasible compared with the all-subset scan statistic. For the
matrix of 100 by 100 targeted people, this new approach delivers gains when the outbreak covers small number of communications
(<121), but in cases where the outbreak involves a larger number of communications, that is, 225 or more (established using simula-
tion), it is better to monitor the total volume of communication counts using TEWMAt . This suggests that the technology will be useful
in crime management because most crimes are committed by gangs small in size.41 Law enforcement agencies would value any in-
telligence tool that offers them timely and trustworthy insights into the behaviour of persons of interest. For example, it would be
helpful to gain insights on how to reduce juvenile crime, which is likely to help reduce the number of people eventually becoming
repeat-offending criminals.
This new plan uses a general multivariate EWMA approach to accumulate temporal memory of communication counts, but then uses
the CUSUM for assessing the significance of these smoothed communication count departures from expected values. The approach
has the added advantage in that it can easily be extended to more than one communication channel. The plan has also been extended
to cover problems with non-homogenous mean counts (as in the application), and its value is demonstrated with an application. The
approach is applicable to monitoring a social networks each involving b targeted people, where a  b  b D 10 000, for example, when
a D 400 andPb D 5. Alternatively, more generally, let there be q targeted social networks, such that the ith social networks involve bi
people and qiD1 b2i D 10 000.
Future research should explore the potential of extending this approach to cover geographical dimensions as well (Carley et al.4 ).
Future research efforts should also focus on optimising the temporal smoothing process or selecting the appropriate level of aggrega-
tion (i.e. selecting ˛ , g and ks ) . If there is an interaction between the group of high-end order statistics g, the CUSUM reference value
ks and ˛ , then optimisation is even more challenging. However, even without this optimisation process, the plan shows significant
promise as an outbreak detection method when trying to find small groups with elevated communication counts.
Future research should avoid having a constant value for ks for all high-end order statistics, but consider increasing the magnitude of
ks as ` the rank of the counts increases. This would help the plan to improve its performance for smaller step changes in communication
counts, but it is unclear how this will affect bigger step changes.
If the outbreak is global across all communications of the targeted people, then using TEWMAT is better than applying the CUSUM or-
der statistic plan, but if the communication outbreaks involves a small subgroup of the targeted people, then the CUSUM order statistic

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

plan performs better. The size of the outbreak group is seldom known in advance, and so applying these plans (TEWMAt and ASS`,ks ,g,t )
simultaneously in a single plan may offer a more robust plan for detecting the full range of potential outbreaks. Another option worth
exploring in the future is to simultaneously apply say three or more CUSUM order statistic plans with different levels of subgroup
memory. For example, use g D 1000 and k D 0.003, k D 0.005 and k D 0.007 and apply ASS`,ks D0.002,gD1000,t , ASS`,ks D0.005,gD1000,t and
ASS`,ks D0.007,gD1000,t simultaneously. A signal is given if any one plan flags an outbreak. Train the overall plan to have a fixed in-control
ATS similar to the several CUSUM approach used in Sparks.23
This paper arbitrarily selected ˛ D 0.075. Therefore, future research effort could be devoted to not only selecting an appropriate
value for the multivariate temporal smoothing ˛ but also establishing an appropriate robust choice for ˛ or alternatively varying the
choice of ˛ for each communication count to exploit local trends such as Capizzi and Masarotto.42

Acknowledgements
The author would like to acknowledge Professor William H. Woodall for helpful comments that have improved an earlier version of
this paper.

References
1. Tambayong L. Change detection in dynamic political networks: the case of Sudan. Chapter 5 in Theories and Simulations of Complex Social Systems,
Intelligent Systems Reference Library 2014; 52:43–59. http://link.springer.com/chapter/10.1007/978-3-642-39149-1-5.
2. Egele M, Stringhini G, Kruegel C, Vigna G. COMPA: Detecting compromised accounts on social networks. In Symposium on Network and Dis-
tributed System Security (NDSS), San Diego, CA United States, 2013. http://scholar.google.com.au/citations?view_op=view_citation&hl=en&user=fM-
s2vQAAAAJ&citation_for_view=fM-s2vQAAAAJ:YsMSGLbcyi4C.
3. Roy RB, Sarkar UK. A social network approach to change detection in the interdependence structure of global stock markets. Social Network Analysis
and Mining 2013; 3:269–283. DOI: 10.1007/s13278-012-0063-y.
4. Carley KM, Liu H, Goolsby R. Near real time assessment of social media using geo-temporal network analytics, 2013; 517–524.
5. Sakaki T, Okazaki M, Matsuo Y. Earthquake shakes Twitter users: real-time event detection by social sensors. Proceedings of the 19th International
Conference on World Wide Web. ACM, 2010,851–860. DOI: 10.1145/1772690.1772777.
6. McCulloh I, Carley KM. Detecting change in longitudinal social networks. Journal of Social Structure 2011; 12:3–37.
7. Everton SF, Cunningham D. Detecting significant changes in dark networks. Behavioral Sciences of Terrorism and Political Aggression 2013; 5(2):
94–114. DOI: 10.1080/19434472.2012.725225.
8. Maxson CL, Klein MW. Investigating gang structures. Journal of Gang Research 1995; 3(1):33–40. http://www.ncjrs.gov/App/publications/abstract.
aspx?ID=158704.
9. Neill DB. Expectation-based scan statistics for monitoring spatial time series data. International Journal of Forecasting 2009; 25:498–517. DOI:
10.1016/j.ijforecast.2008.12.002.
10. Nakaya T, Yano K. Visualising crime clusters in a space–time cube: an exploratory data-analysis approach using space–time kernel density estimation
and scan statistics. Transactions in GIS 2010; 14(3):223–239. DOI: 10.1111/j.1467-9671.2010.01194.x.
11. Zeng D, Chang W, Chen H. A comparative study of spatio-temporal hotspot analysis techniques in security informatics. In Intelligent Transportation
Systems, 2004. Proceedings. The 7th International IEEE Conference on IEEE 2004, Washington, DC, 2004,106–111. DOI: 10.1109/ITSC.2004.1398880.
12. Kim Y, O’Kelly M. A bootstrap based space–time surveillance model with an application to crime occurrences. Journal of Geographical Systems 2008;
10(2):141–165. DOI: 10.1007/s10109-008-0058-4.
13. Weiß CH. Controlling correlated processes of Poisson counts. Quality Technology and Quantitative Management 2007; 6(2):137–153.
14. Weiß CH. Controlling correlated processes of Poisson counts. Quality and Reliability Engineering International 2007; 23:741–754. DOI: 10.1002/qre.875.
15. Sparks R, Carter C, Graham PL, Muscatello D, Churches T, Kaldor J, Turner R, Zheng W, Ryan L. Understanding sources of variation in syndromic
surveillance for early warning of natural or intentional disease outbreaks. IIE Transactions 2010; 42(9):613–631. DOI: 10.1080/02664760903186056.
16. Page E. Continuous inspection schemes. Biometrika 1954; 41:100-115. DOI: 10.2307/2333009.
17. Sparks R. Detection of multiple outbreaks using spatio-temporal EWMA ordered statistics. Communication in Statistics: Simulation and Computation
2014; 43(10):2678–2701. DOI: 10.1080/03610918.2012.758741.
18. Sparks R. Detection of spatially clustered outbreaks in motor vehicle crashes: what’s the best method? Safety Science 2011; 49:794–806. DOI:
10.1016/j.ssci.2010.06.007.
19. Sparks RS, Keighley T, Muscatello D. Improving EWMA plans for detecting unusual increases in Poisson counts. Journal of Applied Mathematics and
Decision Sciences 2009:1–16. DOI: 10:1155/2009/512356.
20. Zhou Q, Zou C, Wang Z, Jiang W. Likelihood-based EWMA charts for monitoring Poisson count data with time-varying sample sizes. Journal of the
American Statistical Association 2012; 107:1049–1062. DOI: 10.1080/01621459.2012.682811.
21. Prusiewicz A. A multi-agent system for computer network security monitoring. Lecture Notes in Artificial Intelligence 2008; 4983:s.842-849. http://link.
springer.com/chapter/10.1007/978-3-540-78582-8-85.
22. Moustakides GV. Optimal stopping times for detecting changes in distributions. The Annals of Statistics 1986; 14(4):1379–1387. http://www.jstor.org/
stable/2241476.
23. Sparks R. CUSUM Charts for signalling varying location shifts. Journal of Quality Technology 2000; 32(2):157–71.
24. Jiang W, Shu L, Apley DW. Adaptive CUSUM procedures with EWMA-based shift estimators. IIE Transactions 2008; 40:992–1003. DOI:
10.1080/07408170801961412.
25. Lucas JM. Counted data CUSUMs. Technometrics 1985; 27:129–144. DOI: 10.1080/00401706.1985.10488030.
26. Soresson C. A CUSUM framework for detecting of space–time disease clustering using scan statistics. Statistics in Medicine 2007; 26:4770–4789. DOI:
10.1002/sim.2898.
27. Han SW, Tsui K-L, Ariyajunya B, Kim S. A comparison between SCAN and CUSUM methods for detecting increases in Poisson rates. Quality and
Reliability Engineering International 2010; 26:279–289. DOI: 10.1002/qre.1056.
28. Schuh A, Woodall WH, Camelio JA. The effect of aggregating data when monitoring a Poisson process. Journal of Quality Technology 2013; 45(3):
260–272.
29. Glaz J, Naus J, Wallenstein S. SCAN Statistics. Springer: New York, 2001. DOI: 10.1007/978-0-8176-4749-0.
30. Shu L, Jiang W. A Markov chain model for the adaptive CUSUM control chart. Journal of Quality Technology 2006; 38:135–147.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
R. SPARKS

31. Tango T. A test for spatial disease clustering adjusted for multiple testing. Statistics in Medicine 2000; 19(2):191–204. DOI: 10.1002/(SICI)1097-
0258(20000130)19:2<191::AID-SIM281>3.0.CO;2-Q.
32. Sparks R. Spatially clustered outbreak detection using the EWMA SCAN statistics with multiple sized windows. Communications in Statistics -
Simulation and Computation 2012; 41(9):1637–1653. DOI: 10.1080/03610918.2011.611314R.
33. Sparks R, Keighley T, Muscatello D. Early warning CUSUM plans for surveillance of negative binomial daily disease counts. Journal of Applied Statistics
2010; 37(11):1911–1929. DOI: 10.1080/02664760903186056.
34. Goldsmith A, Halsey M. Cousins in crime: mobility, place and belonging in indigenous youth co-offending. British Journal of Criminology 2013;
53:1157–1177. DOI: 10.1093/bjc/azt039.
35. Felson M. The Process of Co-offending, in Theory and Practice in Situational Crime Prevention, Smith M, Cornish D (eds). Criminal Justice Press.:
Monsey, NJ, 2003; 149-167.
36. Andresen M, Felson M. The impact of co-offending. British Journal of Criminology 2010; 50:66–81. DOI: 10.1093/bjc/azp043.
37. Grund T, Densley J. Ethnic Heterogeneity in the activity and structure of a black street gang. European Journal of Criminology 2012; 9:388–406. DOI:
10.1177/1477370812447738.
38. Delinquency SM. Skills, social the structure of peer relations: assessing criminological theories by social network theory. Social Forces 2010; 89:
609–632. DOI: 10.1353/sof.2010.0069.
39. Tita. Gaining insight into the spatial patterns of gang violence through social network analysis. Keynote Address Fourth Annual Illicit Networks
Workshop. Vancouver, Canada, 2012.
40. Papachristos AV, Hureau DM, Braga AA. The corner and the crew: the influence of geography and social networks on gang violence. American
Sociological Review 2013; 78:417. DOI: 10.1177/0003122413486800.
41. Morgan KA, Shelley WW. Juvenile Street Gangs. The Encyclopedia of Criminology and Criminal Justice, 2014. DOI: 10.1002/9781118517383.
wbeccj543.
42. Capizzi G, Masarotto G. An adaptive exponentially weighted moving average control chart. Technometrics 2003; 45(3):199–207. DOI: 10.1198/
004017003000000023.

Authors’ biography
Ross Sparks is a Statistician with over 38 years research and teaching experience at universities and with research based consulting
at CSIRO, Australia (over 24 years with CSIRO). His role role is in leading strategic and tactical research projects in CSIRO Data61 in the
areas of statistical analysis and design, multivariate spatio-temporal monitoring and multivariate spatio-temporal modelling.

Appendix A
An R routine for establishing the median order statistics for communication counts with a burn-in period of 25 observations, communi-
cation count means equal to  (lam), the smoothing constant ˛ (al), targeted number of people n D 100 and g the number of ordered
statistics included in the CUSUM is:

gen.median<-function(lam,g,nob=4000,al){
### nob is the number of simulation runs - default is set at 4000
lam<-matrix(lam,100,100)
zordert<-matrix(0,g,nob)
for(j in 1:nob){
### Burn-in period of 25 observations - sufficient if al>0.02
for(i in 1:25){
y<-rpois(10000,lambda=lam)
if(i==1){ew<-al*y+(1-al)*lam
ew[ew<lam]<-lam[ew<lam]} else {ew<-al*y+(1-al)*ew
ew[ew<lam]<-lam[ew<lam]}}
y<-sort(ew,decreasing=FALSE)[(10000-g+1):10000]
zordert[,j]<-y}
zorder<-apply(zordert,1,median)
zorder}

Appendix B

The approach used to generate the models in this table was to first simulate the matrix of counts, assuming mean counts
is homogeneous with mean count equal to . The  values varied systematically using  DD 0.001, 0.002, : : : , 0.03,
0.032, 0.034, : : : , 0.1, 0.105, : : : , 1.0, 1.01, : : : , 3.0, 3.02, : : : , 4.2). These counts were smoothed for 25 consecutive days of generated
counts delivering a steady-state process (0.9252 4 D 0.15, that is, little memory of the starting value). A threshold was selected, and its
respective in-control ATS was determined by simulation. This was repeated until the in-control ATS converged onto 100, and the re-
lated threshold was documented. This simulation approach therefore found the threshold for each of the values of  for a given value
of g and ks . A model was then fitted to these threshold values in each case for given values of . The resulting fitted model is reported
in Table A.

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015
 R. SPARKS

Table B1. Estimated thresholds for problems with homogeneous spatial communication mean with mean counts 0.001   
4.2, ˛ D 0.075, g D 300, 400, 500 & ks D 0.005, 0.006, 0.007
Estimated threshold (hSS .ks D 0.005, g D 200, /=)

 p0.22 p
5.496107 C 0.6028122log./ C 13.01678   16.9087006 C 7.8022614log./ 
 > 0.22
exp.1.418107202 C 0.670180978log./  0.136153078 C 0.0252473372  0.0021766393 /

Estimated threshold (hSS .ks D 0.0005, g D 300, /=)

  0.2
0.09986553  0.05704258log./  11.017102582 C 9.86383785
 > 0.14
exp.1.700329407 C 0.760278397log./  0.206437876 C 0.0346261302  0.0025872073 /

Estimated threshold (hSS .ks D 0.0005, g D 400, /=)

p   0.14 p
7.89673  0.7480382log./  134.9501643  C 129.7435446  31.0928713log./   61.6852917log./

 > 0.14
exp.0.98082 C 0.4346890log./  0.1693147 C 0.031208942  0.024312533 /

Estimated threshold (hSS .ks D 0.0005, g D 500, /=) for g D 500

 p0.2 p
1.80787 C 0.1720078log./ C 4.3017786  C 0.6521600 C 2.6062431log./ 
  0.2
exp.2.127734 C 0.931071log./  0.4139266 C 0.086143852  0.0079244853 /

Estimated threshold (hSS .ks D 0.0006, g D 500, /=)

  0.2
p p
2.54185923 C 0.90760311log./  1.14288814   0.09497326 C 0.33636228log./ 
  0.2
exp.2.2512074 C 1.1319543log./  0.6921437 C 0.15726322  0.01525423 /

Estimated threshold (hSS .ks D 0.0006, g D 400, /=)

p   0.85 p
0.52022992  3.17939255  C 0.05689085log./ C 11.77330871  6.543217112 C 2.422616843  0.31638036 log./
  0.85
exp.1.81058567 C 0.86432924log./  0.24572489 C 0.034866502  0.00185955 3 /

Estimated threshold (hSS .ks D 0.0007, g D 500, /=)

p   0.85 p
2.9214315  7.7427204   0.3401947log./ C 19.0102806  3.38944042  4.5611877 log./
  0.85
exp.1.860867652 C 0.965104601log./  0.284050483 C 0.0446297292  0.0034006352 /

Copyright © 2015 John Wiley & Sons, Ltd. Qual. Reliab. Engng. Int. 2015