Disfribufed Dam Mining in

Credit C u d Fmud Defwdon

Philip K. Chan, Florida Institute of Technology
Wei Fan, Andreas 1. Prodromidir, and Salvotore 1. Stalfo, Columbia University

_-
EDIT CARD TRANSACTIONS CON-
tiiiue to grow in iiuinbcr, taking an ever-larger THIS
SCALABLE BLACK-BOX APPROACH FOR BUILDING
share or the US paymcnt system and leading EFFICIENT FRAUD DETECTORS CAN SIGNIFICANTLY REDUCE
lo a higher rate of stolen nccount numbers
and subscquenl losses by himks. Improved LOSS DUE TO ILLEGITIMATE BEHAVIOR. IN MANY CASES, THE . .il
I "

fraud detection thus has become essential to
maintain Iht: viability OS the US payment sys-
lem. Banks liave used early fraud warning
sysletns for some ycars.
I.arge-scale data-mining techoiqucs can
improve on the statc ol the art i n commercial
practicc. Scalable tcchniqries tu analyEe miis-
sive amount^ oS transaction data that e m -
ciently cnmpole fraud detectors i n a timely
manner is an important problem, especially
Sore-commerce. Besides scalahility and effi-
ciency, thc Sraud-detection task cxhihits lecli-
nical problems that includc skewed distribu-
tions OS training tlntii ;ind oonunil~irincos1
pel- crror, both of which have iiot been widely
studied in the knowledgc-discovery and dala-
niining community.
In this article, wc survey and evaluate a
iiiirnheroStechniques that address lhesc three
main issues concurrenlly. Our proposed
mcthods of combining multiple learncd fraud
detectors under a "cost modcl" are general
imd demonstrably useful; our cnipirical
results demonstrate that we can significantly
reduce loss due to fraud through distributed
dah1 mining of fraud models.
AUTHORS' METHODS OUTPERFORV A WELL-KNOWN, STATE- j . ,
OF-THE-ART COMMERCIAL FRA UD-DETECTION SYSTEM.
Our approach
In today's increasingly elcctronic society
and with the rapid advances orelectronic com-
merce on thc Inlernet, the use oicredit cards
for purchases has become convenient and nec-
essary. Credil card transacti(ins have heconic
the dc facto standard lor Internet and Web-
based e-commerce. Thc US government esti-
niates that credit cards accounted for approx-
imately US $ I1 hillion in Intcrnet sales during
1998. This figure is expected IO grow rapidly
each year. However, the growing number of
credit card transac~ioiisprovides morc oppor-
tunity for thieves to steal credit card numhcrs
and subscquently comniil Sraud. When banks
lose money hccause of credit card Sraud, card-
hddcrs pay fur $111ofthat loss through highcr
interest ratcs, higher fees, ancl reduced bene-
fits. Hence, it is in both the hanks' and thc
cardholders' interest to reduce illegitimate use
of credit cards by early fraud detection. For
many years, the credit card industly has stud-
ied computing models for automated detcc-
tion systems; recently, these models have been
the subject of academic research, especially
with respect to e-commerce.
The credit card fraud-detection domain
presents a number of challenging issues for
data mining:
There arc millions of credit card transac-
tions processed each day. Mining such
massive amounls of data requires highly
- eft'icienl techniques that scale.
The datu are highly skewed-many more
transactions are legitimate than Sriludu-
lent. Typical accuracy-based mining tecli-
niques can generate highly accurate fraud
dctcctors by simply predicting that all
~ ~ _ _ ~ _ -
67
 Output the final hypothesis:
transactinns arc legitimate, although this
is equivalenttonotdeiecting fraudat all.
Each transaction record lias a diSfercnt
dollar amount a i d thus lids a veriable
potential loss, rather than a Fixed misclas-
sificiltian cost per error type , 'IS' ' IS
' corn-
monly assumed in cost-based mining
techniques.
Our approach addresses the efficiency and
scalabilily issues in several ways. We divide
a largc tlala sct OC lahcled transactions (eithcr
Craudulent or legitim;ite) into smallcr subsets,
apply mining techniques lo generate classi-
Ciers i n parallel,and cornhine Ihc resullant
basc models by metalearning h n i the clessi-
fiers' behavior to gencrate a metaclassiSier. I
Our approach treats the chssilicrs as black
boxes so thaL we can employ a variely of learn-
ing algorithms. Besides exlensibility, coni-
bining mulliplc models computed over all
available data produces inctaclassiSiers that
can ofSset the loss of predictive performance
that risuelly occurs when mining Sroin dala
subsets or sampling. Furthermore, when we
USC the leemed classifiers (fur example, dur-
ing transaction autliorizalion). the base clas-
sificrs call executc in parallcl, with the mala-
classilier then combining theirresnlts. So, our
approach is highly cfkient in generaling these
models and also rcletively efficient in apply-
ing them
Another parallel approacli focuses on par-
- ~~
1611 INTELLIGENI SYSIEMS-

allclizing aparticolar algorithm on a particu-
lar parallel architecture. However, a new
algorithm or architecture requircs a substaii-
tial amount or pal.allcl-programining work.
, Although our architeclure- and algorithm
1 independent approach is not as efficient as
some finc-grained paral lelinition approaches.
it lets users plug difSeren1off-the-shelflenrii-
ing programs into a parallel and distributed
envimnineiit with relativc case and eliminatcs
the nccd Sor expensive parallel hardware.
Furthermore, hccause our appruach cmld
generate a potenlially large nuniber orclassi-
fiers from the conciirrcntly processed dala
subsek, and lhereforc potentially ieyiiirc more
cumputational resourccs during detection, we
investigate pruning methods that identify
rcdundant classificrs and remove them from
thc cnsemble wilhout sigiiificaiitly degrading
the predictivc pcrformance. This pruning lech-
nique increases the lcarned &lectors’ coni-
putational pciforniaiice and throughput.
The issue of skewed distributions has not
been sludicd widely because many of the data
sets used in research do not exhibit lhis char-
actciistic. We address skewness hy partition-
inglhedattisct intosubsets withadeaireddis-
tribution, applying mining techniques to ihe
subscts, and combining the mined classifiers
by mctalearning (as we have already dis-
cussed). Other researchers attempt to rcmove
unncccssary instanccs from the majority
class-instances that are i n the borderline
region (noise or redundant exeinplarsj are can-
didales [or rcnioval. In contrasl, our approach
kecps all the data for mining and does not
change the underlying inining algorithms.
We addrclmss thc issue of nonuniform cos1
by dcwloping the appropriate cost model f b r
the crctlit card fraud domain and biasing oiir
methods toward redocing cosl. This cost
modcl dctermiiies thc desired distribution
just mentioncd. AdaCost (a cost-sensitive
version oCAdaBoost)relies on thc cost model
for updating wciglits in the training distri-
bution (FurmoreonA&iCost, sce the“Ada-
Cost ;ilgorithm” sidebar.)Naturally, this cost
model also defines lhc primary evaluation
criterion for our techniques. Furthermwc, wc
investigate techniqucs to improve the cost
perlormance of a hank‘s fraud deteclor by
importing remote classifiers from other
hiinks and cornhiniiig this rcniotcly learned
knowledge with locally stored classificrs.
The law and coinpetitivc coiicerns restrict
hanks Srom sharing inlirrmation about their
custwners wilh othcr banks. However, they
may sliare black-hox Sraud-detection mod-
CIS. Our distributed data-mining approach
pruvidcs a direct and efficient solution io
sharing knowlcdge wilhout sharing data. We
also address possiblc incoinpiitihility of data
schemata among clifrerent banks.
We designed and dcveloped an agent-based
distributed environmcnt to demonstrate our
distributed and parallel data-mining tech-
niques. The1AM (Java Agents fur Mctalcarn-
ingj systcin not only providcs distributed data-
niining capabilities, it elso lcts uscrs monitor
and visimlizc the various learning agents and
derived models i u real time. Rescarchco havc
studied a variety of algoiithnis and techiiiques
hrcomhining multiple coniputed models. The
JAM system pruvidcs generic fealures to cas-
ily implemenl any of these combining tech-
large collection of base-
learning algorithms), and it has been broadly
availahlc~oruse.TlieIAMsystcmis available
h r download at http:llwww.cs.coluinbi;l.edd
-sal/ JAM/PRO.IECT.

Credit card data and cost
models
Chase Bank and First Union Bank, m e n -
hers of the Financial Services Technology
Consortium (FSTC), provided tis with real
credit card data for this study. Thc two data
sets contain credit card transactions labcled
as fraudulent or legitimate. Each bank sup-
plied S00,000records spanning onc yew with
20% fraud and 80% nonfiaud distribulion for
Chase Bank and I S % versus 85% for First
Union Bank. In practice, fraudulent transac-
tions arc much lcss frequent than the 15% to
20% observed in the data given to us. These
data might have been cases where the banks
have difficulty in determining legitimacy
correctly. In some OS our experimenls, we
deliberately creale more skewcd distributions
tu evaluate the effectivencss of our tech-
niques under more cxtreme conditions.
Bank personnel devcloped the schemata
ufthe databases ovcr years of experience and
continuous analysis to capture important
information for fraud deteclion. We cannot
reveal the details ofthe sclicma beyond what
we havc described elsewhere.* Thc records
of one schema have a fixed length of 117
bytes each and about 30 attributes, including
the binary class label (fraudulenrllcgitimatc
transaction). Some ficlds Lile numeric and the
rest categorical. Because accounl identifica-
lion is not present in the data, we cannot
group trailsections into accounts. Thercfore,
inslead of leaning behavior models ofindi-
vidual customer accounts, we build overall
models that try to differentiate legitimate
transactions from fraudulent ones. Our mod-
els are customer-independent and can serve
as a second line of defense, the first being
customer-dependent models.
Most machiidearning literature conccn-
trates on model accuracy (either lrainiiig error
or gencralination error on hold-out test data
computed as overall accuracy, tme-positivc or
false-positive rates, or return-on-cost analy-
sis). This domain provides a considcrdbly dif-
ferent metric to evaluate the learned models'
performance-models arc evaluated and rated
by a cost model. Due to lhe different dollar
amount of each credit card transactiun and
otherfactors,thecostoff;iilingtodctectafraud
varies with cach transaction. Hence, the cost
model for this domain relies on lhe smn and
average of loss caused by Sraud. We define
Cumu1ati"eC"rf = Cust(i)
Table I. Cost model assuming a fixed overhend.
Oll,EOME COST
Miss (false negative-FN) Tranarnt
False alarm (false oositive-FPI ' Overheadif tranamf > overheador 0 if tranarnts overhead
Hit (true posiiive-TP) Overheadif lranamt, overheador fmnainf if tranarnts overhead
Normal (true negative-TN) 0
and
whcre Cost(i) is the cost associated with
transactions i, and n is the total number of
Lransactiuns.
After consulting with a bank representa-
tive, we jointly settled on a simplified cost
model that closely reflects reality. Because it
takes time and personnel to investigate a
potentially fraudulent transaction, each inves-
tigation incurs an averliend. Othcr related
costs-for example, the operational resources
needed for the Iraud-detection system-me
consolidated into overhead. So, if the amount
of a transaction is smxller than the ovcrhwid,
investigating the transactivn is not worth-
while even if it is suspicious. For enamplc, if
it takes $10 to investigate a potential luss of
$1, it is more economical not to investigate
it. Therefore, assuming a Sixed overhead, we
devised the cost inodel shown in Tahle I for
each transaclion, where franamfis thc amount
of a credit card transaction. The overhead
threshold, cor obvious reasons, is a closely
guarded secret and varies over time. The
range of values uscd here arc probably rea-
sonable levels as bounds for this data sct, hul
are probably signilicantly lower. We evalu-
ated all our empirical studics iisiiig this cost
model.
Skewed distributions
Given askewed dislributiiio, we wouldlike
to generate atraining set of labeled transactions
with a desired dislribution witlriul removing
any data, which maximizes classifier perior-
mancc. In this domain, we found that deter-
mining lhe desired distribution is an expcri-
mental art mdrequiresextcnsiveempirical lesls
to find the most eSkctivc training distribution
In our approach, we lirst create data sub-
sets with the desired distribution (determined
by exlensive sampling experiments). Then
we generate classifiers Srom these subscts
and combine them by mctalearning lrom
their classification bchevior. For example, iS
the given skewed distribution is 2W80 and
the desired distribution for gencraling the
hcst models is 5050, we randomly divide the
majority instances into Sonr partitions and
Sorm four data subsets by merging the minor-
ity instances with each oC the four partitions
crmtaioing majority instances. That is, the
minority instances replicate across four data
subsets to gencrale the desired 5O:SO distri-
bntion for each distributcd training set.
For c~incretencss,let N he the size of the data
set with a distribution of x y (r is the percent-
agcoSthemior~rilyclass)andu:vbcthedesired
distribulion. The nuinbcr olminority instanccs
is N x x, atid the desired number of majority
instances in a subset is Nx x v h . The number
of subsets is the nuinher of majority instances
IN x y ) dividcd by the number ol desired
majority illstances in cach subset, which is Ny
dividcd by Nxvlu or ylx x ulv. So, wc have
y/xxulvsubscts,eachofwhichhas Nxminor-
ity illstances and Nxvlu majority instanccs.
The next step is to apply a learning algu-
rithm or algurilhms to each subset. Because
the learning processes on the subsets arc inde-
pendent, the subsets can be distributcd to dif-
ferent processors and each learning process
zan run i n perallcl. For massive amounts of
:lata, our approach can substanlially improve
rpeed for superlincal- time-lcerning algo-
rithms. Thc generatcd classifiers are coin-
billed by inetalcarning from their classifica-
tion behavior. Wc liave described sevcral
metalearning strategies e1sewhere.l
To simplify our discussion, we only
icscribe thc clrrss-combiner (or stuckingj
ulrategy? This strategy composes a inetalevel
lraiiiing set by using the hase classiliers' prc-
iiclions on a validation set as attribute values
and the aclual classilication a s the class label.
rhis training set then serves for training a
InelaclassiSier. For inlegrating subsets, the
:lass-combiner slrategy is inorc eSSective
than the voting-based techniqoes. When the
learned models are used during online Srand
:letcction, transactions feed into the learnccl
mse classifiers and the meteclassifier then
:ombioes lheir predictions. Again, the base
:lassiliers are indepcndent and can exccute in
prallel on diSferent processors. In addition,
)UT approach can prune rcdundant base clas-
sifiers without affecting the cost perlor-
mance, inaking it relatively erlicient in the
:redit card authorization proccss.
 ~~
Table 2. Cost and rovings in the tredil turd fraud domain using tlosr.rombiner(tor1 t 95% ronfidente intervol).
Expcrimcnts and results. To evaluale our
ificr class-combiner approach to
skewed class distributiuns, we perlorined a
se1olexperiments using the credit cm1 fraud
data from Chase? We used transactions from
lhefirsteightinonths(10/95-5/96)lortraio-
ing, the ninth inunth (6196) for validating,
and tlie twelfth month (9196) for testing.
(Because credit card transactions have a nal-
u r d two-month business cycle-the timc to
bill a customer is oiic month, followed by a
one-month payment period-the true label
of atransaction cannot bedelermincd i n less
than twomonths’ timc. Hence, building mod-
els f m n data in one month cannot be ratio-
nally applied fur fraud detection in tlic next
month. We therefore test our models on data
that is at least two months newer.)
Based on lhc empirical results fruni the
effects of class distiibutions, thc dcsircd distri-
bution is 5050. Because the given distribution
i s 20:80, Four subsets are generalcd Crum each
inonthfaratolaloC32suhsets.Wcapplied Cow
learning algorithms (C4.5,CART, Ripper, and
Baycs) to each subset and gcneraletl I28 base
classifiers. Bascd on ourexperiencc with train-
ing metaclassifiers, Bayes is gcnerally more
cflective and efficient, so it is the metaleamer
for all Ihe experiments reported herc.
Furthermore, to invcstigale if our
approach is indccd Ruillul, we rim cxpcri-
ments 011 the class-combincr strategy directly
applied to thc original data sets from the lirsl
eight months (that is, they have the given
20:80 distribution). We also cvaluated how
individual classifiers generated from each
month perform without class-combining.
Table 2 shows the cost and savings from the
class-combiner strategy using thc 5050 dis-
tribution (128 baseclassitiers), Ihe avemgcof
individual CART classifiers generaled using
the desired distribulion (10 cl;issiliers), class-
combiner using the given distribulion (12 base
classifiers-8 niontlis x 4 learning algo-
rithms), and tlie average of individual classi-
fiers using the given distribution (the average
of 32 classiliers). (We did not perform exper-
iments on simply replicating the minority
inslances to achieve S0:50 in one singlc data
- ~~
- _NOVEMBER/DECEMBER
______ 1999
- __
20.07 *.I3 46
23.64 +.96 36
29.08 t 1.60 21
sct bccause this approach increases the train-
ing-set size and is not appropriate in domailis
with large amounts of data-one ol the three
primary issues we address here.) Comparcd
to the otlicr three methods, class-combining
on subsets with a 5 0 5 0 fraud distribution
clcarly achieves a significant increase in s i n -
ings--atleast$IIO,000fortIieinonlh (6196).
When the overhead is $50, inure than halfof
the losses were prevented.
Surprisingly, we also observe that when tlie
50:50 distrihulian (generaled hy ignoring
somc data) achieved significantly more sav-
ings than combining classifiers trained lrom
all eight months’ dala with the given distrih-
uliun. This reaffirms thc importance of
employing the appropriate training class dis-
tribution i n this domain. Class-combiner also
contribuled to the performance improvcrncnt.
Conseqocnlly. utilizing the desircd lriiining
distribution and class-combiner provides ti
synergistic approach to data mining with
nonuniform class and cost distribuliuns. Per-
haps more importantly, how do iiurtcchniques
perform compared to the bank‘s existing
Sraud-detection system? We label thc current
syslein “COTS” (commercial off-the-shelf
systcm) in Table 2. COTS acliicved signiti-
wings than our lechniqucs in the
thrcc overhead aniounts we repoit in Ibis table.
This comparison might not be entirely
accurale becsiisc COTS lies much mure
training data than we have and il might bc
optimized tu a different cost modcl (which
might even bc the simple error rate). Fiir-
lhermore, onlikc COTS, om techniques are
genel-al for problems with skewcd dislribu-
tions and ilu not utilize any domain knowl-
edge in detecting credit card fraud-the only
cxccption is the cost madcl used for evaluii-
tion and search guidance. Nevcrthclcss,
COTS’perfr,rnianccon tlie test data provides
some indication of how the existing fraud-
dctcctioii system behaves in the real warld.
We also evaluated our mctliod with more
skewed distributions (by clownsampling
ininorily instiinces): 1090, 1:99, and I : 9 W
~-
As wc discussed carlier, the desired dislri-
bution is not necessarily 50:5&Ior instaocc,
Ihc desired distribution is 30:70 when the
given ilistributioa is 1090. With 10:90 dis-
tributions, our method rcduced the cost siig-
nificantly more than COTS. With 1 3 9 [lis-
Iributions, our method did not outperform
COTS. Both methods did not achievc any
savings with 1399 distributions.
To cliaractcrize the condition when our
tcchoiques arc cffective, wc calculale R, the
ralio of the overhead amount to the avcragc
cost: R = OverlrrodlAiwra,ye c m f .Our zip-
proach is significantly more cffcctive than
the deploycd COTS whco R < 6. Both mctli-
oils are ti01 effective when the R > 24. So,
under IIrciisonable cost model with a fixed
overhead cost in challenging transaclions iis
potentially fraudulent, when the numbor of
fraudulent transactions is a very small per-
centagc ofthe totiil, it is financially undcsir-
ablc to dclect fraud. Thc loss doe to fhis frrilud
is ycl another cost of conducting business.
Hawcver, fillering out “easy” (or law-risk)
transactions (the data we received were pos-
sihly iiltered by a similar process) can reducc
a high overhead-to-loss ratio. The filtering
process can use fraud detectors that are buill
bascd on indivirlual customer pmfilcs, which
arc now in use by inany credit card cumpa-
nics. These individual profiles charactcrizc
the customers’ purchasing behavior. For
example, if ii customer regularly buys gro-
ceries at a particular supermarket or has set
up a monthly paymenl f i r phone bills, these
triinsactions are closc to no risk; hencc, pur-
chases of similar characteristics cim he safcly
authorized wilhout fiwtlier checking. Ilzduc-
ing the overhead through s1re;imlining husi-
ness operations and incrcased automation
will also lower the ralio.
Knowledge sharing through
bridging
Much of the prior work on combining
multiplc modcls assumes that ;dl mudels
originate frum different (not necessarily dis-
 tinct) subsets of a single data set as a mcans
to increase accuracy (for example, by impos-
ing probability distributions over the in-
stances of the training set, or by stratified
sampling, subsampling, and so forth) and not
as a means to integrate distributed informa-
tion. Although the JAM systcm addresses the
latter problem by employing mctalearning
techniques, integrating classification models
derived from distinct and distributed data-
bases might not always he feasible.
In a11 cases considered so far, all classifi-
cation models are assumed to originate from
databases of identical schemata. Because
classifiers depend directly on the underlying
data's format, minor differences i n the sche-
mata between dalabases derive incompatible
classifiers-lhat is, a classifier cannot be
applied on data of diSCerent formats. Ye1 these
classifiers may target the same concept. We
seek to bridge these disparate classifiers in
some principled lashion.
The banks seck to be able lo exchangc
thcir classifiers and thus incorporate uscful
information in their system that would oth-
erwise be inaccessible to both. Indeed, for
each credit card lransaction, both institutions
record similar inSormation; however, they
also include specific fields containing impor-
tant information that each has acquired sep-
arately and that provides predictive value in
determining Craudulent transaction patterns.
To Cacilitate the exchange of knowledge (rep-
resented as remotely learned models) and
take advantage of incompatible and other-
wise useless classifiers, we need to devise
methods that bridge the differences imposed
by the different schemata.
1. A,,,, # B,,,,: The two attributes are of include it in its schema, and presumably
entircly differcnt types drawn from dis- other altributcs (including tlie common ones)
tinct domains. The problem can then be have predictive value.
ileduced to two dual problems where onc
database has one more attribute than the Method 11: Learn a local model without the
other: that is, missing attribute and exchange. In this
approach, database DO, can leain two local
Sc!iema(DBA)= [Al, A , ...,A,,,A,,+,,C] mudels: one with the attribute A,,,, that can be
Schema(Db',) = [ B l ,B2...., U,,, C) used locally hy the metalearning agents and
one without it that can be subsequently
where we assume thal attribute B<,+lis exchanged. Learning a secondclassifier agent
not prcsent in DB,. (The dual problem without the missing attribute, or with the
has DB,* composed with B,,,,, but A,v,, attributes that belong to the intersection ofthe
is not availahlc to A.) attributes of the two databases' data sets,
implies that the second classifier uses only the
2. A,,,, = B,J+l:The two attributes are of attributes that arc common among the patic-
similar lype hut slightly diffcrent ipating sites and no issue exists for its inte-
semantics-that is, there might be a gration at other databascs. But, remote classi-
map Croni tlie domain OS one type lo the fiers imported by database DB, (and assured
domain of lhe other. For example, A,+I not to involve predictions over the missing
and are fields with time-dependent attributes) can still be locally integrated with
information bul of different dirralion the original model that employs A,,, , In this
(that is, A,,, might dcnote the numher case, the rcmote classifiers simply ignore the
oftimcsaneventaccrirred withinawin- local data set's missing attributes.
dow OS half an hour and lJ,,+l ,nigh[ Bolh approaches address the incompati-
denote lhc number of times the same ble schenra problem, atid rnctnleerning over
event occurred but within IO minutes). these models should procecd in a straight-
forward manner. Compared lo Zbigniew
In both cases (attributeA,,+i is either not Ras's related work," our approach is more
present in DB, UT semantically diffcrent Srom general because our techniques support both
the corresponding &+J, the classifiers CAj categorical and continuous atlributes and are
derived from OB,, are not compatible with not limited to n specific syntactic case or
DB,'s data and LhereForc cannol be directly purely logical consistency of generated rulc
used in DBi:s site and vice vcrs11. But the pur- models. Instead, these bridging techniques
poseof usingadistributedd*a-miningsystein can employ machine- or statistical-learning
and dcploying Icarning agents and metalearii- algorithms to compute arbitrary models for
ing their classilier agents is to he able to com- the missing values.
bine infomiation from different sources.

Database compatibility. The incompatible
schenia problem impedes JAM [rum taking
advantage of all available dalahases. Let's
consider two data sites A and b' with data-
bases DBA and DBB2having similar but not
identical schemata. Without loss of gencral-
ity, we assume that
Schcnta(DB,J = {Al,A,, ...,A,, A,,,,, Cl
Schema(DB,) = [ E , , B2,...,B,,, C)
where, Ai and Bidenote the ilh attribute of
DBA and DB,,respectively, and C the class
label (for example, the fraudllegitimate label
in the credit card fraud illustration) of each
instance. Without loss of generality, we fir-
ther assume lhat Ai = Bi, 1 < i < n. As for the
A,,,! and B,,+l attribute& there arc two
possibilities:
~ ~_______~~
~~
72 ~~~__ ___~..
Bridging methods. Therc are two methods
For handling the missing attribules.s
Method I : Learn a local mode1,for the miss-
ing nrtrihute and exchange. Database UB,
imports, along with tlie remote classificr
rgenl, a bridging agent Srom database DEA
.hat is trained to compute values for the miss-
ing attribute A,,+i in DB,'s dala. Next, DB,
rpplies the bridging agent on ils own data to
A m a l e the missing values. In many cascs,
iowever, this might not be possible ordesir-
ible by DBA (for example, in case the
ittribute is proprietary). The alternative for
latabase Dne is to simply add the missing
ittribute to its data set and fill it with null val-
ies. Even though the missing attribute might
lave high predictive value for DBA, it is of
i o value to DE,,. After all, UB, did no1
Pruning
An ensemble olclassifiers can he unneces-
narily complex, meaning that many classifiers
might bc redundant, wasting resources and
reducing system throughput. (Throughput
here denotes the rate at which a metaclassitier
:a11 pipe through and label n stream of data
items.) We study the efficiency OS metaclas-
sifiers by investigating the effects olpruning
(discarding ceaain base classiciers) on their
performance. Determining the optimal set of
Aassifiers for metalearning is a combinator-
ial prohlem. Hence, the objective of pruning
istuutilizeheiiristictneth~s1osearchforp~-
!ially grown metaclassifiers (metaclassifiers
with pruned subtrees) that are more efficient
ind scalable and at the Same time achieve
:omparable or better predictive peiformance
IEEE INTELLIGENTSYSTEMS
I

Table 3. Results on knowledge shoring and pruning.

CHASE Fins1 UNION

SAVINGS SAVINGS
results than fully grown (unpruncd) mcta-
cUSSlFlCAllON METHOD SIZE ($K) SIZE ($K)
classificrs. To this end, we introduccd two COTS scoring system from Chase NIA 682 NIA NIA
stages for pruning metaclassifiers: tlie prc- Best base classifier over entire set 1 762 1 803
training and posttraining pnining stages. Both Best base classifier over one subset 1 736 1 770
levels are essential and complementary tn Metaclassifier 50 818 50 944
Metaciassifier 32 870 21 943
cach other with respect to the improvement of Metaciassifier (t First Union) 110 800 NIA NIA
the system's accuracy and efficiency. Metaciassifier (tFirst Union) 63 877 NIA NIA
Pretrrrining prunirig rcfers to the filtering Metaclassifier (t Chase - bridging) NIA NIA 110 942
of thc classifiers before they arc combined. Metaclassifier (t Chase + bridging) NIA NIA 110 963
Metaclassifier (t Chase t bridging) NIA NIA 56 962
Instead of combining classifiers in a hrute- ~ ~~~

force manner, we introduce a pscliminary

stage lor analy7,ing the available
and qualifying thcrn for inclusion across six differcnt data sites (each sitc stores For thc Cirst incompatibility, we mappcd
bined metaclassificr. Only those classificrs two months of data) aiid prepared thc sct of thc First Union data values io thc Chmc
that appear (according to one or more prc- candidate bast chssificrs-that is, tlie orig- data's scmantics. For thc second incompati-
defined mctrics) to be most promising par- iiial sct of base classifiers the pruning algu- bilily, we deployed bridging agents to com-
ticipale in tlic final metaclassilier. Hcree,we rithm is callcd to evaluate. Wc computed pute the missing values? Whcn predicting,
adopta black-box ;ipproachthalevaluetcs thc these classifiers by applying fivc learning thc First Union classificrs simply disrcgardcd
selofclassificrs hasedonly on thcir input and algorithms (Bayes, C4.5, ID3, CART, and the real values provided at thc Chase data
output hehavior, not their intcrnal structure. Ripper) to each month of data, crealing 60 sites, while tlic Chase classificrs iclicd on
Conversely,poarmining pruning denotcs the hase classifiers (10 classiliers per data site). both thc commoii attributcs and the predic-
evaluation and pruning of constituent hasc Ncxt, we had each data site import the remotc tions of the bridging agents to dcliver a pee-
classifiers after a complete meta hase classifiers ( S O in total) that wc subse- diction at the First Union data sites.
been constructed. We have impleincntcd and quently used i n the pruning and mctalearn- Tahle 3 sutnmarizcs our results for thc
experimcnted with lhree prctraining and lwo ing phascs, thus ensuring that cach classifier Chase and First Union hanks, displaying the
posttraining pruning algorithms, each with would iiot he tested unfairly on known diatii. savings for cach fraud predictor examined.
different scarch heuristics. Spccificdly, we had cach site use half of its The column denoted as size indicates the
The Sisst prctraining pruning algorithm local date (one month) to tcst, prune, and numher olhssc cizissifiers used in thc cnscm-
rnnks and sclects its classificrs hy evaluating met;iICasn the hase classificrs and the othcr hlc classilictitian system. The first row of
each candidatc classifier independently (mct- halC to evaluatc the pruned and unpruiied T'ihle 3 shows the best possible performance
ric-based), and thc s ~ c ~ nalgorithm
tl ctecidcs rnctacliissifier's ovcrnll pcrlurmance.' In or Chase's own COTS tiutliorizatioii and
by e x m i n i n g the classifiers in correlation csseiicc, this experiment's sctting corrc- dctection system on this data sct. The ncxt
with each other (diversity-based). Thc third sponds to a parallcl sixfold cross-validation. two rows psesent the pcrformance of the hest
reiies on the independent pcrformance o l the Finally, we had tlic two siniulaled banks hase classificrs over the entim sct and over a
classifiers and the manner in which they pse- exchangc their classifier agcnts. In addition singlc month's data, whilc the last four rows
dictwithiespectt~~cachothesandwithrespcctto its 10 locel and SO internal classificrs detail thc pcrformance of tlie unpruned (siue
to the underlying data set (coverage and spc- (those importcrl from their peer duta sites), of SO and 110) and pruncd metaclassifiers
cialty-based). The first posttraining pruning cach site also itnpostcd 60 exlemal classifiers (size of 32 antl 63). The first two of lhese
algorithms are based on a cost-complexity (from the other hank). Thus, we populated metaclassifiers combine only internal (lrom
pruning technique (a lechniquc the CART each simulated Chase d;itii site with 60 (10 t Chase) basc classifiers, whilc the last two
decision-tree learning algorithm uses that S O ) Chase cliissifiers antl 60 First Union clas- combine bolh internal and external (from
scoks to minimize thc cost and size of its trcc sifiers, and we populated each First Union Chasc end First Union) hasc classifiers. We
while iwlucingthc misclassllication mtc). The site with 60 ( I O + 50) Firs1 Union classifiers did not use bridging agcnts in lhese experi-
second is based on the crirrclation between the and6O Chase classificrs. Agaiii,the sites uscd ments, bcuuse Chase defincd a11 attributes
classifiers and thc metaclassilier? half of thcir local data (one month) to test, uscd by the Fiirst Union classifier agents.
Comparcd to Dragos Margineanlu and prunc, and metalearn thc basc classifiers and Table 3 records similar data for thc First
Thomas Dietterich's approach? ours con- the other half to evaluate the pruned or Union data set, with the exception of First
re gencral sctting where ensem- ifier's overall perfor- Union's COTS authorization and detection
fiers can he ohtailled by apply- mance. To ensure fairncss, we did not usc thc performance (it was not made available to us),
different learning algorithms 10 local classifiers in metalearning. and the additional resiills ohtaincd when
over (possibly) distinct datahascs. Further- Thc two databases, huwcvcr, had lhe Col- cmploying special bridging agents born Chase
more, instead of voting (such as AdaBoost) lowing schema differences: to computc the values of First Union's miss-
over thc predictions of ing altsihutcs. Most obviously, these expcri-
classificalion, wc use melalearning to com- * Chasc and First Union dcfined a (nearly ments show the superior performance of meta-
bine thc individual classificrs' predictions. identimil) fcature with dirferent scinan- learning ovcr the single-modcl appsoachcs
tics. and over lhe trxlitional authorization and
Evaluation of knowledge sharing and * Chasc includes two (continuous) features dctcction systems (at leiist for the given data
pruning. First, we distributed the data scts no1 present in the Pirsl Union data. sets). Thc mctaclassifiers outpcrformed thc

73
single base classifiers in every categoiy. More-
over, by bridging the two databases, we man-
aged to further improvc the metalearliing sys-
tem's peiformance.
Howevcr, combining classifiers' agents
from the two banks directly (without bridg-
ing) is not very effective, no doubt because
the attribute missing from the First Union
data set is significant in modeling the Chase
data set. Hence, the First Union classifiers
arc not as effective as the Chase classifiers
on thc Chase data, and the Chasc classificrs
cannot perform at full strength at the First
Union sites without the bridging agents.
This table also shows the invaluable contri-
bution of pruning. In all cLises, pruning suc-
ceeded incomputing mctaclassificrs with sim-
ilar or better fraucl-detcction capabilities, while
reducing their size and thus improving their
efficiency. We have provided deletaileddescrip-
tion on the pmning methods and a compara-
tive study betwcenpredictivepeiformance and
metaclassifier throughput elsewhere?
NE LIMITATION OF OUR AP-
proach to skcwed distributions is the necd to
run preliminary expcriments to determine the
desired training distribution based on a de-
h e d cost model. This process can be auto-
mated, but it is unavoidable because the
dcsireddistributionhighlydepcndson thccost
model and the lcaniing algorithm. Currently,
for simplicity reasons, all the base leamcrs use
the samc desired distribution: using an indi-
vidualized training distribution for each base
learner could improve the perSormance. Fur-
thermore, because thieves also leam and Sraud
patterns evolve over time,some clessifiers are
more relevant than others at a puticular time.
Therefore, an adaptive classifier-selection
method is essential. Unlike a monolithic
approach OS learning oiic classifier using
incremental learning, our modular multiclas-
sifier approach facilitates adaptation over time
and removes out-of-date knowledge.
Our cxperiencc in credit card fraud detec-
tion has also affected other important appli-
cations. Encouraged by our results, we shifted
our attention to the growing probleinof intru-
sion dctection in network- and host-based
computcr systems. Here we seek to perlorm Rough Sers iri Knowledge Iliscovery, Studies
the samc soit of task as in thc credit card fraud in Fuczi,ress and .?@I Compuiin,q, L. Pol-
kowski and A. Skowran, eda., Physicti Ver-
domain. We seek to build models to distin- lag,NewYaik, I99X,pp.98-108.
guish betwcen bud (intrusions or attacks) and
good (normal) conncctions or processes. By 7. A. Pradromidis andS.Stolfo,"PmningMeta~
firs1 applying fcatue-extraction algorithms, classifiers in il Distributed Data Mining Sys-
tem," Pmc. Firrt N d l Conf. New Infomu-
followed by the application of machinc-leani-
tion Technologies, Editinns of New I t c h . ,
ing algorithms (such as Rippcr) to learn and Athens, 1998,pp. 151-160.
combine multiple models for differclit lypes
of intrusions, we have achievcd remarkably 8. D. Margineantu and T. Dietterich, "Pruning
good success in detecting intrusions? This Adaptive Baosting," P,oc. 14lh / d l CO$
Machine Learnbrg, Morgan Kaufmiinn, San
work, as wcll as the results reported in this Francisco, 1997,pp. 211-218.
article, demonstrates convincingly that dis-
tributed data-mining tcchniques that combine 9. W.Lce,S.Stolfa,siidK.Mok,"Mining ins
multiplc modcls producc effectivc fraud and Data-Flow Environment: Experiencc in Net-
work Intrusion Ilctection," Proc. F$Ih inr'l
intrusion rletcctors. V
Conf. Knuwiedfe Discovery and Dnra Mitt-
ing,AAAI Press, Menlo Park, Calif., 1999,
p p 114-124.
Philip K. Chan is an assistant professor af coni-
puter scicnce at thc Florida Institute ofTcchnol-
ogy. His research interests include scalable adap~
tive mcthods, machine learning, data mining,
distributed and parallel computing, and inlelligent
rystems.HereceivedhisPhD,MS,nndBS incom-
putcr science fiom C ol umbi University, Vander-
bilt University, end Southwest Texas State Uni-
versity, respectively. Contact him at Computer
Sciencc, Florida Tech, Melbourne, FL 32901:
pkcC3cs.fit. ~ t l uwww.cs.fit.edd-pkc.
;
References
W ~ i F ~ " i ~ ~ P h D ~ ~ "incomputerscience
didate at
1.P.Chi~nanilS.Stoilo,"Met~leamingforMul~ Columbia University. His research interests are in
tistratcgy and Parallel Learning," Pmc. Sec- machiiielewiiing,detamining,dish.ibuted systcms,
ond Inl'l Workshop Mulrislrure~yLearning, infrirmntian retrieval, iind collaborative filtering.
Centcr for Aililicinl Intelligence, Gcorge Hclrceiveil hisMSciindB.BngfromTsingI,uaUni~
Mason Univ., Fairfnx, RI., 1993, pp, versity andMPhi1h.omColumhiaUiiiversity. Cnn-
150-165. tact him tit thc Computer Science Dept., Calumbin
Univ., Ncw York, N Y 10027; wlan@cs.columbiii.
2. S.Stollpetiil.,"JAM liivaAgentsfurMcta- -du:www.cs.columbii,.~d~l-wf~,,.
leiirning over Distributed Datnhases," Proc.
Third h l ' l Cmf.K,io,ubdge Uiscmery atid Andreas L. Prodromidis is il dircctoi in rcseeluch
Darn Mining, AAA1 Picss, Menlo Park, md developmentat iPrivacy.His research intcmts
Calif., 1997, pp. 74-81, include data mining, inachine learning, elrclraoic
mnmcrcc, and distributed systcms. Hc received a
3. D. Wolpen,"Stacked Gencraliriition," Neaml PhD in computer scicnce from Columbia Univw
N ~ m w r / k ~ , V 5o ,l .1992,pp. 241-259. ity and R Diploma i h elechical cngineaiingfrom thc
Ntitiunal Technical University of Athens. He is a
4. P. Chan and S . Stolfo, "Toward Scalable mcmbernf thclEEEandAAA1.Contact him at iPri-
Learning with Nonuniform Class and Cost vecy, 599 Lexington Ave., 82300, New York, NY
Distributions:A Ciise Study in Credit Card 1W22: andreas~iprivacy.com:www.cs.columbin.
Fraud Detection," I'roc. Fourlk Int'l Conf, :dul-n"dreils.
Knuwlcdp Dircovevy nnd Dora Minin,q,
AAA1 Prcss, Menlo Park, Calif., 1998, pp. ?alvatoreJ.Stolfu is a pmfcssor olcamputer sci-
164-168. mccat CulumbiaUniversity andcadirectorofthe
USCllSl and Columbia Centcr for Applied
5. A. Prodramidis andS. St,,lfo,"MiningData- Research i n Digital Govemmcnt Information S y s ~
bases with Different Schemns: Integrating .ems. His mmtreccnt research has focuscd on clis-
Incompatible Classifiecs," Proc. E'ourlh bill ributed data-mining systems with applications to
C w j : Knowledge Discovery atid Dma Mim rnud and intmsinn detection in network informa-
i n g , A A A l Pless,Mcnlo l'ark, Calif., 1998, ion systems. Hc receivcd his PhD lrom NYU's
pp.314-31x. 2oourant Institute. Contact lhim at the Compulcr
Sciencc Dcpt., Columbia Univ., New York, NY
6. 2. Ras,"Answering Nonstandard Qucrics in 10027: sal@'cs.columbia.edu:www.ca.coluntbia.
Distributed Knowledge-Based Systems.'' :dulsnl; www.cs.colunibh.cdui-saI.
IEEE INTELLIGENTSYSTEMS