How to Choose a Machine

Learning Platform to Detect

and Prevent Financial Crime
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Contents
9 Questions Financial Institutions Need to Ask a ML Vendor 04

Deployment: Cloud vs. On-Prem 18

Responsible AI: Ensure Your Platform Makes Fair Decisions 19

The FRAML Factor 22

A Guide to Seamless ML Implementation 23

2
Introduction
We get it. Banks and financial institutions (FIs) constantly face a tough balancing act. On the one
hand, your FI’s top priority is to deliver a top-notch customer experience. At the same time, you
can’t afford to let fraudsters blend in among legitimate customers and use your platform to
commit financial crimes - exposing your institution to financial losses, regulatory oversight, and
reputational harm.

At this point, you realize that machine learning is the key to both delivering seamless customer
experiences and stopping financial crime, including fraud and money laundering activities. So
where do you go from here? Especially when there are so many machine learning solutions to
choose from.

Choosing the right machine learning solution for your FI can feel intimidating. But the good
news is you don’t have to do it alone. This guide is designed to help make the machine learning
selection process easier for you by outlining the questions you should ask a vendor, red flags to
watch for, how to ensure the system makes fair and ethical decisions, and more.

Read on to learn how to make your machine learning selection as informed as possible.

3
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

9 Questions Financial Institutions

Need to Ask a ML Vendor
Realizing that you need to invest in machine learning solutions is just the first step of your
machine learning journey. It’s also the easiest. Figuring out how to choose the right ML solution
and vendor for your specific business goals is far more challenging.

If you’re not prepared, you’ll risk getting overcome with buzzword bingo sales approaches and find
your system doesn’t meet your needs. Here are nine questions you should ask a machine learning
vendor to help you avoid making a costly mistake.

4
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

1 Is the platform future-proof?

The AI space moves fast and evolves rapidly. Unfortunately, this means some technologies
currently being used today will become obsolete in the future. The last thing you want is to invest
money and resources into a system that will need to be replaced in three to five years.

It’s impossible to predict which technologies will remain in use and which ones will be phased
out eventually. Instead, you’re better off focusing on the flexibility of your machine learning
investment. Ask how well it responds to new scenarios. For example, maybe you’ll want to
introduce a new banking channel, like an upgraded mobile app or a virtual assistant, or take on
a new category of clients with unique banking needs. Ask how well the platform responds to
demands like this.

Bear in mind that your priorities will inevitably change. When that happens, you’ll need a robust
system in place that can easily pivot. As we saw during the pandemic, your customers could
also rapidly change their habits by shifting to digital banking channels in droves. A 2020 survey
found 46% of banking customers used online and mobile channels more often, while their use of
physical banking methods declined during the pandemic. If the solution can’t easily adjust to meet
new conditions, it could become a blocker for your business.

Make sure the system is flexible enough to accept new features and integrations. Gauge how
robust the solution is by asking whether it can be adjusted to take on heavier data volumes than
you had initially planned, how it avoids overfitting scenarios (in which it struggles to handle new
data sets), and if it relies on individual behaviors instead of generalized cohorts.

Red Flag

It comes with built-in limits

How to solve
The more successes your organization achieves with machine learning, the more use cases
you’ll want to add. That’s why it’s important to ask the vendor upfront about the range of
services available in the solution and the length of your contract. If you encounter limits in
the number of use cases you can add and transactions you can process, it might stall your
expansion plans until you can upgrade your service or your contract expires.

5
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

2 How much control do you have?

When you’re ready to introduce a new model or new rule, the last thing you want is to ask the
vendor’s permission. Agile models enable your teams to adapt to new fraud and financial crime
trends by easily introducing new models when you’re ready.
Ask whether the platform supports multiple
and custom models.

Can it support multiple models?

Multiple models are essential because fraudsters behave differently depending on where they are
and what they’re doing. A global model, for example, won’t reflect the nuances of fraud patterns
unique to a specific region. Platforms that can only run one model will use a generalized view
to look for specialized behaviors, which results in lower fraud detection rates and higher false
positives.

Can the platform keep pace with fraud as it evolves and be proactive rather than reactive?
A platform with multiple models must also score transactions at scale without compromising on
the response time by taking all the different input streams and translating them into one single
storyline or risk profile that a human reviewer can further evaluate.

6
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Can it support custom models?

In the rush to buy a machine learning platform, an organization runs the risk of purchasing a “one
size fits all” model, with point solutions for single-use cases. But every business is different, and
every model should be different, too.

Consider a platform with a flexible and agile architecture developed by data scientists with domain
expertise in specific industries. Custom models built with domain expertise in fraud science,
specifically for financial services, produce more power and more insight by considering the unique
fraud patterns that banking and payments face.

Consider a platform with a flexible

and agile architecture developed
by data scientists with domain
expertise in specific industries.

Red Flag

The vendor controls the data

How to solve
Your data is your most valuable asset. If the vendor can’t guarantee that you’ll have access
to your data when using their system, that’s another red flag. This will make it challenging to
understand how the vendor’s system is using your data. There is an exception to this rule.
Some vendors will allow you to access your data only if you agree to take ownership of the
system and take responsibility for any issues it encounters. Ideally, you’ll want to work with a
vendor who will allow you to access your data without making you responsible for the system.

7
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

3 Do you understand how the system arrived

at its risk score?

Machine learning systems that make decisions inside black boxes lack transparency around their
decisions, which creates two serious problems.

First, there’s a control problem, because humans can’t manage and improve a system they don’t
understand. Second, there’s a regulation problem, because an organization can’t audit or validate
the decisions that happen inside a black box and provide a reasonable explanation to their regulator
or governance teams.

Compare that to a platform that does whitebox processing to provide clear, human-understandable
reasons for its decisions. Certain machine learning platforms predict patterns using an algorithm
called Random Forest, which is made of tens of thousands of decision trees. A whitebox system
takes the few top-most factors from the decision trees, then weighs and communicates them to the
human in a simple way.

8
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime
Algorithm Pros
Random Forest • Generalizes patterns well
aka Ensemble • Robust to different input types
of Decision Trees (texts, numbers of scales, etc.)
• Robust to missing data
• Robust to outliers and errors
• Fast to train and score
• Trivially parallel
• Requires less tuning
• Probabilistic output
• Can adjust threshold to tradeoff
between precision and recall
• Very good predictive power
• Found to win many ML
competitions
Neural Network • Able to represent complex patterns
early generation • Good predictive power
from the 1980s
Deep Learning • Able to detect complex patterns in
vast amounts of data
• Offers multiple architectures for
different use cases
• Little/no feature engineering
needed
• State-of-the-art results in a wide
variety of machine learning tasks
Cons
• Can become complex to interpret
as number of decisions grow
(inherent nature of increased
capacity to make decisions)
• Requires labeled data
• Typically requires careful feature
engineering
• Cannot handle different input
types without preprocessing
• Needs scaling of inputs
• Can’t handle missing values
• Takes a long time to train
• Requires tuning
• Lack of interpretability
• Difficult to fine tune architecture
and hyperparameters
• Takes a long time to train
• Can be seen as black box model,
no explanations provided
depending on governance process
• May require specialized hardware
(e.g., GPUs)
9
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

The Anatomy of a Whitebox: Clear Answers

Here are some examples of how an easy-to-understand whitebox system explains its findings to
allow or block transactions.

Reasons to allow
Description Risk Risk Factor

IP address is only used by this customer 0.50% 0.9 x

IP country matches ATM country 0.50% 0.9 x

IP country matches customer’s home

and ATM countries
0.40% 0.7 x

IP country matches customer’s home,

ATM, and billing countries
0.30% 0.5 x

10
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Reasons to block
Description Risk Risk Factor

Customer is in distant locations in a short

period of time
60.0% 100.00 x

Customer is behind a proxy 6.0% 10 x

Customer has used two different IP

addresses
0.90% 1.5 x

Customer uses a free or disposable email

address
0.70% 1.1 x

The idea is to put humans in control by communicating clearly about

why certain transactions were approved or blocked and arming fraud
analysts with the information they need to review customer account
activity more intelligently.

Red Flag

It’s a black box system

How to solve
In order to trust the machine learning system’s outputs, you’ll need to understand how the
system arrived at its score. A black box solution won’t let you review how the score was
reached, see which data was used, or how it was tested. Only pursue this option if you’re willing
to fully trust the system’s score without accessing an explanation - and your model governance
audits will accept this outcome.

11
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

4 Does the system enable omnichannel data?

If the system can quickly transmit feedback
from the machine learning models back
into the model-building process,
you’ll be able to make adjustments
and improve the model much faster.
By accessing data across these
different customer touchpoints –
including mobile apps, websites,
ATMs, chatbots, and more, you’ll get
a 360-degree view of your customers’
behaviors and be able to respond
accordingly. A good model
can also easily pull in data
from both external
and internal sources.

Red Flag

Data sources are limited

How to solve
Ask how the model pulls in data from different sources, how the vendor’s system accesses
data from across your existing channels, and what steps are involved in accessing relevant
external sources, such as a government agency database. If the solution struggles to share
data internally, it is likely to struggle to access external data also. If a model can’t easily
access data from multiple sources, it could stall your efforts to add new features or launch
certain products.

12
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

5 Is it scalable?
As you scale your business, your machine learning system will need to keep up with your new
demands. For example, suppose your organization processes twice as many transactions as it did
when you first implemented your machine learning model. In that case, the model should be able
to handle its new workload with a few adjustments. The system should prove its resilience and
enable you to scale horizontally for each new business use case that you implement.

The system should prove its

resilience and enable you to scale
horizontally for each new business
use case that you implement.

Even if the model fails, it should fail gracefully. In other words, if a surge in volume causes the
model to crash, you should be able to recover the models and the data you were using easily.
The solution should enable you to rebuild the models you were using before the crash without
compromising the historical data used to build them.

Red Flag

‘It’s the only system you’ll ever need!’

How to solve
When it comes to AI, vendors can only share their insights about how the available technology
works at that moment. If you press them on how it responds to new developments, they should
present a list of options for adjusting it. On the other hand, a vendor who boasts their machine
learning system is the only one you’ll ever need is trying to make a quick sale and move on.
Trust the vendors who have thought about the future and research emerging use cases.

13
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

6 Is expertise
included in the
price?

Buying a machine learning system isn’t

a one-and-done transaction where you
give a vendor money in exchange for
their product and then part ways. It’s
a complicated investment, and you’ll
need more than just the technology to
make sure it meets your expectations
and helps you achieve your business
goals. You’ll also need access to
trusted experts in machine learning
and data science to help you make the
most of your investment - especially if
data science and machine learning are
new concepts for your organization.

Red Flag

It’s a sale, not a partnership

How to solve
Ideally, your arrangement with any machine learning vendor should be a partnership more
than a purchase. Ask the vendor if you will just get access to their technology or if your
arrangement includes access to expertise when you need consultation. Having a lifeline that
enables you to connect with the vendor’s team of experts is just as important as accessing
the machine learning technology. View vendors who aren’t willing to guarantee your
organization access to their expertise skeptically.

14
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

7 How specific does it get?

A platform reaches its decisions to create profiles of normal behavior and flag instances of
abnormal behavior. But some machine learning platforms can zoom in to a higher level of
resolution than others.

Does the platform create profiles of loose-fitting cohorts? For example, does it look at “women
between the ages of 30 and 35,” or does it look at “Susan”? Does it look at “devices in this zip
code,” or does it look at “Susan’s device”?

Red Flag

The platform isn’t narrow enough

How to solve
Delve into how the vendor’s platform builds profiles and establishes a baseline of “normal”
customer behavior. Watch for the specifics that the platform uses to build its profiles,
including whether it goes beyond broad information such as age brackets and activity based
on local zip codes. The more specific a platform can be in building a user’s profile, the more
fraud it can stop. If you get the sense a vendor uses data too broadly, it’s a red flag that could
lead to your organization losing money in fraud losses.

15
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

8 Does it offer flexible case management?

Organizations with 24/7 operations and globally-dispersed fraud and AML teams can have many
hundreds of people logging into the same system. A case management tool needs to have self-
configurable workflows so that everyone in this diverse user base sees the right actionable
insights at the right time.

Look for case management tools that:

 Automatically distribute work based on different criteria;

 Prioritize the most important transactions to review;

 Allows fraud teams, branch employees, and customer service departments to get exactly what
they respectively need from the same system;

 Prioritizes manual review queues and sorts in a way that allows for business continuity.
Otherwise, there will be gaps in your queue management that a fraudster can breach.

By creating profiles of many different entities (the branch, the day of the week, the credit card),
machine learning platforms can reach new sensitivity levels and detect fraud with greater power
and accuracy.

Red Flag

Specific workflows aren’t easily designated

How to solve
Ask the vendor how the platform distributes works based on the different departments
that will engage with the system and the platform’s decision-making process to prioritize
transactions. Consider the perspectives of your various teams (fraud analysts, customer
service, etc.) and how the platform works for them while they are doing their respective
jobs. If the vendor can’t speak to how the solution works for each department, that’s
another red flag.

16
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

9 Is it self-configurable?
In a world of gas-powered cars, machine learning systems are the new electric cars. And a platform
that’s self-configurable goes one step further. It means you’re not just buying the car. You’re buying
the car factory.

A platform with self-configurable dashboards lets you customize specific metrics to control your
command center and flexibility to choose relevant business KPIs.

Maybe you want to measure churn rates by department, or country, or partner.

Maybe you want to see churn rates based on team members,

business units, or product lines.

Typically, if you want to change your dashboard

reporting, you need to go back to the
vendor. But with self-configurable
dashboardsand reporting,
the control is yours.

Red Flag

The vendor controls dashboard configuration

How to solve
Just as you would with a car, ask the vendor for a test drive to see how much control you
have over the dashboard. Ask the vendor how long it takes to adjust the dashboard to your
liking. A red flag should go off if the vendor can’t guarantee that you’ll be able to configure it
to your organization’s specific needs at your convenience.

17
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime
Deployment: Cloud vs. On-Prem
Determine which method of deployment works best for your organization. There is no one-
size-fits-all approach. It’s important to consider all the factors that impact how your organization
utilizes its ML investment.
Vendor Cloud-Hosted
Deployments Considerations
• Vendor assumes responsibility for operation and
maintenance of the platform.
• No hidden costs in licensing agreement.
• FIs have transparency into the platform’s
operational costs and how it can scale.
• No platform operation and maintenance.
• Vendor manages troubleshooting and
remediation.
Tip
Perform due diligence to ensure the vendor
meets industry standards for secure handling
of sensitive data, including PCI DSS, ISO 27001,
SOC2, and more.
On-Premise Deployment
Considerations
• FI assumes operational and maintenance
responsibilities, including procuring hardware,
licenses, and QA governance inclusion of the
system into existing frameworks.
• Platform sits on the FI’s existing infrastructure.
• Hidden/unanticipated operational costs can
contribute to long-term spending (maintenance,
scalability).
• Ongoing issue troubleshooting and remediation
is less effective due to environment access and
privilege restrictions applied.
• Licensing agreement may limit FI’s scalability
agenda.
Tip
Make sure your TCO and/or ROI reflects the
quantitative and qualitative cost items. An
adequate support model is critical for the
success of your Risk Management program.
18
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Responsible AI: Ensure Your Platform

Makes Fair Decisions
Too often, financial institutions overlook a critical factor when implementing a machine learning
platform into their organization: fair risk decisions. A lack of clear guidelines and governance on
addressing AI Fairness, Accountability, Transparency, and Ethics (FATE) confuses FIs worldwide.
A 2018 study, for example, found that while FinTech lenders charge minority borrowers less than
traditional FIs, these groups often pay extra mortgage interest. Other major financial firms faced
accusations of racial discrimination in their lending practices.

That’s why it’s important to consider ethical AI questions at the beginning of your machine
learning journey.

All machine learning models are impacted by bias eventually. No matter how careful you are,
human beings run your organization which means human biases will eventually infiltrate your
models. One large Tier 1 FI learned this lesson the hard way when a viral Tweet revealed that the
credit card limits for female customers were significantly lower than those of male customers. And
that’s just one example.

If left unchecked, machine learning biases can:

Discriminate against certain groups of customers;

Alienate both your existing customers and prospective ones and contribute to abandonment;

Cause significant damage to your FI’s public reputation;

Leave your FI vulnerable to lawsuits and unwanted scrutiny from regulators.

Establishing fairness goals at the beginning is a much more cost-effective way to adress any
ethical AI issues that you encounter. You can develop machine learning models that reflect your
organization’s values and make it easier to ensure your AI behaves ethically by taking the following
steps.

19
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime
Step 1
Audit your models for bias
Your models will need to be adjusted to
respond to new financial crime threats -
which is where biases can unintentionally get
introduced. Performing regular reviews of
your models will ensure they are not delivering
unfair decisions for specific customer groups.
These audits can also mitigate the risk of
your models getting reviewed by a regulatory
agency. Some regulators are already asking
to see reports on how models reached their
decisions. Bias auditing can be included with
these reports.
Step 2
Get access to model
explanations
Several stakeholders will need to understand
how models arrive at decisions. This includes
internal teams like data scientists and fraud
analysts who need these explanations to
understand how the model performed, do their
jobs more effectively, and meet your business
goals. Access to clear model explanations can
save your teams’ time and save your organization
money. A vendor that only offers a black box
solution can undercut your teams’ efficiency.
20
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Step 3 Step 4

Invest in high-quality Focus on ‘Fairness-Awareness’

explanations
You need more than just access to model
explanations. Vendors should evaluate the
explanations to assess whether they provide
your organization with insights into making
more informed and effective decisions. Ideally,
the explanations should clearly outline why one
model performs more effectively than another.
Without access to high-quality explanations,
your FI could struggle to explain decisions to
stakeholders, customers, and regulators.
Fairer decisions should not be a trade-off
for effective fraud detection. Vendors that
prioritize FATE considerations from the start
are more likely to develop models that don’t
compromise on either objective. If you only
consider a model’s fairness after it goes into
production, you’ll have to choose between a
new model that either makes unfair decisions
or detects fraud less effectively – and you could
be forced to explain your model selection to
regulators.

Please note that there’s no such thing as a one-time fix for model fairness.
Even if your team addresses a specific, known bias, new biases will eventually emerge
that will require a similar response. Prioritizing model fairness at the start of your machine
learning journey is the most effective way to head off any ethical issues that arise.
Otherwise, you will have to redirect valuable time and resources into addressing bias
problems when they are discovered. Addressing ethical AI issues at this stage can take
several weeks and result in lost productivity for your team.

Making AI ethics a priority can save your organization valuable

time and money. Ensuring your models align with your
organization’s values is a critical step toward ensuring your
machine learning models are productive and cost-efficient.

21
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

The FRAML
Factor
A significantly changing payments
landscape calls for an equally
significant change in tactics.
FRAML aligns fraud prevention
and anti-money laundering (AML)
solutions and provides FIs with a
more nimble approach to fraud
prevention, lower total cost of
ownership (TCO), and enhanced
AML compliance.

Here’s why your FI should consider investing in FRAML:

Eliminate Upgrade
data silos operations

Converge data from traditionally siloed areas Get data from across an organization,
(such as transaction fraud and AML) to more empowering teams to make smarter, faster
accurately assess risk. decisions.

Improve customer Improve

relationships AI performance

Get a complete view of customer activities to Shift from investigating fraud after an event
improve customer experiences and remove to fraud prevention and identify performance
unnecessary friction. gaps.

22
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

A Guide to Seamless ML
Implementation
Implementing a machine learning platform at your FI requires careful planning and preparation.
If you’re not ready for the changes that machine learning will introduce to your operations, it
could compromise the platform’s effectiveness and delay your ROI. Here’s what you need to
ensure your shift to machine learning goes as smoothly as possible.

Prepare for big changes

If your FI relies on a legacy banking system to detect fraud, implementing a machine
learning platform will feel like a culture shock at first. That’s because legacy systems
depend on rules to stop fraud and are often isolated from other data sources.

When your machine learning platform is up and running, don’t be surprised if your
existing rules do not generate the same results. Machine learning platforms review
numerous new data points to deliver more accurate assessments over whether
a transaction carries a risk of fraud. Remember, machine learning technology is
designed to help you find more fraud than your existing rules-based system can find
on its own. If fraud detection rates suddenly jump, that’s a sign that the machine
learning platform is working correctly.

Get a handle on your data

Data is the most valuable asset to improve fraud detection and customer
experiences. The more clearly labeled your historical data is, the more effective your
new machine learning platform will be moving models from “shadow mode” - an
offline environment where models can access and test live transaction data - into
production.

It’s critical to label your data clearly. Consider the difference between training a child to
ride a bicycle on pavement versus an unpaved, mountain road. If you teach the child
to only ride on smooth pavement (like in the training model), they will fall over once
they ride on a dirt road (the live environment with unforeseen circumstances). Using
clearly labeled data in shadow mode can help your model prepare and adjust to new
circumstances when it goes live.

23
How to Choose a Machine Learning Platform
to Detect and Prevent Financial Crime

Look for a partnership, not a sale

Thinking of the machine learning vendor as a partner who is equally invested in
the success of your business can significantly help your organization’s ROI. Talk
to the vendor about the business problems you would like to solve - as opposed
to a specific technical problem - so that they take the time to understand your
operational and machine learning goals. This enables the vendor to understand if
their platform is indeed working as intended.

A good partnership is a two-way street. While your organization gains technology

that enhances your fraud prevention capabilities and delivers ROI, your vendor can
use your transactional data to enhance their technology and enrich their data. A
system capable of connecting to online and offline environments allows the vendor
to access a feedback loop, resulting in quick upgrades and system improvements.

Designate ownership of the platform

The business goals for implementing a machine learning platform will eventually
shift as your priorities change. It’s important to designate someone on your
internal team who will take full ownership of your machine learning investment at
all stages. This includes the initial request for proposal (RFP), procurement stages,
and implementation. Appointing this individual from the beginning will help ensure
your organization can adapt to new circumstances at all stages of the process. This
person will later consider how the platform can be used for new business goals

Think flexibly
While machine learning platforms can significantly enhance fraud detection, it’s
important to remember this technology isn’t a silver bullet for financial services. Your
FI will still experience false positives even after implementing a machine learning
platform, but at a lower rate than before. The only way to stop false positives entirely
is to block every transaction on your system, which would harm your business.
Instead, it’s better to find a happy medium where your false positive rate declines
while your system continues to catch more fraud.

24
One Cloud Platform to
Manage Financial Crime
Every year, Feedzai’s risk management platform scores trillions of dollars of
transactions to protect the world’s largest companies. Fully AI-enabled to stay
ahead of emerging financial crime and money laundering patterns, Feedzai
mitigates even the most deceptive criminals so that banks, issuers, acquirers,
and merchants can focus on growth.

Account Opening | Anti-Money Laundering | Transaction Fraud

feedzai.com info@feedzai.com sales@feedzai.com