(https://www.phdata.

io)

JANUARY 24, 2022

Why Do AI Initiatives Fail?

By Christina Bernard

Share    

Table of Contents 

1. You Don’t Have the Right Data Available

2. You Don’t Establish a Metric for Success

3. You Create a Solution with Excessive Complexity

4. You Don’t Prototype the Expected Outcome

5. You Don’t Have Budget for Unexpected Complexities

Best Practices for Setting up Data Science Projects

Maybe you’ve worked on your first few AI or ML projects

(https://www.phdata.io/machine-learning/) and encountered some difficulties.

Or maybe you’re just getting started on your first one. Whatever the case may be,
it’s important to recognize that – even though they can provide great value – AI
initiatives (https://www.phdata.io/blog/how-to-implement-ai-strategy/) require
a certain amount of diligence.

If you have already struggled, you should give yourself some grace, because even
Google Research has called machine learning “the high-interest credit card of
technical debt.”

AI initiatives fail for a plethora of reasons. But with a bit of foresight, failures can be
avoided. In this article, I’m going to outline some of the problems that I’ve seen and
hopefully lay out a path for you to avoid becoming part of a failed AI project.
1. You Don’t Have the Right Data Available
(https://www.phdata.io)
Have you ever tried to bake a cake without a pan? Or change a tire without one of
those turner thingies?

That’s what it’s like to be told to build an AI project (https://www.phdata.io/ai-

strategy/) without the right data. You need a solid foundation to build upon.
Oftentimes, everyone on the team is expecting that a certain piece of data is being
collected. But with all the data coming in, sometimes a key piece falls through the
cracks.

For instance, maybe a particular column is missing data or has a strange value
associated with it instead of the desired clarity needed to build a model. Data
Quality is a necessary component of deploying a successful ML model
(https://www.phdata.io/blog/the-ultimate-mlops-guide-how-to-deploy-ml-
models-to-production/). These techniques are usually set up on the data
engineering (https://www.phdata.io/data-engineering/) side of the projects.

Are there ways to make up for missing data?

Yup, but not all will work for every use case.

For example, if you know customers are leaving your platform but you’re not saving
the right behavioral data to your data warehouse, then we really can’t build a
model that predicts customer churn. Data scientists will try to hunt and find some
speck of meaning to feed the model but that’s not what builds the most successful
outcomes. We can try to mock-up certain metrics to fill the data in, but “your results
may vary.”

One way that may help you prevent this problem is to give them access to all of
your data and tell your data scientist to come back with a list of any missing data
they think is necessary. Sometimes, it’s actually there but being stored in an
unexpected place.

And other times you will have to hire a data engineer or API specialist to start
collecting the right data so that you can have a successful AI project
(https://www.phdata.io/blog/how-to-implement-ai-strategy/).

2. You Don’t Establish a Metric for Success

What’s sometimes more important than the data is your metric for success. This
doesn’t have to be a number. It can be an improvement on the distribution of a
piece of data or the curve of a graph. Looking at a certain percentage or number
isn’t always possible.

It’s a science.
(https://www.phdata.io)
There’s a lot of experimentation involved and we don’t know when the experiment
may have a successful end.

With that being said, if your team isn’t setting and sticking to a measure of success,
then your team is living on the wild side. You’ll never know if something was
achieved or not.

A metric for success removes a lot of ambiguity for data science projects. The DS
team knows what they are working toward and you can also hold them
accountable for doing more research and finding new ideas in that direction.

It’s easier to make a list of techniques to try and test in a single direction than to just
be thinking of all the techniques that have ever been created to solve the
ambiguous problem.

3. You Create a Solution with Excessive

Complexity
I’m sure you’ve heard the joke about, “how many people does it take to screw in a
lightbulb?” Well, the answer is one. And certain techniques are like using ten people
to screw in a lightbulb.

Take deep learning for example. It has many great use cases. However, in many
business use cases, it’s simply too much. It sounds fancy. The VCs love it and it
makes us look cool but it’s probably not the most effective way to solve a problem.

Not only that but it’s not easy to know why deep learning
(https://www.phdata.io/blog/deploying-an-emotion-classification-system-for-
videos-with-deep-learning/) algorithms make decisions. In other words, they can
be a bit of a black box. If you don’t want to work with a black box, or if you have to
comply with any form of regulation. A deep learning technique
(https://www.phdata.io/blog/techniques-for-labeling-data-in-machine-
learning/) may not be viable.

A data-driven operations, marketing, or sales strategy needs answers. Those

answers can be used to improve processes.
4. You Don’t Prototype the Expected Outcome
(https://www.phdata.io)
Prototypes are amazing! And you need one.

A prototype helps you to know what the expected output from the model needs to
look like. Oftentimes, we have one idea in our heads and the business has another.
By starting with a prototype’s expected output, we can merge these two ideas.

This can be a dashboard that needs to be viewed by managers or the sales team.
It can be how a feature will be displayed in an application. It can be how the
customer will interact with it on their end or simply how it needs to feed into the
email automation system.

None of these has to be fully built out, but the visual will create another opportunity
to ensure all stakeholders are on the same page.

5. You Don’t Have Budget for Unexpected

Complexities
Data Science is a Science. The process for putting models into a usable
environment is new. This means that you uncover so many new things along the
way to your solutions. Some of those discoveries could add value or insights, but
other ones will mean that you need to rethink your solution.

It is a good idea to communicate appropriate expectations, set a cautious budget

and timeline projections, and work in an agile manner to account for discoveries or
changes. Beyond that, you should include time and budget for testing, deployment,
and integration to your product. The length of each phase will vary based on the
size of your team and their expertise.

Best Practices for Setting up Data Science

Projects
1. Make it reproducible (https://www.phdata.io/blog/the-ultimate-mlops-
guide-how-to-deploy-ml-models-to-production/#tracking). As with any
science, others need to be able to recreate the work or the experiment isn’t valid.
Documenting the experiment and verifying the output statistically is a good
place to start. This is important to validate your model assumptions.

2. Set up Automation and Testing (https://www.phdata.io/blog/the-ultimate-

 2. Set up Automation and Testing (https://www.phdata.io/blog/the ultimate
mlops-guide-how-to-deploy-ml-models-to-production/#automation-and-
devops). While adding testing to Ml projects is new, it is possible. In recent years,
people have been modifying software engineering techniques to ensure that
(https://www.phdata.io)
notifications, alerts, and thresholds are set to not deploy bad models into
production (https://www.phdata.io/blog/beginners-guide-to-mlops-
deploying-ml-to-production/).
3. Data Availability and Validation are so important. Knowing where the data is,
how it’s being pulled, and setting up stops to ensure what is coming into the
model doesn’t break it.

Our sincere hope is that this blog provided some insights to help your AI initiatives
find success. If you have any questions or could use a little guidance, our team of AI
experts are happy to jump in and help.

We (humble brag alert) have helped numerous clients across many

verticals/business sizes find early success in their AI initiatives by leveraging our
vast expertise. If you’re ready to take the next step in your AI journey, don’t hesitate
to reach out!

More to explore
CONTACT US
(HTTPS://WWW.PHDATA.IO/CONTACT-US/)