Artificial Intelligence and Data Analytics (AIDA) Guidebook

resulting decisions can degrade with time due to data drifts and organizational overconfidence
in the model. Machine learning is not a one-and-done process, creating an algorithm that is
infallible for all time, but an ongoing and indeed constant evolution, where the AI algorithms
repeatedly encounter new data and modify themselves to account for it. To counter this,
organizations use continuous integration (i.e., merging code changes into a central repository),
continuous deployment (i.e., using automated testing to validate if changes to a codebase are
correct and stable), continuous training (i.e., testing of the model’s validity), and a human
element in the development loop.

Conducting these additional steps is what differentiates DevOps from MLOps, democratizing
and streamlining the analytics process. On the technical side, MLOps bypasses the bottlenecks
in the deployment process, i.e., between machine learning design and implementation or
deployment framework. Strategically, MLOps makes machine learning accessible to those with
less data and coding expertise. Additionally, an organization may benefit by exposing the
quantitative rigor to qualitative subject matter, and by combining strategy and tactics to work
together. This is important since only 13% of machine learning projects10 make it into
production due to a lack of organizational engagement.
There are risks to MLOps in addition to the benefits stated above. MLOps may oversimplify the
development process, cloaking intermediate steps, which may pose a challenge to those with
less data and coding expertise. This may lead to downstream impacts if the code & data fall out
of alignment. Developers often weigh the risks and rewards of MLOps, asking questions such as:

• How much additional infrastructure is required to make MLOps sustainable?

• Does the organization already have a substantial infrastructure?
• How to measure the increase in productivity vs. the increase in risk?

Each organization will then identify acceptable risk level when determining how to proceed.
Additionally, organizations must often consider how tightly they link operational SMEs and data
or modeling SMEs; if model accuracy monitoring includes ethical metrics (race, gender, etc.);
and maintaining an organizational culture of respect for all contributors’ expertise.
Infrastructure SMEs manage the CI/CD technical side, working closely with other partners on
CT. Data SMEs understand operational SMEs as well as manage the CI/CD data side and work
closely with other partners on CT. Operational SME coordinates with data SMEs and are
responsible for proactively engaging data and infrastructure SMEs on CT.

10
https://venturebeat.com/2019/07/19/why-do-87-of-data-science-projects-never-make-it-into-production/

Page 17