07/05/2020 How to QA Test Software That Uses AI and Machine Learning - DZone AI

 

LIVE WEBINAR - Thursday, May 14th: Build Your Own CDN in Under 1 Hour
Sign Up Now 
DZone > AI Zone > How to QA Test Software That Uses AI and Machine Learning

How to QA Test Software That Uses AI and

Machine Learning
by Vakul Gotra · Jul. 12, 19 · AI Zone · Opinion

Smartphones, smart speakers, smart cars, smart coffee makers...the list goes on. It seems like
everything around us is coming to life and becoming intelligent. And though the sci- i genre thrives on
our ever-present fear of a hostile robot takeover, smart devices are anything but dystopian — they’re
actually here to make our lives easier so we can spend more time on the important stuff instead of
tedious busywork.

Tech companies know that increased automation is the way of the future, just like it was when Ford
pioneered the assembly line. Advanced technology like arti icial intelligence (AI) and machine learning
(ML) is fueling the most exciting innovations in recent history — think self-driving cars, virtual and
augmented reality, automated investing, improved medical imaging, and more. The bene its of this
technology are becoming more and more obvious, and companies are rushing toward adoption and
racing to build it into their products.

As this technology begins to become more commonplace in sensitive, high-stakes spaces (like the auto,
medical, and inance industry), it’s vital that DevOps teams take a strong approach to QA testing. When
public safety, a customer’s livelihood, or a patient’s data is at risk, even the smartest algorithm must be
checked and checked again by a human engineer.

Before diving into the approach for testing smart products, let’s differentiate between arti icial
intelligence and machine learning. Though the terms are often used interchangeably, there are some
key differences.

Artificial Intelligence Machine Learning

Applies knowledge or skills Gains knowledge or skills over time

Prioritizes success over accuracy Prioritizes accuracy over success

Simulates natural intelligence Learns continuously from a set of data

Simulates human responses to problems Creates continuously learning algorithms

Searches for the optimal solution Searches for any solution, whether optimal or not

Put simply, arti icial intelligence refers to a system that performs tasks in a way that we humans might
https://dzone.com/articles/how-to-qa-test-software-that-uses-ai-and-machine-l 1/3
07/05/2020 How to QA Test Software That Uses AI and Machine Learning - DZone AI

consider smart or ef icient, while machine learning is automated, continuous self-training performed
by a system that leverages pre-existing data.

Testing artificially intelligent systems

Challenges and potential complications
Huge volumes of collected data present storage and analytics challenges — scrubbing this amount
of data can be incredibly time-consuming
Data may be collected during unanticipated events or circumstances, making it dif icult to gather
and use for training purposes
Human bias may appear in training and testing data sets
Defects quickly fester and grow more complex in AI systems

Key aspects of testing

Data validation
The key to successful AI is good data. Before it’s supplied to an AI system, your data should be
scrubbed, cleaned, and validated. Your QA team should be wary of human bias and variety that can
complicate the system’s interpretation of the data — think of a car navigation system or smartphone
assistant trying to interpret a rare accent.

Principle algorithms
At the heart of AI is the algorithm, which processes data and generates insights. Some common
algorithms relate to learnability (the ability of Net lix or Amazon to learn customer preferences and
serve new recommendations), voice recognition (smart speakers), and real-world sensor detection
(self-driving cars).

These should be tested thoroughly with model validation, successful learnability, algorithm
effectiveness, and core understanding in mind. If there’s an issue with the algorithm, there are sure to
be more serious consequences down the road.

Performance and security testing

Just like any other software platform, AI systems require intensive performance and security testing,
along with regulatory compliance testing. Without proper testing, niche security breaches (using voice
recordings to fool voice recognition software or chatbot manipulation) will become more common.

Systems integration testing

AI systems are built to hook into other systems and solve problems in a much larger context. For all of
these integrations to work correctly, it’s necessary to perform a complete assessment of the AI system
and its various connection points. With more and more systems absorbing AI characteristics, it’s vital
that they’re tested carefully.

Testing machine learning systems

The goal of ML systems is to acquire knowledge on their own, without being explicitly programmed.
This requires a consistent stream of data to be fed into the system — a much more dynamic approach
th t t diti l t ti i b d (i di t i d t t) A
https://dzone.com/articles/how-to-qa-test-software-that-uses-ai-and-machine-l
di l QA t ill dt 2/3
07/05/2020 How to QA Test Software That Uses AI and Machine Learning - DZone AI
that traditional testing is based on ( ixed input = ixed output). Accordingly, QA experts will need to
think differently about implementing test strategies for ML systems.

Training data and testing data

Training data is the set of data that is used to train the model for the system. In this data set, the input
data is supplied along with the anticipated output. This is typically prepared by collecting data in a
semi-automated way.

Testing data is a subset of the training data, logically built to test all the possible combinations and
determine how well your model is trained. Based on the results of the test data set, the model will be
ine-tuned.

Model validation
Test suites should be created to validate the system’s model. The principal algorithm analyzes all of the
data provided, looks for speci ic patterns, and uses the results to develop optimal parameters for
creating the model. From there, it is re ined as the number of iterations and the richness of the data
increases.

Communicating test results

QA engineers are used to expressing the results of testing in terms of quality, such as defect leakage or
the severity of defects. But the validation of models based on machine algorithms will produce
approximations—not exact results. The engineers and stakeholders will need to determine the
acceptable level of assurance, within a certain range for each outcome.

Like This Article? Read More From DZone

DZone Article DZone Article
QA: Why Machine Learning Systems Towards Auto-Tuning Systems
Are Non-Testable

DZone Article Free DZone Refcard

The AI Impact on Software Testing Introduction to TensorFlow

Topics: ARTIFICIAL INTELLIGENCE , ARTIFICIALLY INTELLIGENT SYSTEMS , MACHINE LEARNING , QA TESING , SOFTWARE TESTING

Published at DZone with permission of Vakul Gotra . See the original article here. 
Opinions expressed by DZone contributors are their own.

https://dzone.com/articles/how-to-qa-test-software-that-uses-ai-and-machine-l 3/3