A Guide to Solving the Toughest

Mobile Testing Challenges

How to address the most
complex use cases with
real-world examples.
The features and capabilities available to today’s mobile
application developers are seemingly endless. From biometric
authentication to scanning bar codes on your own, the application
development experience is growing vaster to keep up with the
needs of today’s consumer.
As application development capabilities keep growing, the needs
for more comprehensive application testing are growing as well.
2 | www.perfecto.io
While testing these features manually is always an option, teams
cannot rely on manual testing as a long-term, scalable solution.
Testers need to test these capabilities quickly, efficiently, and with
the rest of their application.
This eBook dives into some of the toughest test automation
challenges that testers face today and will show you how teams
can use Perfecto to address each of these challenges head-on.
By integrating with and expanding the capabilities of popular
open-source frameworks, you will see how you can leverage
Perfecto to automate some of the most complex use cases.
Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0820KS22)
 Contents

Biometric Authentication ......................................... 4

Image Injection....................................................... 5

Network Virtualization............................................. 5

Voice Recognition................................................... 8

Geolocation..........................................................10

Two-Factor Authentication...................................... 11

Quality Visibility.....................................................13

Mobile UX Measurements.......................................13

Test Coverage........................................................14

Accessibility .........................................................14

Conclusion............................................................15

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

3 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
Biometric Authentication First, Quantum already provides biometric validation step
definitions. Therefore, you simply add a step to validate a successful
Biometric authentication is becoming increasingly essential to
(or failed) Face ID, Touch ID, or fingerprint validation in your test
providing a high-quality mobile application user experience.
scenario as part of the feature file.

By allowing customers to sign into their apps or authenticate their

transactions by validating their biometric data, leading banks and
other enterprises are providing mobile experiences that are both
convenient and secure.

It should therefore be no surprise that biometric validation will

Second, most IDEs let you develop Appium tests in the same way.
become increasingly important to mobile application testing as
Adding step definitions from Quantum into your Appium tests is all
well. Enterprises should expect an increase in scenarios that will
you need to do to authenticate biometrics on either iOS or Android.
involve not just one, but two forms of biometric validation. Teams
The test will automatically know whether to validate Face ID, Touch
will need to test these biometric validations at scale, both manually
ID, or fingerprint depending on which device you are using to
and in automation, to achieve a high level of CI/CD.
execute your test. When you run these tests on Perfecto, which
uniquely allows teams to test both successful and failed validation
For true end-to-end test coverage, it is essential to test real journeys
of Face ID, Touch ID and fingerprint on real iOS and Android
on real mobile devices. Therefore, you need to be able to validate
devices, adding biometric authentication to your automated test
biometrics on real devices as well to provide a high-quality mobile
suite is quite a simple process.
user experience.

THE NEED FOR INSTRUMENTATION

With Perfecto, testing fingerprints and other types of biometric
authentication is possible with our integrations with Appium and To override the application’s biometric sensor API and allow

Quantum, an open-source BDD framework that already has built-in Perfecto to inject the successful or failed biometric authentication,

Perfecto capabilities, including biometric authentication. users must first prepare their application with “instrumentation,”

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

4 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
a process that allows the testing framework to gain access to
parts of the application.
In the case of biometric authentication, users will want to
instrument for the sensors (both the biometric sensor and the
camera for image injection). This instrumentation will enable
Perfecto to override these sensors and allow access to the web
view for hybrid applications.
Image Injection
Mobile application capabilities that require camera use are
also continuing to increase, from barcode readers to bank
check scanners. Image injection helps add cases that test these
functions into your automated testing suite.
Image injection takes an image (in either .jpeg, .jpg, .png, or .bmp
format) that is stored in the Perfecto repository and presents it to
the application as if it was read by the device camera. This allows
you to test different ways that application engages with the camera,
such as different types of focus and image placement.
Like biometric authentication, image injection requires that you
install the application with sensor instrumentation. Yet in this case,
you will need to add a target image from the Perfecto Repository
that you would like to inject into the application in order to test.
5 | www.perfecto.io
Teams can also address the challenge of testing image injection
with Quantum, which offers built-in steps that allow you to start
and stop the image injection.
This capability makes the complex issue of testing how an
image is injected into an application easy to automate and
simple to execute.
Network Virtualization
All mobile application developers and testers experience the same
challenge: ensuring the application will respond gracefully and
as expected no matter where their mobile applications are being
used and under which conditions. Such conditions include airplane
mode, no network connectivity, poor network connection and 2G,
3G or 4G network speeds.
If a user is using a banking application and goes into an elevator or
a subway, does your application handle that scenario gracefully?
By testing real end user conditions, teams can ensure a high-quality
mobile application user experience, advanced test automation
coverage (and ensuring they will test the application before their
users do).
With Perfecto and Appium, you can simulate all the network
scenarios we have listed above and more. You can use the out-of-
Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0820KS22)
the box-network user profiles to simulate cellular network speeds. NETWORK VIRTUALIZATION, SIMPLIFIED
You also have the flexibility to create your own custom network
When it comes to embedding network virtualization functionality
conditions by manipulating metrics like latency, packet loss and
using real mobile devices, Perfecto provides a testing platform with
bandwidth in and out. We make it easy to simulate the cellular
advanced network simulation functionality built on top of open-
networks with user profiles, and if you need custom control,
source Appium. Perfecto provides an easy to call method that can
you have easy access to that as well.
be added to your Appium scripts in about 5 to 10 lines of code,
depending on what you are trying to do.
NETWORK VIRTUALIZATION TESTING WITH
PERFECTO AND APPIUM: CAPABILITIES Below we call the method in “startNetworkVirtualization” and send

Perfecto provides advanced network virtualization on top of open the necessary parameters to the method that implements the actual

source Appium. Network virtualization plus additional Perfecto lines of code shown in a later snapshot.

capabilities enable you to:

1. Disable Wi-Fi or enable airplane mode.

2. Simulate network conditions using

predefined profiles (2G, 3G, 4G).

3. Simulate network conditions using a custom

network profile (e.g., latency (ms), packet loss (%),
Bandwidth in and out (kbps)).

4. Get the current network settings.

5. Update network conditions or reset the device

to default network configuration.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

6 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
PERFECTO USER PROFILES • bandwidth.in/bandwidth.out – Limitation of download and
upload network bandwidth from the device
Perfecto makes testing network conditions easy by providing you
with a number of user profile network conditions already packaged • delayJitter – Random latency variation

for you and ready to use. These enable easily simulating different
cell network speeds from 2G to 4G. After the test has been executed, you will be able to see each line
of code and corresponding action at the test step level within the

Here is a small sample of User Profiles: Perfecto UI. You also have a rich set of artifacts to select from for
testing, such as video recordings, Snapshots, Device Vitals, PCAP
Network to use Latency (ms) Bandwidth-in Bandwidth-out
files, HAR files, Crash logs and other relevant data like page load
2g_gprs_poor 700 30/20 2
time duration. Testers can quickly find the root cause of any issues
3g_umts_average 140 300/50 0 they are investigating with all this data. You can share the data

3.5g_hspa_poor 190 900/20 1 amongst your teammates so they can reproduce any issues quickly.

4g_lte_poor 120 7000/3000 1

4g_lte_ave4rage 100 10000/4400 0

NETWORK CONDITION CUSTOMIZATION

You can also customize your network conditions. If you want
to customize your own network speeds and other variables for
advanced user scenarios, you have that flexibility. It is as easy as
providing the parameter and the value you want the parameter to
simulate. Some of the more popular parameters include:
• packetLoss – Network packet loss

• latency – Latency applied on packets in the Network

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

7 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
Voice Recognition While voice assistants bring opportunities to developers, they pose
new challenges for QA teams as they do their mobile testing. This
Many industries have recently started incorporating some form of
section will give a detailed overview of Perfecto’s support of test
voice assistant or voice recognition system as part of their mobile
cases for voice recognition systems.
apps. Banking is one major industry that has made strides in this
area, adopting virtual assistants to help customers save time on
4 STEPS TO TESTING VOICE RECOGNITION
basic banking tasks.
Automating voice assistant flows as part of your end-to-end

Retailers and grocery chains are hopping on the voice recognition testing can be achieved, in part, with an open-source automation

bandwagon as well, incorporating major platforms such as Siri, framework such as Appium.

Alexa, and Google Voice as another way to interact with customers.

There are four general steps to running automated tests for voice
recognition software:
WORLDWIDE VOICE ASSISTANTS IN USE
1. Screen navigation to voice assistant (supported by Appium).

8.0B 2. Activate voice assistant (supported by Appium).

3. Say a voice command (requires advanced automation).

6.4B
4. Validate screen/response (requires advanced automation).
Units, www.t4.ai
5.1B
4.1B While automated tests can handle the first two steps, the last
3.3B
two are challenging to perform as part of an automated script.
Traditional libraries such as Appium do not support audio out
of the box.

2019 2020 2021 2022 2023

EXAMPLE: TESTING GOOGLE VOICE
WITH PERFECTO
In the following example, you’ll see how you can automate a simple
end-to-end flow that includes Google Voice with Perfecto.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

8 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
There are three basic steps. Once you are in Object Spy, you can locate all elements on the
1. Open a device. screen, including system level objects like the Google Voice icon.

2. Click on Google Voice.

3. Inject a pre-recorded audio file.

For this sample script, we will be writing our code in Java

and testNG.

First, you open a device in the Perfecto lab. This example shows
a user selecting a Samsung Galaxy S8.
Then you can incorporate these locators as part of your automation
script. Here is a sample snippet:

Next, use Perfecto’s “Object Spy,” which shows the application’s

object properties, to access all the element locators.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

9 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
Note that you can also upload your simple audio file to
the Perfecto repository.
You are now ready to execute your automated test in testNG and
view your results in the Perfecto dashboard.
When breaking down the test steps you can see exactly where the
audio was injected and that the step passed successfully.
10 | www.perfecto.io
Executing automated tests for voice recognition systems no longer
needs to be a daunting task for QA. With Perfecto, teams can
expand their test automation horizons with these additional audio
capabilities at their fingertips.
Geolocation
Geolocation testing is the testing of a web or mobile
application from IPs of various geographical locations.
Some of the use cases include checking notifications,
performance, notifications, and more.
GPS navigation apps are not the only apps using geolocation.
Today, routes, locations, and the estimated time of arrival are
implemented into a variety of apps including rideshare, food
delivery, e-commerce, games, social media, and more.
An app may load successfully from one location, but not as
expected when the user moves to another location. In addition,
an app will present a contextual pop-up message based on the
user’s location. As a result, it is critical to run through all the
possible test scenarios for every user with geolocation testing.
Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0820KS22)
HOW TO AUTOMATE GEOLOCATION TESTING The most efficient way to perform geolocation testing is to
automate the testing with a tool like Appium. Users can simulate
Geolocation testing is easy with a commercial cloud solution like
web and mobile app behavior from various locations using
Perfecto. Simply access the device you need and simulate your user
Appium's IP geolocation feature, which uses IP addresses from
behavior by entering the location you wish to test.
various countries.

By adding IP geolocation into your web and mobile tests, you can
By adding IP geolocation into your tests, you can validate
validate scenarios like languages, currencies, time zones, and
scenarios like languages, currencies, time zones, and more.
more. These automated tests may be coupled with other testing
These automated tests may be coupled with other testing
paradigms like end-to-end regression to mimic a complete test
paradigms like end-to-end regression to mimic a complete
environment in a single location.
test environment in a single location.

Two-Factor Authentication
Two-factor authentication (2FA) is a commonplace way to
strengthen app security by requiring two means to authenticate
your identity. We see 2FA in our emails, SMS text messages,
banking apps, and many other platforms.

The first layer of 2FA is usually the username/password, fingerprint,

or Face ID a user provides to authenticate their account and move
onto step two. The second layer of authentication is added on top
of the first. The second layer may be an email with a confirmation
link, SMS containing One-Time Password (OTP), or an installed app.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

11 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
2FA protects logins against phishing, social engineering, and
password brute-force attacks. It also prevents logins from attackers
who can exploit weak or stolen credentials. 2FA authentication
testing use cases include:
• User registration

• Device authentication

• Password reset

• Banking transactions

If 2FA is a vital feature that is well on its way to becoming a

necessity, then testing 2FA must be a priority. Automating your
2FA testing is an important step toward improving and securing
your online and mobile apps.

Perfecto’s unified platform experience ensures all your test

scenarios are in one place and helps synchronize your QA workflow
across applications. Like other testing challenges discussed in this
eBook, testing 2FA is done by expanding the capabilities found in
open-source Appium. While standard Appium APIs do not allow
you to access third-party applications, this is possible with Perfecto.
Below is an example using Perfecto’s APIs to implement the next set
of steps.
• Launching an SMS application.

• Retrieving the “code.”

• Launching an application of choice (in this case – Salesforce1).

• Entering the retrieved “code” for verification.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

12 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)
Quality Visibility
DevOps enforces fast releases of new value to customers that need
to be validated quickly. Validating software quality quickly and
continuously can only be done via smart test automation.
Depending on the type of app, there will be different quality
constraints and considerations. These can be tools, coverage
needs, market events, and more. You cannot assess and measure
what you cannot see, which is why quality visibility is vital to teams.
Defining proper KPIs around development productivity and quality
and constantly reviewing them makes the difference between
standing still and continuously growing the organization’s maturity.
Management needs to know at any given time how the teams
perform, how the apps look, how healthy the CI pipeline is, and
many other insights. It’s clear that when implementing continuous
testing in DevOps, the level of test data constantly grows, therefore
knowing how to slice and dice the data and make sense out of
the data is a critical aspect in the overall strategy. Understanding
product areas that are buggier than others, platforms that are
buggier than others, and test scenarios that are flakier than others
can help in establishing continuous testing that really works for
everyone. Without test data analytics and governance, the data
only piles up and becomes a bottleneck in the entire software
release cycle.
13 | www.perfecto.io
Mobile UX Measurements
Mobile applications are integral in today’s world, and the mobile
app market is projected to generate more than 613 billion U.S.
dollars in revenues in 2025. Users have high expectations for
mobile apps—they expect them to function perfectly, without delay
or interruption.
Users have higher expectations for mobile apps than ever,
therefore you must invest in designing positive experiences on
mobile devices and test for such.
Unless you perform load testing against your mobile UX, you can’t
just assume that your apps will provide the same, excellent user
experience across extreme high vs. low traffic periods. Without
proper load testing, you risk exposing your customers to sudden
app crashes and slow performance. And you can guarantee that
you’ll see a rise in support tickets and poor mobile app ratings
rolling in.
Do not wait for your mobile users to have these negative
experiences. You must incorporate load in your overall mobile UX
testing so you can simulate real user scenarios and ensure your
apps work well under all types of traffic conditions.
Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0820KS22)
With Perfecto, you can generate detailed server and end-user
experience reports. This helps teams measure and understand
how a mobile app is impacted under a load.
Test Coverage
It is important to continuously update your test coverage as devices
progress and proliferate. Make strategic decisions about test
coverage based on your users, their location, device trends, and
your comfort with risk.
There are so many versions of mobile operating systems, devices,
and browsers. And new ones are released all the time. Plus,
foldable phones are on the horizon and new features are coming
in Android. You’ll need to ensure proper test coverage across these
in order to deliver a better mobile UX across all these devices.
Using a mix of real devices, emulators, and simulators in a test
lab can help you ensure that you have your bases covered.
Accessibility
You need accessibility testing to meet the needs of all users.
It’s also the law. Between the Web Content Accessibility Guidelines
(WCAG), Section 508, and the Americans With Disabilities Act
(ADA), you have plenty of regulations to meet. Failure to do so
could result in fines of $75,000-$150,000.
14 | www.perfecto.io
With Perfecto, you can automate a large portion of accessibility
tests for mobile and reduce the overall time it takes to run them.
In addition, Perfecto will generate an accessibility test report that
can help your developers find and fix accessibility issues
in advance.
You need to ensure that your apps work with screen readers,
speech recognition software, screen magnification, and more.
Perfecto can help you eliminate manual non functional testing
and accelerate automated accessibility testing.
You need to test:
• Labels — Used by assistive technologies,
like VoiceOver or TalkBack.
• Text contrast — Ratio between text or images
and background color.
• Hit area size — Area designated for user interaction.
• View hierarchy of UI — Determines how easy the
Android app is to navigate.
• Dynamic font size — Option for users to increase
font size to fit their needs.
Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered
trademarks are the property of their respective owners. (0820KS22)
 About Perfecto
Most accessibility tests are done manually or not at all.
Perfecto powers exceptional digital experiences by combining
Many teams outsource accessibility testing to third party services.
the power of flexible test authoring, cross-platform execution,
When it is done, it’s usually not part of the cycle. Accessibility
and AI-driven analytics into one secure, cloud-based web and
testing is not executed as frequently as it needs to be. This leaves
mobile testing platform.
risk for expensive fees and escaped defects.

Visit www.perfecto.io for a free trial, demo, or to learn more.

Fortunately, with Perfecto, accessibility testing is a seamless
and free add-on to existing test automation cycles — for both
web and mobile apps. RELATED RESOURCES
• The 2022 State of Test Automation

Conclusion • What Is Good Testing in 2022?

• Don’t Crash the Party: Mobile UX Testing

If you want a top-notch mobile app that boasts features like
biometric and two-factor authentication, network virtualization,
voice recognition, geolocation, and image injection, you need
to put it through a robust gauntlet of testing to ensure a flawless
UX before you deploy. Otherwise, you run the risk of offering
a faulty app — who wants one of those?

With Perfecto, you get the only testing platform that covers all
these advanced capabilities. End-to-end testing creates a seamless
experience, so you don’t have to patchwork your tests to get the
result you want. When you only need one source to conduct every
test you need, your testing is simpler, more effective, and the results
will speak for themselves.

Perfecto by Perforce © Perforce Software, Inc. All trademarks and registered

15 | www.perfecto.io
trademarks are the property of their respective owners. (0820KS22)