Fused Testing:

A Pragmatic Test
Automation Journey
Fusing Manual and Automated Testing
Table of
Introduction 1
Testing apps at scale is not for the faint of heart 2
Challenges for today’s engineering teams 2
The vital roles of automation and manual testing 3

Contents


Automated Testing
Automation Hurdles
Manual Testing
3
4
5
Manual Testing Hurdles 6
A leap towards continuous integration 7
How fused testing empowers engineering leaders 7
How fused testing works 8
Fused testing readiness checklist 9
Applied Example Of Fused Testing 10
About Testlio 11
Fused Testing
Introduction
Software development leaders must make tradeoffs
between quality, speed, and coverage while also dealing with
time, talent, and budget constraints. Finding the balance
between rigorous testing and fast releases is complicated.
Manual and automated testing strategies offer unique
advantages. Automated testing ensures consistency and
immediate test execution. Manual testing offers flexibility
and actionable insights. Fused testing is a methodology
that strategically combines manual and automated testing in
new ways, with fewer tradeoffs.
“All organizations, even those in risk-averse
industries like financial services, are feeling the
need to deliver software faster.”
Chris Condo,
Principal Analyst at Forrester
What to expect
This guide explains how the logical principles
of fused testing debunk common test
automation journey assumptions. Together,
humans and machines run rapid test cycles
conducted in short timeframes to release
quality apps and better end-to-end customer
experiences.
Readers will come away from this guide
understanding:
• A framework and tools to launch or expand
your test automation
• How to optimize your testing by fusing
automated and manual testing
• What fused testing strategy looks like in
practice
• How fused testing increases testing
velocity, quality, and coverage
1
Fused Testing

Testing Apps At
Scale Is Not For
The Faint Of Heart
End-users expect a great experience, period. They don’t
care if your software is challenging to build and release.
Unfortunately, the tension between scaling quality and
timely releases keeps engineering leaders awake at night.

We’ll share examples to scale and nearly fail-proof your

testing with smart ways to combine your automated and
manual testing. Challenges for today’s engineering teams
Software development realities Too much to juggle

• Great expectations: 76% of end-users • Scaling test automation

say their expectations of digital services
have increased. (AppDynamics | 2021)
• Shorter release timelines: 84% of
developers report releasing code faster
than ever before. (GitLab | 2021)
• Limited testing talent: 64% of DevOps
teams indicate a scarcity of talent
(Techstrong Research | 2021)
• Sourcing QE and QA talent
• Maintaining automated test scripts
• Ensuring full device/OS coverage
• Managing integrations
• Covering location, language, and network
requirements
• Hidden costs

2
Fused Testing
The Vital Roles Of
Automation And Manual
Testing
Automated Testing Unit tests
Automation plays a critical role in a holistic Unit tests run quickly, are usually
software testing strategy. It’s a repeatable, written in the same language as
efficient approach — especially for unit, API the underlying code, and are tightly
and UI tests. coupled to the code.
The standard approach is to start with
unit tests to ensure that an automated
testing suite is sufficiently broken down
into small parts, reducing the flakiness
and run time of your automated testing
operation.
For all of these reasons, unit tests
make up the base of the test
automation pyramid.
API Tests
Different parts of an app communicate
with each other through APIs
(Application Program Interface). API
tests will simulate one side of that
communication to ensure the AP
connection is functioning.
API tests run fast, but not as quickly
as unit tests. They are also reliable,
and often run without providing false
results (flaky tests). We typically test
the business logic of the application
using API tests.
UI Tests
UI tests are generally created
through frameworks and tools that
are accessible to testers lacking
programming skills. This helps make UI
tests a natural fit for test automation.
Moreover, UI tests generally map to
existing manual tests, so automated
UI tests can be a direct replacement if
your goal is to reduce manual tests.
3
Fused Testing

For all of its strengths, automated testing

comes with a set of challenges that impact
continuous integration.

Automation Hurdles
• A scarcity of qualified QE talent
• Bottlenecks caused by flaky, broken scripts,
or unmanageable test suites
• Automated tests can miss obvious issues a
human using real devices can easily detect
• Maintaining integrations across a diverse
DevOps toolchain
• Necessary governance and oversight
to ensure cost-effective writing,
implementing, and script maintenance

4
Fused Testing
Manual Testing
While automated testing offers immediate
test execution, manual testing provides
flexibility and coverage that only humans
with real devices and localized insights can
perform. Manual testing is essential for
verifying new features and offering actionable
reproduction instructions for issues missed
by flaky or broken automated scripts. Manual
testing is best suited for digital experience
testing (e.g., usability and accessibility),
localization, payments, exploratory testing,
and overall issue validation.
Exploratory testing
Software bugs are sneaky — they hide
in the nooks and crannies of project
requirements and user stories.
Structured exploratory testing
helps uncover issues missed from
automated scripts before your
customers find them.
What to look for:
• Experienced testers
• Flexible and burstable teams
• Bugs vs. noise
• Global coverage
Usability testing
Usability revolves around the entire
app-driven experience — something
that requires human testing on real
devices and in real locations.
Usability testing uncovers problem
areas where the customer experience
falters, both inside and outside of the
app.
What to look for:
• Planning, management, recruitment,
recordings, and recommendations
to improve the end-to-end digital
experience.
Localization testing
Users will pass up products whose
graphical or UI elements are
incompatible with their culture,
language, or preferred devices.
Localization testing matches your
app with an expert network of global
testers to assure your app passes the
“locals” test.
What to look for:
• Design, language, and bidirectional
UI issues
• Confirm that localization efforts
don’t create breaks
5
Fused Testing

Like automation, manual testing comes with a set of

challenges.

Manual Testing Hurdles

• Not efficient for daily unit, API, or clean (non-flaky) UI
testing
• More labor intensive than automated testing
• Sourcing in-house or outsourced global QA for
localization
• Unskilled or inconsistent testers are more likely to
uncover low quality bugs
• Requires strategy, planning, management, and
expertise to effectively improve the end-to-end digital
experience

6
Fused Testing

A Leap Towards
Continuous Integration
How fused testing empowers
engineering leaders
Fused testing goes well beyond executing manual and
automated testing in parallel. By seamlessly merging
human and machine testing resources into unified
workflows, development is one step closer to continuous
integration (CI). Fused testing enables shift-left and shift-
right principles by identifying and fixing bugs earlier,
keeping up with ambitious software release and update
schedules, and ensuring high levels of usability.

• Platform - Acts on signals from CI/CD, test automation,

and issue management tools
• Process - Continuous testing with fluid transitions
between manual and automated workflows for parallel
testing, manual continuation, manual validation, and
more
• People - Governance ensures you have the right
test automation strategy, tools, and enforcement of
scripting best practices. On-demand resources and
expert Services help you scale QE and QA The technology, process, and people building blocks for fused testing

• Open Architecture - Extensible approach that lowers

switching costs between automation frameworks and
drives better ROI for manual and automation efforts.

7
Fused Testing
How Fused
Testing Works
Many software development teams already
combine manual and automated workloads.
What makes fused testing unique is that it
starts with the assumption that neither form
of testing can replace the other. It offers a
license to mix and match between automated
and manual in smarter, signal-based
combinations. This allows product teams to
make better optimization decisions regarding
speed, coverage, and capacity.
• We review and validate test results and
Mix and match
Fused testing enables parallel testing, manual
defects while also taking care of noise and
Flaky or failed automated tests immediately
kick off manual testing, ensuring an optimal
continuation for flaky tests, manual validation
for failed tests, and more.
flaky tests.
• We combine tester insights, a thorough
combination of humans and machines. This
fallback mechanism prevents bottlenecks
in the testing flow by deploying human eyes
review, and validation to create actionable
How Testlio fuses manual and automated on the testing surface to confirm if issues
bug reports for automated tests failures
testing were related to the automated tests or the
and integrate all reporting into your
application.
• We help manage the right blend of manual workflows for intelligent decision making.
and automated testing by recommending
manual tests to automate, and vice versa.
• We tap into a network of expert freelance
quality engineers for scripting, running,
analyzing, and maintaining automated tests.
• We activate our global network of
experienced, on-demand manual testers to
work in parallel with automated scripts.
• We manage test runs and setups for you.
• Integrations with Testlio’s automated
features and/or leading, third-party
automation platforms pull automation test
failures for manual validation or fallback.
• Failure signals are then pushed to manual
testers who investigate, reproduce, and
quickly offer actionable insights.
8
Fused Testing

Fused testing readiness checklist

Open architecture
Answering these questions will help you decide if
Fused testing helps maximize attainable ROI on current & future investments in
fused testing is for you:
test automation tooling with an open and extensible architecture. An agnostic
1. Are my manual test cases stable and candidates approach to DevOps integrations lowers switching costs and provides flexible
for automation? tooling options for task management, test management, and test automation
2. Am I evaluating the right automation strategies through partnerships and integrations.
and tools?
3. Are my manual and automated testing efforts
siloed?
4. Is our existing automation not yielding expected
speed gains?
5. Do we struggle to find quality engineering talent to
support automation?
6. Do we have gaps in testing coverage and capacity
that limits our ability to scale?

9
Fused Testing

Applied Example
Of Fused Testing
To provide an applied example, see the basic concept
used today by one of Testlio’s fintech clients.

Fused testing helps support LGND.art push for continuous integration “Fused Testing allows us to dedicate more
manual testing energy for new product
development. Unfinished products are
Manual testing: Automated candidates: Manual fallback: Seamless workflow: CI/CD: under very heavy development with
frequent changes to the code base. We
Frontloaded Most of the repeatable If anything broke during Reported bugs were Any time a merge now begin testing new products much
manual testing for manual work shifted automated runs, a immediately pushed to happened into a sooner. The early feedback is much
early feedback on to nightly automation manual regression a JIRA backlog offering specific branch, LGND better than waiting to test at the end
functionality, log-in, runs. testing fallback initiated visibility into the entire could automatically and receiving a huge list of bugs upon
sign-ups, change quick verification within feedback cycle, with push and deploy. feature complete. We’re on the path to
passwords, connected the same cycle. filtered searches accelerating our development cycles
digital wallets, to identify issue by simply frontloading a little bit of the
payments, UI, and small magnitude. testing.”
interactions.
Adam Kneisler,
VP of Engineering, LGND

Fused testing combines the strengths of
automation with the judgment of human
testers for continuous testing. It does
this through a powerful platform, flexible
integrations and partnerships, on-demand
testing resources, and managed services
that enable parallel testing and quickly
address the inevitable gaps caused by
automation test failures with manual
continuation for flaky and failed automated
tests. This automation fallback mechanism
prevents bottlenecks in the testing flow by
deploying human eyes on the testing surface
to confirm if issues were related to the
automated tests or the application.

10
Fused Testing

Testlio is the Finance and Investing

leader in software
testing Learning and Education

Mobility and Travel

165+ 10K+
full-time employees vetted freelancers

Commerce and Retail

3% 150+ Health and Wellness

applicants are tester countries
accepted into the
network

Media and Entertainment

30k+ 1200+
test runs completed devices covered
Business Software

11
See the Testlio Approach
to Fused Testing
Request a demo