Automating for a

Smarter Future
Streamline | Empower | Perform

Spirent Automation Platform Technologies

Marc HOREM
Pre-sales Systems Engineer
Velocity Platform Breaks Barriers to Automated
Continuous Test

Managers Developers Testers Lab Operations Customer End Users

Browser Based User Interface

Resource Control and Test Authoring and Abstract Execution and Test Analysis
Automated Workflows Testbed Orchestration Network Connectivity
Dashboards Automation Libraries Reporting and Dashboards

REST API

DevOps Ready

2
Test as a Service
Features and Value Propositions
Turbo-charge automation proficiency
Leverage Python automation for faster
releases and increased test coverage
NFVi, VNF and Service
Level Testing
Control life cycle testing
using one single tool
Unified Test Reports
Streamline root cause
analysis using a single report
from all testbed devices
DevOps Ready
Maximize organizational efficiency
by plugging into common DevOps applications
Capture and replay
Increase quality and predictability with
automated tests matching test procedures
Test case abstraction
Quickly and efficiently generate reusable
tests to validate multiple testbeds
Automated data extraction
Gain actionable information by
automatically deconstructing
device responses
Spirent’s LaaS Solution
Integration
Maximize reuse and test coverage
by deploying and consuming tests seamlessly
4
Build out Automation with Capture/Replay
Test Engineers Create Automation Following Their Natural Workflow

CLI
Telnet, SSH, CMD, Serial, Process, File, Syslog, TCL Shell, Python Shell,
PowerShell

GUI
Web, Java, Flex, VNC, Selenium/Ranorex, TestPlant

Network Traffic Generator

Spirent (STC, Landslide, Avalanche, CloudStress, Smartbits)
Ixia (IxLoad, IxTrffic, IxNetwork, N2X)

Enterprise, Virtual & Cloud

VMware, OpenStack, Database, Web Services (RESTful/SOAP/ XML-RPC)

Protocol & Communications

SNMP, HTTP, UDP, XMPP, NETCONF, SMTP/Email, POP3, Wireshark

Mobile
ADB, VNC, Ranorex (APPs), TestPlant

Desktop App
Ranorex (Windows), TestPlant (Any GUI), VNC
PROPRIETARY AND CONFIDENTIAL 5
Build out Automation with Capture/Replay
Test Engineers Create Automation Following Their Natural Workflow

Technical Benefit
CAPTURE REPLAY • Capture every action during
manual test
• Replay captured steps into
automated tests

Economic Benefit
• Save time and money
(Domain experts follow
natural workflow)
• Increase quality and
predictability (automated
test cases exactly match
test procedure)
Extend with Analysis
Validate with Spirent Patented Response Maps

Technical Benefit
• Extract relevant data from
responses for
measurements and
validation

Economic Benefit
• Save time on low value
pattern matching that is
better applied to expanding
test coverage
 Portabilize via Abstraction
Run the Same Test Cases Against Different Testbeds

Technical Benefit

• Abstract the device specific

Abstract
interface iTest
in QuickCalls
Test Case

Economic Benefit
• Reduce maintenance costs

Vendor A
Vendor C
Vendor B
Publish – Share – Consume
Share test cases with all teams

Technical Benefit
Test Case Test Case Test Case
Creation Publishing Consumption • Publish tests for wide
consumption both in labs and
production

Economic Benefit
Abstract • Increase productivity and
iTest reduced cost (full team
Test Case empowered to consume
automation from central
portal

PROPRIETARY AND CONFIDENTIAL 9

Interpret with Rich Reporting
Quickly Find Failure Points Using a Consolidated Test Report

Technical Benefit
• Leverage detailed reports
to perform root cause
analysis
• Capture every single test
step in detailed, unified
customizable report

Economic Benefit

• Save time in determining

root cause of failures
• Increase quality and
customer satisfaction
 Integrate with Python frameworks
Convert iTest to Python for Automated Regression Testing

Sprint Testing iTest to Python Regression Testing

Conversion Technical Benefit
• Turbocharge Python code
with iTest’s powerful
QuickCalls and Response
Maps
• Publish automation to
Python test regression
systems

Economic Benefit
• Save time and money
• Increase developer
productivity
Enable DevOps Continuous Testing
DevOps Tool Chain integrations enable Continuous Test and Immediate Feedback

Source Control SDLC / Bug

Technical Benefit
Management Management
• Integrates with DevOps Tool
Chain applications
• Identify defects immediately
after software builds

Economic Benefit Code Results and

Changes Developers, Testers, Bug Reporting
• Increase quality and
Product Owners
customer satisfaction

Test Jobs

Jenkins Build Test Execution

PROPRIETARY AND CONFIDENTIAL 12
 Validate at Scale with Distributed Agents
Deploy test agents near devices under test (DUT)
Technical Benefit

• Integrates Execute at scale

intelligent, high
performance, distributed
automation agents
Developers and • Parallelize test execution
Testers locations

Economic Benefit
• Increase Accelerate time to
market
• Increase product/service
quality

PROPRIETARY AND CONFIDENTIAL 13

Differentiated Lab and Test as a Service Workflows
Over 15 Automated Workflows!

Testbed Data
Cable
connectivity Collection
Workorder
(L1, L2, and
Dispatch
virtual) Management
Test
Resource
Execution
Selection
Queueing
Utilization and
Resource
Test Trending
configuration
Reports
Testbed
Standups Reservation Triggered
Request and Automation
(physical,
Approvals
Inventory virtual, hybrid)
Test Reservation
management
Execution Resource
and
Scheduling Test Resolution
population
Execution Regression
Agent Reservation
Organization
Selection Sharing
and Dispatch
Lab as a Service
Features and Value Propositions

Built-in Automated Workflows

Turbo-charge automation proficiency
Streamlined natural and automated workflows
Leverage Python automation for faster
for effective use of commonly deployed lab and test
releases and increased test coverage
processes/procedures

Topology and Resource Abstraction

Physical, Virtual and Hybrid Testbeds
System intelligently creates testbeds based on abstract
Supports all resource classes to
criteria of resources ensuring high utilization of
facilitate transition from physical to virtual to cloud
devices/applications

Global Instantaneous Access Automated Testbed Creation

to Lab Resources & Orchestration
Enabling secure remote access Enable zero touch self-service testing to
and control of testbeds from anywhere at anytime increase productivity and user satisfaction

Fully Automatable and Extensible Deploy Test as a Service

Integrate and execute any automated Seamlessly deploy and consume
scripts with multi-language support, tests to maximize reuse and test coverage
customize with script-based automated workflows
16
Automated Testbed Creation &
Orchestration for Physical, Virtual and
Hybrid Resources
Create Automated Reusable Testbeds for Increased
Self-Service Efficiency
Supports OpenStack and VMware clouds

• Technical Benefits
– Reduce testbed setup times
– Guaranteed, repeatable testbed setup
– Eliminate human errors

• Economic Benefits
– Catalog of predefined/ preconfigured testbeds
– Powerful inventory search
– Optimized resource selection and routing across
L1/L2/SDN
– Manual cabling workflow management
– Triggered Automation and configurable drivers
17
Resource Abstraction
Lab consolidation is a non-starter without Abstraction
User Asks For:
Server 32-bit, 16GB

• Technical Benefits
– Convert static into dynamic environments
– Eliminate stranded resources Resource Inventory: Servers
– Isolate users from lab complexity
Match Match

• Economic Benefits
32-bit, 16GB
32-bit, 16B
– Rapidly find/reserve any resource that in use

meets the requirements

– Optimally employ resources
32-bit, no 32-bit, 32-bit,
– Increase testing coverage – combinations connection 8G 8G

and permutations 32-bit, no

connection

64-bit

PROPRIETARY AND CONFIDENTIAL 18

Topology Abstraction
Portable, reusable, scalable & maintainable topology

Resource Inventory

Firewall

Router
Router Router Switch

Switch

Server Firewall Server

PROPRIETARY AND CONFIDENTIAL 19

Easily Integrates with Your
DevOps Tool Chain
Embrace and Extend Existing Test Automation

• Technical Benefits
– REST API
– Supports Python, Bash, iTest and
other languages
• Economic Benefits
– Knowledge reuse reduces time to
market
– Extend best-of-breed Lab Ecosystems
and/or DevOps Tool Chains

20
 Architecture

Stateful Lab E2E Lab Topology Topology

Inventory Connectivity Resolution Scheduler

Interactive Configuration Policy Automated

Control & Provisioning Enforcement Workflows

Distributed Automation Automation

Automation Scheduler Repository

Velocity LaaS Platform

Physical/Virtual/Hybrid Lab

Location 1 Location 2 Location N

Lab Lab Lab
1 3 N
Velocity Agent Velocity Agent Velocity Agent
Velocity Agent Velocity Agent Velocity Agent
Lab Velocity Agent Lab Velocity Agent Lab Velocity Agent
2 4 N+1

21
VNF Testing

Testbed
Physical lab System under Test

Test VNFO
Manager VNF Controller
FG
Test VNF VNF VNF Test VNFM
PNF / VNF 1 2 3 PNF / VNF

Router Switch

Virtualization
VIM
Compute Storage Network

Firewall Server

Orchestrating a virtual testbed requires instantiation of virtual devices on top of the virtualization infrastructure and existing environment.
Adding further complication are combined physical and virtual environments, known as Hybrid. 22
Dashboards and Visualizations
Reveal CapEx, ROI, test coverage and effectiveness

• Technical Benefits
– Centralized scalable data collection
– Resource metrics – Availability, Utilization,
Conflicts
– Usage metrics – reservations & executions
• Economic Benefits
– Analytics dashboards for trending analysis
– Real-time insight into test processes and
results
– Determine CapEx ROI

PROPRIETARY AND CONFIDENTIAL 23

5G CI/CD/CT Demo Sequence
Engineer Develops Automated 5GC Test Case

Open iTest and Open Landslide Parameterize Convert to test case

import testbed session and connect test case and execute

2 3 4 5

Landslide AMF
1 6
SMF
Automation
Engineer
Automation automatically
Create and reserve testbed appears in Velocity Push automation
content to Git
PROPRIETARY AND CONFIDENTIAL 25
 5G Automation Dev/Execution Sequence
Engineer Develops Automation, Publishes, and Executes in Velocity

2 1 Developer creates automation in iTest, Python, or Robot

2 iTest content is easily exported to Python or Robot

1 3 Developer pushes automation to Git, appearing instantly in Velocity

4 Velocity spins up dynamic topology or reads from existing environment

5 Topology is passed to iTest, Python, or Robot upon execution

3 6 All results and artifacts are published to common destination

4 Reports and
Artifacts

PROPRIETARY AND CONFIDENTIAL

5 6 26
 5G CI/CD/CT Demo Sequence
Automated Regression Triggered by Appearance of New 5GC Build

Poll / Trigger
Upload Download Regression
Image 5G Regression Setup Testbed Dynamic 5G
Registry Build Reserve Resources Testbed
VNF
Build 1 2 3 Instantiate VNFs 4
Supplier Connect and Configure
Execute Test Sequence
NFVi Readiness Testing Setup
5G Functional Testing Test
Jenkins 5G Performance Testing Teardown
E2E Integration Testing
1 VNF supplier uploads new 5GC build Teardown Testbed
Testbed Teardown

2 Jenkins detects new build available for testing Release Resources

Gather Results Publish
Report Generation Reports and
3 Jenkins invokes regression test on Velocity Save Artifacts Artifacts
Velocity spins up dynamic topology, performs regression test,
4 generates results and artifacts

PROPRIETARY AND CONFIDENTIAL 27

 5G CI/CD/CT Reference Architecture
Continuously Test 5G Builds, Configurations, and VNF Templates
Push Hook Store
Testcase Test Suites
Repo
Automation
Engineer Tests
Setup Testbed
Push Hook Reserve Resources
Service Trigger Instantiate VNFs
Config Repo Regression Connect and Configure
Configs
5G SME Execute Test Sequence
Engineer NFVi Readiness Testing Setup
5G Regression 5G Functional Testing Test
Push Hook Teardown
Build 5G Performance Testing Dynamic 5G
Templates VNFD Repo E2E Integration Testing Testbed
Cloud Teardown Testbed
Engineer Testbed Teardown
Release Resources
Gather Results Publish
Poll / Report Generation Reports and
Download Jenkins Save Artifacts Artifacts
Upload
Image
VNF Builds Registry Any relevant change to the 5G service (builds,
Supplier configurations, virtual environment) triggers full end-to-
PROPRIETARY AND CONFIDENTIAL end regression testing. 28