Introduction To Writing Network Tests With Pyats: Hank Preston Twitter: @hfpreston April 2020

Introduction to Writing Network
Tests with pyATS
Hank Preston
Twitter: @hfpreston
April 2020
• What is pyATS
• A Network Test Project
• Testbeds and Device
Agenda Connections
• The Anatomy of a Testcase
• Combining Testcases into Jobs
• Network Test Demonstration
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public Twitter: @hfpreston | Email: hapresto@cisco.com
What is pyATS?
Is the ”network down”?
Is there a better way?
What should
we check?
ping it!
Like the cartoons? Checkout https://dcgubbins.com

pyATS and the pyATS Library
define an ecosystem that
streamlines and standardizes
how you set up and run
automated network tests.
The pyATS ecosystem is available under Apache

License Version 2.0
https://pubhub.devnetcloud.com/media/pyats-getting-started/docs/intro/introduction.html
pyATS Ecosystem
Installing pyATS
pip install 'pyats[full]’
or
docker run -it ciscotestautomation/pyats
pyats version check

• Available on PyPi and
Docker You are currently running pyATS version: 20.2.1
Python: 3.7.6 [64bit]
• Requires Linux (or macOS) Package Version

---------------------------- -------
• Python 3.5+ genie
genie.libs.conf
20.2
20.2
genie.libs.ops 20.2
genie.libs.parser 20.2
genie.libs.robot 20.2
genie.libs.sdk 20.2.2
genie.telemetry 20.2
genie.trafficgen 20.2
pyats 20.2.1
pyats.aereport 20.2
pyats.aetest 20.2.
unicon 20.2
unicon.plugins 20.2
Reference Notes
• Python library names
• pyATS – Core Test Infrastructure and Libraries
• https://pubhub.devnetcloud.com/media/pyats/docs/index.html
• Genie – “Library” of reusable parsers, models, triggers, verifications, APIs
• https://pubhub.devnetcloud.com/media/genie-docs/docs/index.html
• Unicon – “Universal Connector” provides device control functionality
• https://pubhub.devnetcloud.com/media/unicon/docs/index.html
A Network Test Project
What makes up a pyATS Network Test Project
• Testbed
• YAML file
• Describes network topology ├── testbed.yaml
├── network_test_project
• AEtest Testscripts │ ├── testbed_connection.py
(Automation Easy Testing)
• 1 or more Python files
│ ├── interface_errors.py
• Define setup, execution and cleanup of
│ └── network_test_job.py
tests
• Easypy Jobfiles
# Convenience commands
• Python file pyats create testbed
• Combines testscripts and defines pyats create project
runtime, logging, and archives https://pubhub.devnetcloud.com/media/pyats/docs/cli/pyats_create.html
pyATS Testbed testbed:
name: Multi Platform Network
credentials:
default:
• Defines all devices in the network username: '%ENV{PYATS_USERNAME}’
• Types: switch, router, linux, firewall, etc password: '%ENV{PYATS_PASSWORD}’
• OS: iosxe, nxos, asa, linux, junos enable:
password: '%ENV{PYATS_PASSWORD}'
• Describes connection methods for
each device devices:
internet-rtr01:
• SSH, Telnet, NETCONF, etc os: iosxe
• Credentials type: router
series: csr1000v
• Optionally can define topology, or links connections:
between devices defaults:
class: unicon.Unicon
• Creation options a:
• Manual, CSV/XLS, Question/Answer protocol: ssh
ip: 10.10.20.181
https://pubhub.devnetcloud.com/media/pyats/docs/topology/index.html#
https://pubhub.devnetcloud.com/media/unicon/docs/user_guide/supported_platforms.html
Phases
AEtest Testscript 1. CommonSetup
2. Testcase(s)
Basics 3. CommonCleanup
from pyats import aetest class testcase02(aetest.Testcase):
'''The Second Testcase '''
class CommonSetup(aetest.CommonSetup):
@aetest.subsection @aetest.setup
def connect(self, testbed): def setup(self):
'''connect to all your testbed devices.''' pass
Testcases can have Setup/Cleanup
# connect to all testbed devices @aetest.test Steps to prepare network for testing
testbed.connect() def test(self): Ex: Perform Configuration, Start
pass Traffic generation
class testcase01(aetest.Testcase): @aetest.cleanup

'''The First Testcase ''' def cleanup(self):
pass
# you may have N tests within each testcase
@aetest.test
def test01(self): class CommonCleanup(aetest.CommonCleanup):
pass '''CommonCleanup Section'''
@aetest.test # uncomment to add new subsections

def test02(self): # @aetest.subsection
pass # def subsection_cleanup_one(self):
# pass
https://pubhub.devnetcloud.com/media/pyats/docs/aetest/introduction.html
Easypy Jobfiles
• A “Job” is the combination of from pyats.easypy import run
testscripts together for def main(runtime):

'''job file entrypoint'''
execution
# run script(s)
• Each testscript to be run is a run(testscript=’testscript1.py’,
taskid=“Script 1”,
“Task” runtime = runtime)
run(testscript=’testscript2.py',
• Note: taskid is optional and taskid=“Script 2”,
defaults to Task-# runtime = runtime)
run(testscript=’testscript3.py',
• Each job execution generates runtime = runtime)
single log and archive run(testscript=’testscript4.py',
runtime = runtime)
https://pubhub.devnetcloud.com/media/pyats/docs/easypy/index.html
Running a pyATS Network Test Job
pyats run job sample2/sample2_job.py --testbed testbed_min.yaml
2020-04-11T16:23:52: %EASYPY-INFO: Starting job run: sample2_job
.
.
2020-04-11T16:23:57: %EASYPY-INFO: +------------------------------------------------------------------------------+
2020-04-11T16:23:57: %EASYPY-INFO: | Task Result Summary |
2020-04-11T16:23:57: %EASYPY-INFO: +------------------------------------------------------------------------------+
2020-04-11T16:23:57: %EASYPY-INFO: Task-1: sample2.common_setup PASSED
2020-04-11T16:23:57: %EASYPY-INFO: Task-1: sample2.test01 PASSED
2020-04-11T16:23:57: %EASYPY-INFO: Task-1: sample2.common_cleanup PASSED
2020-04-11T16:23:57: %EASYPY-INFO:
2020-04-11T16:23:57: %EASYPY-INFO: +------------------------------------------------------------------------------+
2020-04-11T16:23:57: %EASYPY-INFO: | Task Result Details |
2020-04-11T16:23:57: %EASYPY-INFO: +------------------------------------------------------------------------------+
2020-04-11T16:23:57: %EASYPY-INFO: Task-1: sample2
2020-04-11T16:23:57: %EASYPY-INFO: |-- common_setup PASSED
2020-04-11T16:23:57: %EASYPY-INFO: | `-- connect PASSED
2020-04-11T16:23:57: %EASYPY-INFO: |-- test01 PASSED
2020-04-11T16:23:57: %EASYPY-INFO: | `-- test PASSED
2020-04-11T16:23:57: %EASYPY-INFO: `-- common_cleanup PASSED
2020-04-11T16:23:57: %EASYPY-INFO: Done!
Pro Tip
-------
Use the following command to view your logs locally:
pyats logs view
Note: During execution stdout shows a great deal of detail and logging https://pubhub.devnetcloud.com/media/pyats/docs/cli/pyats_run.html
pyats logs view
Logfile: /Users/hapresto/.pyats/archive/20-
Apr/sample2_job.2020Apr11_16:23:51.170967.zip
pyATS HTML Logs Viewer View at:

http://localhost:61306/result
Press Ctrl-C to exit
View Details of a Testscript/Task
pyats logs view

Review all captured output from Test
pyats logs view

Network Test
Demonstration
The Test Network
• Run the demo yourself
• Code Available on GitHub
• Sample Network Provided by
DevNet Sandbox
• Test Goals
• Verify every device in testbed
is reachable
• Verify no network interfaces
report errors
Live Demo
https://github.com/hpreston/intro-network-tests
Twitter: @hfpreston | Email: hapresto@cisco.com
© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public
A look at the connection verification test
class verify_connected(aetest.Testcase):
@aetest.test
def test(self, testbed, steps):
# Loop over every device in the testbed
for device_name, device in testbed.devices.items():
with steps.start(f"Test Connection Status of {device_name}", continue_=True) as step:
# Test "connected" status
if device.connected:
logger.info(f"{device_name} connected status: {device.connected}")
else:
logger.error(f"{device_name} connected status: {device.connected}")
step.failed()
• Verifies that each device from testbed was successfully connected to

in CommonSetup
• Uses “steps” to better control flow
https://pubhub.devnetcloud.com/media/pyats/docs/aetest/steps.html
https://github.com/hpreston/intro-network-tests/blob/master/network_test_project/testbed_connection.py#L43
A look at the interface errors test – Part 1
class interface_errors(aetest.Testcase):
# List of counters keys to check for errors
counter_error_keys = ("in_crc_errors", "in_errors", "out_errors")
@aetest.setup
def setup(self, testbed):
"""Learn and save the interface details from the testbed devices."""
self.learnt_interfaces = {}
for device_name, device in testbed.devices.items():
# Only attempt to learn details on supported network operation systems
if device.os in ("ios", "iosxe", "iosxr", "nxos"):
logger.info(f"{device_name} connected status: {device.connected}")
logger.info(f"Learning Interfaces for {device_name}")
self.learnt_interfaces[device_name] = device.learn("interface").info
• Testcase “setup” step used to “learn” the interface details

• Leverages pyATS Library to convert device state to Python objects
https://pubhub.devnetcloud.com/media/pyats-getting-started/docs/quickstart/learndevices.html
A look at the interface errors test – Part 2
@aetest.test
def test(self, steps):
for device_name, interfaces in self.learnt_interfaces.items():
with steps.start(f"Looking for Interface Errors on {device_name}", continue_=True) as device_step:
for interface_name, interface in interfaces.items():
with device_step.start(f"Checking Interface {interface_name}", continue_=True) as interface_step:
# Verify that this interface has "counters" (Loopbacks Lack Counters on some platforms)
if "counters" in interface.keys():
# Loop over every counter to check, looking for values greater than 0
for counter in self.counter_error_keys:
# Verify that the counter is available for this device
if counter in interface["counters"].keys():
if interface["counters"][counter] > 0:
interface_step.failed(f'{interface_name} has {interface["counters"][counter]}')
else:
# if the counter not supported, log that it wasn't checked
logger.info(f"Device {device_name} Interface {interface_name} missing {counter}")
else:
# If the interface has no counters, mark as skipped
interface_step.skipped(f"Device {device_name} Interface {interface_name} missing counters")
• Nested loops to check each counter for each interface for each device
• If error counters are found, fail the test
• If “counters” not available, log or skip the test
https://pubhub.devnetcloud.com/media/pyats/docs/aetest/steps.html
Summary and Resources
Where to go for more resources and information
• pyATS Homepage on DevNet We just scratched the surface of
what’s possible with the pyATS
• pyATS Getting Started Guide Ecosystem.
• pyATS on PyPi • Hundreds of included Triggers,
• Explore pyATS Library (aka Genie) Verifications, and APIs
of parsers/models
• Integrating with CICD pipelines
• Join the pyATS Community Chat on
Webex Teams • Integrating with frameworks like
Robot
• Learning Labs and other Resources
• Developing custom parsers and
models
Interested in Network Automation? Stay in Touch!
Hank Preston developer.cisco.com

hapresto@cisco.com @CiscoDevNet
@hfpreston facebook.com/ciscodevnet/
http://github.com/hpreston http://github.com/CiscoDevNet

Introduction To Writing Network Tests With Pyats: Hank Preston Twitter: @hfpreston April 2020

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Introduction To Writing Network Tests With Pyats: Hank Preston Twitter: @hfpreston April 2020

Uploaded by

Copyright:

Available Formats

Introduction to Writing Network

Tests with pyATS

• Combining Testcases into Jobs

• Network Test Demonstration

Like the cartoons? Checkout https://dcgubbins.com

The pyATS ecosystem is available under Apache

pyats version check

• Requires Linux (or macOS) Package Version

class testcase01(aetest.Testcase): @aetest.cleanup

@aetest.test # uncomment to add new subsections

testscripts together for def main(runtime):

pyATS HTML Logs Viewer View at:

Press Ctrl-C to exit

View Details of a Testscript/Task

pyATS HTML Logs Viewer View at:

Press Ctrl-C to exit

Review all captured output from Test

pyATS HTML Logs Viewer View at:

Press Ctrl-C to exit

• Verifies that each device from testbed was successfully connected to

• Testcase “setup” step used to “learn” the interface details

Hank Preston developer.cisco.com

You might also like