Citrix SD-WAN-R10.0 Lab Guide-Full-v4

Citrix SD-WAN
Lab Exercises
Worldwide Product Readiness

April 7, 2018
Prepared by: Shoaib Yusuf

Authors
The following authors contributed to the creation of this deliverable.
Citrix
Shoaib Yusuf
Solutions Architect
shoaib.yusuf@citrix.com
Revision History
Revision Change Description Updated By Date

1.0 9.3.3 SD-WAN Shoaib Yusuf 04/2018
2.0 10.0 SD-WAN Shoaib Yusuf 04/2018
citrix.com 2
Lab Guide Overview .............................................................................................................. 4
Lab Guide Overview........................................................................................................................5
Lab Environment Details .................................................................................................................6
Lab Scenario and Instructions..........................................................................................................8
Module 1 (Basic): SD-WAN Installation and Configuration ...................................................... 9
Exercise 1: Understanding the Underlay Network .......................................................................... 11
Exercise 2: Understand Overlay Design (NYC, LON, DAL, SJC sites) .................................................. 22
Exercise 3: Start lab with administrative tasks on the MCN ............................................................ 33
Exercise 4: Configuration Editor – Build MCN Node ........................................................................ 44
Exercise 5: Configuration Editor - Build Client Nodes (LON and SJC) ................................................ 54
Module 2 (Basic): SD-WAN Provisioning and Change Management ...................................... 65
Exercise 6: Provisioning on the MCN (NYC) .................................................................................... 67
Exercise 7: Applying the Configuration to the Remote Appliances (LON & SJC) ............................... 77
Exercise 8: Adding additional WAN links (NYC, LON, SJC) ............................................................... 94
Module 3 (Basic): Troubleshooting and Validating SD-WAN Overlay................................... 107
Exercise 9: Troubleshooting path state ........................................................................................ 109
Exercise 10: Redirect traffic using OSPF to Virtual Inline Mode SD-WAN (NYC) ............................. 130
Exercise 11: Dynamic Virtual Path between LON and SJC ............................................................. 148
Module 4 (Basic): Deployment Features and Services ......................................................... 160
Exercise 12: Configuration of BGP and Intranet Service to DAL (LON and SJC) ............................... 162
Exercise 13: Internet Service (LON) .............................................................................................. 176
Exercise 14: Stateful Firewall (LON) ............................................................................................. 185
Exercise 15: Routing Domains for Internet Service (LON) ............................................................. 199
Exercise 16: High Availability Deployment (NYC) .......................................................................... 206
Exercise 17: Standby Links (LON) ................................................................................................. 215
Module 5 (Advanced): Pre-stage Environment ................................................................... 227
Exercise A: Import pre-staged configuration to SD-WAN devices .................................................. 229
Module 6 (Advanced): Central Management with SD-WAN Center ..................................... 249
Exercise B: Setup SD-WAN Center ............................................................................................... 251
Exercise C: SD-WAN Center Network Configuration ..................................................................... 261
Exercise D: Zero Touch Deployment ............................................................................................ 275
Module 7 (Advanced): Features Introduced in SD-WAN 9.3.0 ............................................. 280
Exercise E: Single-Step Upgrade bundle ....................................................................................... 282
Exercise F: Application Preferred WAN link.................................................................................. 288
Module 8 (Advanced): Features Introduced in SD-WAN 10.0 .............................................. 296
Exercise G: Upgrade environment to release 10.0 ........................................................................ 298
Exercise H: Application Route ..................................................................................................... 308
Exercise I: Scaled deployments with Multi-Region Mode ............................................................. 319
Exercise J: Enable Multi-Region on SD-WAN Center ..................................................................... 340
Exercise K: Centralized Licensing with SD-WAN Center................................................................. 352
Lab Guide Appendices ....................................................................................................... 363
Appendix A: Additional Resources and Information ..................................................................... 364
citrix.com 3
Lab Guide Overview
citrix.com 4
Lab Guide Overview
Objective
These lab exercises are designed to educate a user on how to properly setup a Citrix SD-WAN
environment. The environment is designed as a mock-up of a typical SD-WAN proof of concept (PoC),
with the aim to help the user build experience in deploying and troubleshooting real production
environments.
 Module 1 teaches design, installation, and configuration of SD-WAN, using virtual appliances.
 Module 2 focuses on the provisioning and change management procedures of SD-WAN.
 Module 3 walks-through troubleshooting techniques and reviews the SD-WAN overlay
functionality.
 Module 4 focuses on additional deployment modes and available WAN services for the SD-WAN
environment.
 Module 5 provides an option to pre-stage the environment to skip the previous modules and dive
into focused feature releases.
 Module 6 introduces the SD-WAN Center tool intended to be used as a central controller for the
entire SD-WAN environment.
 Modules 7+ dives into the more advanced features of Citrix SD-WAN, as introduced in iterations
of software releases.
Required Prerequisites
Basic knowledge of TCP/IP, routing protocols (Static, OSPF, BGP), and basic firewall operation.
Audience
Target
Partners
Citrix Internal Sales Engineers
Citrix Internal Consultants
Citrix Internal Technical Support
Lab Guide Conventions

Indicator Purpose
This symbol indicates attention must be paid to this step
Special note to offer advice or background information
Focuses attention on a part of the screen (R:255 G:102 B:0)
citrix.com 5
Lab Environment Details
The Citrix SD-WAN lab environment has been developed to focus on typical PoC deployments where SD-
WAN is deployed as an overlay to an existing underlay network that is not seeing optimal performance
and usage of WAN links. In this lab, the entire environment is laid out all on the same XenServer
hypervisor. SD-WAN virtual machines (SD-WAN VPX) instead of physical appliances are being utilized to
provide the link aggregation/bonding ability between distinct WAN links.
This lab consists of three branch offices and a single data center. The underlay network (without any
SD-WAN solution) is fully operational. The data center office (location NYC) portrays a very commonly
seen environment with two existing edge devices; a router facing the private MPLS WAN link, and
another router/firewall facing the Internet WAN link with a core routing the traffic accordingly across the
two available WAN links. Branch office traffic is delivered across the MPLS link, which is typically
saturated and Internet traffic is delivered across the Internet link which has proven to be unreliable. This
is a common customer scenario where Citrix SD-WAN is introduced to aid the network to securely and
reliably deliver traffic between sites.
The three branch offices are laid out as follows.
 London (LON) branch consists of an existing customer edge (CE) router handles only the MPLS
WAN link. The aim is to deploy SD-WAN as an overlay solution right behind the existing LON
customer edge (CE) router in transparent inline mode and to add two new WAN links Internet
4G/LTE to augment the MPLS bandwidth and operate as a backup/last resort link.
 Dallas (DAL) branch consists of an existing CE router with only a single private MPLS link. This
site will not have a SD-WAN solution purposely to showcase how NetScaler SD-WAN seamlessly
interoperates with the existing underlay network.
citrix.com 6
 San Jose (SJC) branch will be deployed as a new branch with NetScaler SD-WAN installed as
the edge/gateway solution, highlighting the benefit of SD-WAN as a router and firewall
replacement.
The lab exercises will showcase the flexibility of Citrix’s SD-WAN solution in the various deployment
modes and highlight how the solution can seamlessly be introduced in any network, either as a SD-WAN
overlay solution or as an edge device consolidating hardware of commonly found branch office
components.
Credentials
User Name Password Description
Training\Administrator Password1 Domain Administrator
admin password SD-WAN Standard Edition VPX
admin password SD-WAN Center
vyatta Password1 Vyatta Routers/Firewalls
Topology
Get a closer look at the topology using your web browser:
https://realtimeboard.com/app/board/o9J_k0Nik2k=/
It is critical to have a topology with this level of detail agreed upon by the network admin well in
advance to any on-site activity. Without it you run the risk of prolonging the onsite activity beyond
the desired timeline. Building an SD-WAN configuration becomes very simple with the network IP
address information is laid out well in advance.
citrix.com 7
Lab Scenario and Instructions
As a Sales Engineer, you have been tasked with introducing Citrix SD-WAN into a customer’s lab network
for a proof of concept. Also, in helping the customer better understanding the solution so that they are
fully equipped to best utilize the available features for their specific needs.
You should be aware that the Citrix’s SD-WAN solution consists of three editions that provide different
features and functionality:
 SD-WAN Standard Edition (available as physical or virtual appliances)

o Provides link aggregation and sub-second application resiliency by monitoring and
delivering traffic across multiple WAN links
 SD-WAN WANOP Edition (available as physical or virtual appliances)
o Provides WAN optimization techniques (TCP flow control, compression, deduplication)
across a single WAN link that is struggling to deliver applications without impacting end-
user experience
 SD-WAN Enterprise Edition (only available as physical appliance)
o Provides the link aggregation, sub-second resiliency, as well as WAN optimization
features in a single appliance for application delivery across multiple WAN links
Standard and WANOP Editions can be deployed back-to-back in a network and is equivalent to
Enterprise Edition. The only differences being throughput capabilities and single pane of management.
Since the foundation of SD-WAN technology begins with Standard Edition, which can easily be upgraded
to Enterprise Edition (on select models) through software and license upgrade, this lab will primarily
focus on SD-WAN Standard Edition (SD-WAN SE) features and functionality. What is learned with
Standard Edition is applicable to Enterprise Edition deployments.
If you would like to start the lab using a pre-built

configuration to jump right into the more advanced
features of SD-WAN, you can optionally skip ahead to
Module 5: Exercise A
citrix.com 8
Module 1 (Basic): SD-WAN
Installation and Configuration
citrix.com 9
Module 1 Overview
This module will lead you through the design, installation and configuration of the SD-WAN
solution.
It is critically important that before configuring the solution for any environment, you complete
the following pre-requisites:
1. Familiarize yourself with underlay network details

2. Identify the possible SD-WAN deployment mode for each site
3. Obtain all the needed IP address for both the management plane and the data plane
4. HAVE A COMPLETED SD-WAN NETWORK TOPOLOGY BEFORE BEGINNING
CONFIGURATION
citrix.com 10
Exercise 1: Understanding the Underlay Network
Overview
In this exercise, we will better understand the existing environment and the issues it has with
application delivery. This presents us with a before picture so that we can later compare with
the after picture when SD-WAN is up and running and have a better understanding of what SD-
WAN is doing to benefit the network.
In this exercise, you will:

 Understand the traffic flow of the underlay network and run performance tests.
Estimated time to complete this exercise: 20 Minutes
Step by Step Guidance

Step Action
1. Before jumping into any SD-WAN configuration, one always needs to properly
understand the existing (underlay) network and to fully understand the limitations of it
that require the need of a SD-WAN solution.
If the limitations of the underlay network are not properly understood, then
proving the SD-WAN functionality and meeting the success criteria laid out by the
customer to win the PoC becomes more difficult.
From the lab portal, log in to the lab session by launching your Student Desktop with the
Launch Lab button. You will need the latest Citrix Receiver installed (USA hyperlink -
https://www.citrix.com/go/receiver.html )
Once the HDX/ICA session connects to the Student Desktop, you can administer the
entire hypervisor environment through XenCenter using the provided “admin”
credentials (this can also be found in the lab portal). Also, from the Student Desktop,
you have connectivity to the management network and all the virtual machines in the
environment. Please make use of the lab topology to better orientate yourself within the
environment. Having the topology open throughout all the lab exercises will make for
better understanding of the lab and the SD-WAN solution.
https://realtimeboard.com/app/board/o9J_k0Nik2k=/
From the Student Desktop, first we are going open the XenCenter application and
connect to the XenServer hosting the environment.
2. If you encounter the Microsoft Windows pop-up, click the Restart Later option.
citrix.com 11
3. XenCenter on the Student Desktop will be set to automatically launch upon log in. If it is
not already running, begin by launching the XenCenter shortcut available on the
desktop.
4. In the Infrastructure pane, right-click on the “XenCenter” node, and select Add….
5. Log in using the “XenServer” credentials supplied via the lab web portal. Enter the IP
address of the server (192.168.10.5) and make sure that you use the username
“admin”, and use the password listed in the lab portal. Then click Add.
citrix.com 12
6. If a pop-up window appears. Close the Health Check Overview pop up window.
7. The environment will begin with some virtual power-on and others powered off. We will
systematically power on the virtual machines as we work through the labs.
In XenCenter, right-click the NYC_Server and LON_Client virtual machine and select
Start. Navigate to the Console tab to view the boot process.
8. First log in to the LON_Client VM, by clicking the “Send Ctrl+Alt+Del” button. Then
login with credentials TRAINING\Administrator (Password1).
citrix.com 13
9. Once logged in, we will launch a Command Prompt. In the Search, type in “cmd”.
10. Right click Command Prompt and select Run as Administrator.
11. Issuing the command “ipconfig” will show that this VM has two IP addresses; one
connected to the data network (172.70.1.28/24) which will have no gateway defined and
a second connected to the management network (192.168.10.28/24) of the lab. We will
use persistent routes on the VM to make sure traffic is going the correct direction.
citrix.com 14
12. Issuing the command “route print” will output some persistent routes that are pre-
configured on this virtual machine to route traffic appropriately through the correct
interface for management or data networks.
Any traffic destined for the NYC Data Center subnets (172.16.0.0/16) or any of the
remote sites (DAL 172.80.1.0/24, SJC 172.90.1.0/24) will be delivered out the data
interface to the 172.70.1.1 gateway owned by the CE router for the LON network.
We can test this by issuing a “tracert 172.16.10.12” command to run traceroute to the
NYC_Server virtual machine in the data center.
Use the topology to validate the hops displayed by the tracert command.
172.70.1.1 is the LON MPLS CE Router, 169.15.70.1 is the MPLS gateway on the
Provider Edge router, 169.15.50.2 is the NYC MPLS CE router, 172.16.20.2 is the NYC
Core, and 172.16.10.12 is the NYC Server host.
Any traffic destined for the management network will be directed out of the management
interface to the 192.168.10.1 lab router, not illustrated in the topology.
citrix.com 15
13. With IP connectivity confirmed between the LON_Client and NYC_Server, we can utilize
an iPerf tool to check the available bandwidth between these two site hosts.
On the LON_Client desktop you will find a jPerf application shortcut. Launch jPerf to
start the application.
1 2
14. With the jPerf application started, configure to Server Mode, then click the Run IPerf!
button. In server mode, the application will wait for any connecting clients to push traffic
to it using default TCP 5001.
2
1
citrix.com 16
15. On XenCenter, navigate to the Console tab of the NYC_Server virtual machine, and log
in with credentials TRAINING\Administrator (Password1). Click the “Send
Ctl+Alt+Del” button to get to the login page.
16. On the NYC_Server desktop, you should also find the jPerf shortcut to double-click to
launch. Once started, configure jPerf in Client Mode pointing to server IP address of
the LON_Client host (172.70.1.28), increase the transmit from 10 seconds to 1000
seconds. Then click the Run IPerf! button.
2 4
citrix.com 17
17. As the iperf tool generates traffic, you should see roughly 1.9 Mbps of available
bandwidth.
This would identify the first limitation of this “underlay” network. Based on the
number of users at the LON site, and the type of content that they pull down from
the NYC data center, 2Mbps is not enough bandwidth and typically the link is
congested, and user experience is poor.
18. With the iperf traffic still running, on the Student Desktop, open the chrome browser to
the saved hyperlink to the WANem management interface.
citrix.com 18
19. In the WANem interface, select Advanced Mode, then select eth1 from the drop-down
and click Start.
If you are unable to see the menu bar in the WANem GUI, then it is the resolution
of the ICA session and you either need to zoom in on the browser or increase the
size of the ICA session of the Student Desktop.
20. Eth1 and eth2 on the WANem represent the MPLS link, if we induce 100% packet loss,
we can bring down the MPLS to see the behavior of the underlay network.
Manually enter 100 in the Loss (%) input field, then click Apply settings.
citrix.com 19
21. Immediately after applying the settings, and inducing an outage on the MPLS link, the
iperf connection should stop throughput, and the TCP port 5100 should eventually fail.
This would identify the second limitation of this “underlay” network. Network
outage of the MPLS link results in downtime for this LON branch. Some customer
networks may have BGP routes configured to take secondary paths through VPN
tunnels across public internet links, if available, but even in those underlay networks,
fail-over can take several seconds and the network disruption is not sustainable for
business during production hours.
This is generally enough for most customer to start thinking about the benefits the Citrix
SD-WAN solution can provide:
 Ability to make use of multiple WAN links, aggregate all the available
bandwidth to provide a thicker virtualized WAN for delivery.
 Sub-second fail-over ability between WAN links with zero impact to the end
user experience.
Not all SD-WAN solutions in the market can provide this specific ability which we will
highlight in later exercises.
citrix.com 20
22. Before beginning the next exercise, restore the loss on the WANem to 0 for eth1 and
apply settings.
This WANem browser tends to timeout quickly. So, refresh the browser and make
sure the changes have been applied.
Exercise Summary
In this exercise, we assessed the underlay network and identified some limitations that could be
addressed by implementing the SD-WAN solution.
citrix.com 21
Exercise 2: Understand Overlay Design (NYC, LON, DAL,
SJC sites)
Overview
Before beginning any configuration of the SD-WAN solution, the customer must first agree upon
the desired deployment mode on a per-site basis. The available deployment modes allow the
solution to seamless be deployed in any network or physically be a replacement for existing
network gear that is quickly becoming obsolete and not cost-effective to manage as individual
solutions. Citrix SD-WAN can simply be deployed on the LAN side of existing Routers,
Firewalls, VPNs and intercept traffic and make use of its overlay technology to delivery across
any available WAN paths with UDP tunnels created between two partner SD-WAN devices. In
some scenarios, SD-WAN can replace your existing Router, Firewall, VPN and encompass all
those technologies as a single solution and be placed at the WAN edge of the network and
provide a single pane of management for all components. This can scientifically reduce the cost
of maintaining and administrating individual network components, especially if the consolidation
of hardware into SD-WAN is considered across numerous branch offices.

 Identify the desired deployment mode at the NYC datacenter
 Identify the desired deployment mode for the remote office branches (LON, DAL, SJC)

Step Action
1. Before jumping into any SD-WAN configuration, one always needs to properly
understand what the existing (underlay) network looks and the existing components that
the solution needs to interoperate with. Here is a link for better viewing of the underlay
topology: https://realtimeboard.com/app/board/o9J_k0WiUMY=/
Obtaining a topology of customer’s network with this level of detail would be ideal
before discussions of SD-WAN implementation begin.
citrix.com 22
2. In the above topology, the existing network consists of a primary active MPLS link that
provides connectivity from the remote offices (LON and DAL) to the data center (NYC)
for application and data access. The NYC data center also has an internet link, which is
used for internet access for local users as well as traffic that is backhauled from the
remote offices through the saturated MPLS link. Most customers introduce internet links
at the branch offices and setup VPN tunnels across public infrastructure as a fail-over
mechanism should the primary MPLS link go down.
This very common network design but has its limitations which are outlined briefly here.
Limitations of the underlay network:
 The NYC and LON office MPLS link are limited in capacity and at times the
users accessing applications and data from the datacenter are saturating the
MPLS circuit causing poor application experience across all users.
 The internet links are underutilized at the NYC and any remote offices with
internet links for fail-over. Internet links are used in a standby/unused state
when providing connectivity between sites.
 In the event of MPLS failure and if additional links are available, routing is used
to converge traffic over the Internet VPN link, which takes several seconds to
accomplish and causes disruption in the network for all sessions to be re-
established across the newly available path.
This customer is looking for an SD-WAN solution that can make better use of the
available WAN links, add additional internet links where needed in an attempt to
address the application delivery performance issues seen throughout the existing
network.
3. In the past network performance issues have primarily been addressed by increasing
the capacity of the single link or by introducing WAN optimization technology to better
utilize single links that are causing performance issues due to congestion, latency and
loss. Citrix SD-WAN WANOP Edition is still a viable solution and can provide benefit to
help single link environments. However, newer technology is available in SD-WAN
Standard Edition (SE) that provides a more advanced solution that better address
network performance issues and adds much more features and functionality to simplify
and automate the network.
In this lab scenario, the customer is amicable to adding additional WAN links, so
Standard Edition is the ideal solution to use.
citrix.com 23
4. From our understanding of the Citrix SD-WAN technology, we can pick and choose the
deployment mode that best fits the existing network. Each site’s selected SD-WAN
deployment mode is independent of other sites. Typically, it is recommended to deploy
SD-WAN appliances in Virtual Inline mode at the Data Center location with a High
Availability pair, and Inline or Gateway/Edge mode at the branch offices. Each
deployment mode has its advantages and requirements from the underlay network to
help accomplish SD-WAN’s Virtual Path connectivity. SD-WAN also provides additional
flexibility in providing mixed mode deployments allowing one deployment mode from the
perspective of one WAN link and using a different deployment mode from the
perspective of another WAN link. Interface groups can be configured to identify
commonality between physical interfaces but can also be used to identify virtual
interfaces so that a single interface can manage multiple VLANs or be used to
communicate with multiple WAN links that may be further upstream in the network.
We will focus on commonly seen deployments in the subsequent exercises, but
keep in mind that deployment examples extend beyond what is exercised in this
lab.
citrix.com 24
5. SD-WAN SE can optionally be deployed in out-of-path or virtual inline (one-arm) mode
in networks where the solution needs to be introduced gradually and only see targeted
traffic. This is typical in Data Centers where there is less likelihood of a maintenance
window in order to physically cable the solution directly in path.
Virtual Inline Mode
Or the solution can be deployed directly in the path of traffic in inline mode. Branch
offices have a higher chance of a maintenance window to sustain the disruption caused
by physically cabling the solution in path, directly behind existing edge devices.
Inline Mode
For new sites, or sites that are being rearchitected, the solution can be deployed in edge
(gateway) mode, directly terminating the WAN links and replacing the functions of
existing edge devices.
Edge Mode
citrix.com 25
6. Let us start by focusing first on the existing NYC data center topology. From the
previously described deployment modes, we have multiple options of how we can best
integrate Citrix SD-WAN.
Deployment options include (recommended from top down):

1. A pair of SD-WAN appliances deployed in Virtual Inline mode in High Availability
(HA) pair, hanging off the core or the edge devices
2. A single SD-WAN appliance deployed in Virtual Inline mode, hanging off the
core or the edge devices
3. A High Availability pair of SD-WAN appliances deployed in Inline mode between
the edge devices and the core (additional switch hardware may be required)
4. A single SD-WAN appliance deployed in Inline mode between edge devices and
the core, operating in fail-to-wire across both links
5. Single SD-WAN appliance deployed in gateway mode acting as the new edge
device, replacing both existing edge devices (most customers are not
comfortable with this at their data center site due to the lack of one or another
feature capabilities in Routing, Firewall, or SD-WAN)
In this lab, in the NYC data center network we will step through the first
recommended deployment mode; Virtual Inline Mode off the Core. Most customer
agree to this method of deployment at the data center because it’s the least intrusive
deployment which provides the highest availability of the SD-WAN overlay. Redirecting
traffic to the Virtual Inline deployed SD-WAN include the following:
 Dynamic Routing (OSPF, BGP)
 Policy Based Routing (PBR) with IP SLA

For this lab we will be using Dynamic Routing to get traffic redirected to the SD-WAN
devices deployed in HA pair.
citrix.com 26
7. With the deployment mode selected for the data center site, the next task is to lay out
the overlay topology for SD-WAN solution, which includes interfaces to use and IP
addresses. We could have optionally deployed the HA pair of SD-WAN devices directly
off each edge devices, but would require additional cabling between each SD-WAN to
each edge device. A cleaner approach would be to deploy the HA pair off the single
core router and configure the core router to route accordingly between the two SD-WAN
devices and the two edge devices. The following tasks need to be completed with the
help of the network admin before beginning any SD-WAN overlay design for Virtual
Inline HA deployments.
1. Identify the interface and subnet to be used on the network router to connect to
the SD-WAN HA pair.
(The network admin identifies eth3 on the core router and created a new
172.16.40.0/24 subnet. The subnet is new, so we are okay to make use of any
IP address without worry of any duplicate IP addresses.)
2. Identify the switch hardware between the core router and the SD-WAN HA pair.
(The network admin VLANs out 3 ports on an existing switch to connect eth3
from the core router and interface eth1 from each SD-WAN)
3. To be on the safe side, a maintenance window will be required for configuration
changes to the core router, since mistakes can be business impacting and this is
a data center network.
(The network admin can submit a request, which usually takes a few days if not
weeks to identify the timeframe)
4. Identify needed rack space for the SD-WAN device to be deployed in HA.
(Depending on the model appliance, which can be 1U or 2U, the HA pair could
need up to 4 adjacent rack spaces)
5. Identify two available IP addresses on the management network, one for each
SD-WAN device.
(The network admin has identified two available IP address 192.168.10.23 and
192.168.10.24 /24 subnet with gateway 192.168.10.1)
6. Identify the routing protocols in the network that can leveraged for traffic
redirection.
(The network admin will allow a new OSPF Area 3 to be created with loopback
address 20.0.1.14 for the SD-WAN HA pair)
It is important to get a full understanding of the underlay network and get the
Network Admin to agree upon the SD-WAN overlay design prior to any on-site
activity. As you can see from the five simple tasks laid out above, it could take time and
effort for the network admin to prep the environment and get it ready for the SD-WAN
implementation, so again it is important to understand that the SD-WAN topology should
be agreed upon and designed before scheduling any on-site activities.
citrix.com 27
8. Based on the provided detail from the network admin, we can begin design of our Citrix
SD-WAN overlay network for the NYC site. We have identified the following interfaces,
subnets and IP addresses to be used for the SD-WAN HA pair of devices.
 Interface eth3 on the core router was available and a new subnet was introduced
for that interface 172.16.40.0/24
 Two WAN links are available at this site, so we will need two Virtual IP (VIP)
addresses for the SD-WAN configuration, one for each WAN link: 172.16.40.2 &
172.16.40.3 (these IP addresses will be shared by both HA pair)
 The SD-WAN will be deployed in HA, so another set of IP addresses will be

needed for the heartbeat between the two devices: 172.16.40.23, 172.16.40.24
(these will not be shared and will be unique per device)
 Each SD-WAN appliance will require a unique IP address on the management

network: 192.168.10.23 & 192.168.10.24 (these will not be shared and will be
unique per device)
 SD-WAN will utilize OSPF Area 3 with loopback address 20.0.1.14 for traffic
redirection to the Virtual Inline deployment
citrix.com 28
9. With the data center site taken care of, lets focus on the first branch office site located in
London.
In the LON Branch, a single edge router is handling the 2Mbps/2Mbps MPLS link for the
flat 172.70.1.0/24 network. This is an important site for the customer and is expecting to
increase the number of employees for this branch as well as the demand for bandwidth
with video and graphics-based applications. There were already plans to add a
broadband internet link for this site to act as a backup VPN connection to the data
center, but with SD-WAN we will make more effective use of the newly added link and
we can suggest adding a third 4G/LTE WAN link to provide further resiliency and
connectivity to this very important branch. With the implementation of the Citrix SD-
WAN technology we are also eliminating the need to bump up the capacity of the 2Mbps
MPLS link.
citrix.com 29
10. With the suggestion above, the customer is onboard with deploying the SD-WAN in
most convenient deployment mode possible. The one requirement is to allow the
network to fallback to existing operation if SD-WAN stops operating.
Similar to the data center site, we can choose to deploy in Virtual Inline mode, and allow
the network to fallback to existing operation using dynamic routing or policy based
routing with IP SLA.
The recommendation for this branch is to deploy Inline mode, directly behind the edge
router. This provides easy implementation and configuration, as well as allows for
fallback to the existing operation of the underlay network using the fail-to-wire bypass
bridge pairs on the SD-WAN.
The newly introduced Internet and 4G/LTE link can be directly terminated to the SD-
WAN device using ethernet handoff. When the SD-WAN is active, the MPLS and
Internet link can be used simultaneously, with the 4G link configured in standby mode.
When the SD-WAN is not active, fail-to-wire will the network to fall back to the exact
same state it was in before SD-WAN was introduced. This allows for minimal
configuration and change to the existing infrastructure.
In this lab, the SD-WAN VPX will be deployed in Inline mode to make the student
aware of the possible deployment modes, however in production environments it
is recommended to utilize a physical appliance with built-in bypass cards for inline
deployments.
citrix.com 30
11. Moving onto the second branch office in Dallas, the customer has identified that there is
very little number of employees in this office and existing 2Mbps MPLS link is sufficient
for their bandwidth needs. Likewise, connectivity for the site is not paramount and
business if not impacted if the site has a complete outage due to the single MPLS link.
The only requirement for this site is to allow for connectivity to new and existing sites
that will have SD-WAN deployment. Which can be accomplished with the SD-WAN
Intranet Service.
The DAL site will be deployed with no SD-WAN solution.
12. Lastly, the customer is interested in introducing a new branch office in San Jose and
would like to take the opportunity to add SD-WAN technology to serve as a replacement
for an edge router and firewall.
For this SJC site, we will use Edge mode deployment for SD-WAN and directly
terminate the MPLS and Internet link to the SD-WAN. The LAN interface of the device,
in this deployment mode, will serve as the Gateway IP address of the LAN subnet.
citrix.com 31
13. With the deployment modes confirmed for each site, the SD-WAN PoC will be judged by
the customer on the following success criteria:
 Supply more bandwidth to the remote offices (LON, SJC), than what is currently
limited by the single MPLS link
 Network resiliency to sustain business operation even during outage of WAN
links
 Seamless interoperability with non-SDWAN deployed sites (DAL)
 Routing and Firewall hardware consolidation for remote offices (SJC)
With sign-off by the customer, confirmation of available resources to accomplish the
networking implementation of the SD-WAN solution, we will begin installation and
configuration in the next exercise.
Certain circumstances my prevent proper preparation before jumping into a SD-
WAN PoC, however timeline and even rate of success are dramatically impacted if
the basics of the existing underlay network are not understood, the deployment modes
of the SD-WAN devices are not discussed, and the success criteria is not defined
agreed upon by the customer.
Exercise Summary
In this exercise we designed an SD-WAN deployment to address the needs for the NYC data
center site, as well as the needs for the three branch offices (LON, DAL, SJC) and created a
network topology to reference when building the configuration.
citrix.com 32
Exercise 3: Start lab with administrative tasks on the MCN
Overview
With our SD-WAN overlay topology detailed, we will begin performing required administrative
tasks on the appliance. Each SD-WAN deployment will require the following basic task to be
completed. For this exercise we will focus on the data center SD-WAN.
 IP address for web browser access
 Set Date/Time
 Set license
 Switch Console

Step Action
1. In XenCenter, right-click the NYC_SDWAN_SE1 virtual machine and select Start.
Navigate to the Console tab to view the boot process.
For more information about VPX, refer to Citrix documentation: (USA hyperlink)
https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/virtual-wan-vpx.html
citrix.com 33
2. While on the Console tab of the NYC_SDWAN_SE1 VM, after the VM has fully booted
(wait 2-3 minutes), press the Enter key on your keyboard to get to the cbvw login:
prompt
DHCP is generally enabled by default on physical devices and the management
interface will respond on the assignment IP. If a DHCP server is not available on
the connected management interface of SD-WAN, then after booting the device will
respond to a default IP address of 192.168.100.1.
In this lab environment the VM eth0 interface is connected to the “Internal” network for
management and have set aside the IP address of 192.168.10.23 per the design
topology.
Log in with credential admin/password.
3. Once logged in to the factory default SD-WAN VPX, to configure a new management
IP, enter the following command:
 management_ip
Optimally use the tab button on the keyboard to help complete commands in in the
connected console.
You will be prompted with a menu with instructions on how to configure an IP address
for web interface access.
citrix.com 34
4. While in the “set_management_ip” menu, type the following command to IP address
the NYC_SDWAN_SE1 VPX. We will use 192.168.10.23 for this particular VM.
 set interface 192.168.10.23 255.255.255.0 192.168.10.1
After striking the Enter key you will notice that the menu will update with this message;
“the following changes have been staged:”
Enter the “apply” command to accept the changes and enter “y” to acknowledge the
change.
citrix.com 35
5. On the Student Desktop, open a Chrome browser and navigate to the configured IP
address to access the web interface of the NYC_SDWAN_SE1 VPX. A shortcut link for
Chrome will be available on the desktop. A bookmark link to the web interface of
NYC_SDWAN_SE1 is also pre-staged.
2
1
The Release Notes capture supported browsers. It is best practice to use

supported browsers only. Generally, if you encounter GUI related issues with SD-
WAN it is recommended to first clear the cached history then try another browser.
https://www.citrix.com/content/dam/citrix/en_us/documents/downloads/cloudbridge/Net
Scaler-SD-WAN-9.1.0-Release-Notes.pdf
Supported Browsers
The SD-WAN release 9.1 is supported on the following browser versions:
o Mozilla Firefox 35.0+ (Recommended version 43.x)
o Google Chrome 40.0+ (Recommended version 49.x)
Supported browsers must have cookies enabled, and JavaScript installed and enabled.
citrix.com 36
6. If you get a “Your connection is not private” message in the browser, proceed by
clicking “Advanced” and “Proceed to 192.168.10.23”, then confirm security exception.
The reason for this browser alert is that the factory shipped certificates are not
trusted by the browser because the issuer certificate is unknown.
Once on the SD-WAN login landing page, type the following credentials and then click
Login:
User Name: admin
Password: password
citrix.com 37
7. Once logged in to the NYC_SDWAN_SE1 VPX, on the landing page (Dashboard) you
will observe One Touch Start option to quickly import existing configuration packages,
which will be covered in a later exercise, and the System Status which indicates the
current state of the machine; “Citrix Virtual WAN Service is currently disabled.”
8. Next, we will need to run through a few administrative items that are typically done
when configuring an SD-WAN for the first time:
 Set Date/Time
 Set license
 Promote to MCN
 Change GUI timeout
9. The first administrative task is to set the correct system data and time.
In the NYC_SDWAN_SE1 web interface, navigate to Configuration > System
Maintenance > Date/Time Settings.
citrix.com 38
10. Scroll down to Timezone Settings pane, select “America/New_York” from the time
zone dropdown, and then click the Change Timezone button.
Click OK to accept the change.
Verify the Date/Time after the updated change.
For proper log file timestamp correlation, it is important that the devices be
configured with correct data/time.
11. Next, navigate to the Appliance Setting > Licensing page and select the Remote
license radio button.
Your lab virtual machines may already be licensed locally.
citrix.com 39
12. A remote license server is setup for the lab for easier licensing of Citrix products. In the
Configure Licensing Server pane, input IP address (10.0.76.37) and default port
(27000). Also, select the 50Mbps (V50VW) license file from the Model drop-down.
Then click Apply Settings.
Validate that the VPX is licenses before proceeding.
Common license installation failures include mistyping the host id of the system
when generating the license file in the license portal, or issuing the license file for
the incorrect edition of SD-WAN. If you do not see “VW”, which stands for Virtual WAN,
in the license file name, then assume it’s a WANOP Edition license and not the needed
Standard Edition license.
13. The next administrative task is to change the console of the SD-WAN VPX to the
Master Control Node so that we can enable network-wide SD-WAN configuration ability
on this Data Center SD-WAN node.
Navigate to Appliance Settings > Administrator Interface, and click the
Miscellaneous tab.
1
2
citrix.com 40
14. In the Miscellaneous tab, click the Switch Console button to switch from default Client
node to the Master Control Node (MCN) console.
Since the majority of SD-WAN devices are expected to be deployed in remote
offices as Client nodes, and only one MCN will be assigned per SD-WAN
environment, the devices are all defaulted as Client Console.
Then click OK to confirm the switch to the MCN Console. This will require the GUI to
automatically reload in the browser window.
Make sure you are only changing the head-end NYC_SDWAN_SE1 (192.168.10.23) to
be promoted as the MCN. The remote SD-WAN devices stay as a client nodes.
15. When you return to the login screen, use the admin/password credentials to log in
again.
citrix.com 41
16. The menu options available under Configuration > Virtual WAN are different between
the device that is enabled as Master Control Node (MCN) and the devices that are left
default Client nodes:
 Only the MCN has ability to configure network changes, not only for itself but
also for all remote appliances. This ability is also available on SD-WAN Center,
which we will explore in a later exercise.
This is a screen capture of the datacenter SD-WAN VPX Configuration
> Virtual WAN pane, showcasing the difference in console modes
 Client nodes only allow access to managing local administration processes and
report on flows local to the device.
This is a screen capture of the remote SD-WAN VPX Configuration >
Virtual WAN pane, showcasing the difference in console modes
There should only be one device promoted to MCN in an SD-WAN environment.

The Configuration Editor and Change Management are features that should only
be available on the MCN and not any client node.
17. Increase timeout.
citrix.com 42
Exercise Summary
In this exercise, we identified our head-end SD-WAN device, promoted it to Master Control
Node and performed basic administrative tasks.
citrix.com 43
Exercise 4: Configuration Editor – Build MCN Node
Overview
With the Data Center SD-WAN enabled for MCN Console, the Configuration Editor will be used
to start building the SD-WAN overlay for the data center site.
 Use the Configuration Editor Basic mode to build the NYC SD-WAN configuration

Step Action
1. While logged in to the SD-WAN we recently promoted to MCN, we can now start
building the configuration for our SD-WAN environment.
From the NYC_SDWAN_SE1 VPX web interface, navigate to Configuration > Virtual
WAN > Configuration Editor:
The Configuration Editor is where the configuration for all the SD-WAN nodes, for
both the local and remote sites, are centrally configured.
Please reference the network topology as you build the configuration. First, we will step
through building the configuration for the NYC SD-WAN network. Once that is complete
we will then do the same for the branch office networks.
2. First create a new configuration file by clicking the “New” button
citrix.com 44
3. A new file named “Untitled_0” will be created. The configuration editor has two modes:
 Basic – Basic mode enables for quick creation of basic SD-WAN interface and
WAN link settings to create and establish a Virtual Path between nodes.
 Advanced – Advanced mode enables configuraiton of advanced features of SD-

WAN, which include Firewall, Routing and Quality of Service policies.
Move the Network Map out of the way by clicking the Show/hide the Network Map
button.
4. With Basic tab highlighted, select Global view. In Global view, we can define the global
encryption mode settings, and create Service Provider WAN link templates that can be
called upon when creating our site notes.
citrix.com 45
5. With Virtual WAN Network Settings highlighted, select the edit (pen) icon to see the
available encryption mode settings.
The Network Encryption Mode settings defines the algorithm used for all encrypted
Paths in the SD-WAN device. These settings do not apply for non-encrypted paths.
Click Cancel, since for this lab we will keep default settings.
Default AES-128 encryption is used in marketing the performance numbers on

the NetScaler SD-WAN Datasheet, enabling more advanced levels of encryption
will impact those reported performance numbers. For more detail on SD-WAN security
best practices, please refer to the following: https://docs.citrix.com/en-us/netscaler-sd-
wan/9-3/security-best-practices.html
citrix.com 46
6. Under the Basic tab, let’s change the view to Sites.
Here is where will build our NYC, LON, DAL, etc. sites, and the inputs for interfaces and
WAN links will determine the deployment mode. So we must first reference our design
topology.
citrix.com 47
7. We will begin by defining the NYC site, and start building the SD-WAN configuration for
NYC_SDWAN_SE1.
Click the + Site button.
Then populate the Add Site pop-up window with the following.
 Site Name: NYC

 Model: CBVPX
 Mode: Primary MCN
 Enable Site as Intermediate Node: unchecked
Click Add to accept the settings.
1
2
The selected model should be accurate for each site. The interfaces for each model
are unique, reference the Citrix SD-WAN Data Sheet, and based on the selection,
the interface options may be different when configuring the site. For this lab we are
using virtual appliances, so we will select CBVPX. CBVPXL images assume more
memory and CPUs to achieve higher virtual paths (128) versus the non-L images (16), it
does not increase the throughout specifications.
This site will also be a Primary MCN node, which is not a reference to High
Availability pair, but rather an option to have a second site (Secondary MCN) pick
up responsibility for Configuration Editor and Change Management if the primary MCN
or MCN HA pair site goes offline.
Enabling this site as an Intermediate Node, will allow this node to determine when it
is appropriate for two client nodes to bring up or tear down their own Dynamic
Virtual Path. Reference “Dynamic Paths for Branch to Branch Communication” for more
detail: https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/use-cases-sd-wan-virtual-
routing/use-case-dynamic-paths-branch-branch-communication.html. We will look
further into this in a later exercise.
citrix.com 48
8. The right pane for the newly added site will provide insight as to what components are
required in Basic mode to complete the configuration for this site. Those include:
 Interfaces
 WAN Links
 Static Routes
citrix.com 49
9. For this NYC site, begin by clicking the add (+) icon next to Interfaces in order to define
the SD-WAN interface to use for the NYC site.
In the add interface pop-up window, we will reference the design topology to identify
which data interface on this SD-WAN VPX will be used. As per our topology we will
only need one interface to deploy in one-arm mode, and we will configure the following
settings with that in mind:
 Ethernet Interfaces: 1
 Bypass Mode: Fail-to-Block
 Security: Trusted
Click the add (+) icon next to VLANs and add the following
 VLAN ID: 0
 IP Address / Prefix: 172.16.40.2/24
Click the Add button to accept the settings.
Bypass mode will be fail-to-block since we are only leveraging one interface and
not a bypass pair of interfaces
Security mode is set to trusted because we are not directly connecting interface 1
to the public internet.
Default VLAN 0 is used because we do not need to adapt to any VLAN
configuration on the underlay network, and also one of the Virtual IP (VIP)
addresses is selected as the interface IP (VIP2 172.16.40.2 for the MPLS WAN link).
We will add the second INET WAN link later.
citrix.com 50
10. Next we will define the first WAN link for the NYC site.
Begin by clicking the add (+) icon for WAN Links.
In the Add Site Settings pop-up window, enter the following:
 WAN Link: NYC_MPLS

 Access Type: Private Intranet
 Virtual Interface: E1Vlan0
 IP Address: 172.16.40.2 (auto populated)
 Gateway: 172.16.40.1
 Rate Unit: Mbps (default)
 LAN to WAN Physical Rate: 10
 WAN to LAN Physical Rate: 10
WAN Link templates are generally reserved for creation of branch office / client
node sites. We are also using this basic mode configuration editor to quickly
build a path. We will need to create the second WAN link later. For private MPLS links
that do not have MPLS queues operating select Private Intranet as the Acces Type. For
private MPLS links that have MPLS Queues operating, then Private MPLS should be
selected.
citrix.com 51
11. The last item for the site creation is Static Routes. In our design topology, we can
identify that we do have some backend subnets that we would like to advertise in our
SD-WAN overlay. These subnets (172.16.10.0/24, 172.16.11.0/24 and 172.16.12.1/24)
can initially be defined statically, then later updated to be learned dynamically through
OSPF with advanced mode configuration editor. Each of these data center subnets is
reachable through the core router so we define the gateway as the IP (172.16.40.1)
assigned to interface connected to the SD-WAN device.
Click the add (+) icon to add a new static route for the NYC site.
In the Add Site Settings pop-up window, add the first subnet (connected to eth0 of the
NYC_Core_Rtr) by populate the following, then clicking Add to save the settings:
 Networking IP Address/Prefix: 172.16.10.0/24

 Gateway IP Address: 172.16.40.1
Add the second static route for the second subnet (connected to eth1 of the
NYC_LAN_Rtr), then click Add to save the settings:

Add a second static route for the second subnet (connected to eth2 of the
NYC_LAN_Rtr), then click Add to save the settings:
citrix.com 52
By defining these routes in the SD-WAN configuration, we are enabling remote

SD-WAN devices to distinguish that these subnets are local routes in the NYC
SD-WAN and if delivered to the NYC SD-WAN via the Virtual Path, the NYC SD-WAN
will successfully deliver to the subnet by handing off to the defined gateway.
12. We are now fully complete with the configuration of the NYC node. We can now save
the configuration so that we do not lose our work.
At the top of the configuration editor, click the Save As button.
In the Save As pop-up window, give the file a name (e.g. Exercise4), and click the Save
button.
Exercise Summary
In this exercise, we utilized the configuration editor in basic mode to quickly build the
configuration specific to the MPLS link of the first SD-WAN device in the NYC site.
citrix.com 53
Exercise 5: Configuration Editor - Build Client Nodes (LON
and SJC)
Overview
The configuration editor is a tool that allows for central configuration of the SD-WAN
environment. We have already built our head-end NYC MCN node in the previous exercise,
now we will continue with building the configuration for the first branch site.
 Use the Configuration Editor Basic mode to build the LON SD-WAN configuration
 Use the Configuration Editor Basic mode to build the SJC SD-WAN configuration

Step Action
1. While still logged in to the MCN Configuration Editor and the current configuration file
still open, we will now create another site.
Reference the topology to the LON site to identify what needs to be built.
Since we are in basic mode, we will only define the first WAN link to establish a path to
this site.
citrix.com 54
2. Click the + Site button to add the next site, and to continue building the configuration.
3. In the Add Site pop-up window, populate the following:
 Site Name: LON

 Model: CBVPX
 Model: client (default)
 Enable Dynamic Virtual Paths: unchecked
Just like the MCN, client nodes can also optionally be enabled as Intermediate
Node and serve as the intermediary between two other client nodes for Dynamic
Virtual Path establishment. In this lab, we want the LON, DAL, and SJC branches to
have dynamic virtual paths established through communication of the MCN in the NYC
site, we will enable that feature in a later lab exercise.
citrix.com 55
4. As per the design topology, we will deploy the SD-WAN in Inline mode behind the
existing edge router. Ethernet 1 and 2 will be used in fail-to-wire mode.
With the new LON site selected, click the add (+) button to add Interfaces:
In the add interface pop-up window, enter the following detail:
 Ethernet Interfaces: 1 & 2

 VLAN ID: 0
Please take special note that this environment leverages virtual appliances (SD-
WAN VPX, which does not have fail-to-wire capabilities. Only the physical SD-
WAN appliance have fail-to-wire hardware interfaces and are recommended to be
deployed in Inline Mode. Fail-to-wire enables two network interfaces on an appliance to
be bridged together and enables connectivity through the appliance in the event of
power or software failure by closing a relay between the two interfaces. In short, fail-to-
wire is a method to protect against complete site outage when an appliance is directly
placed in the path of all incoming and outgoing traffic. If the desired functionality is to
keep the SD-WAN service up and running regardless of the failure point, SD-WAN can
optionally be deployed in High Availability pairs.
With the limitation of no fail-to-wire capabilities on the VPX you can still deploy in Inline
Mode, but the recommendation would be to deploy VPX in Virtual Inline Mode (Policy
Based Routing). Alternatively, VPX can also be deployed in Edge Mode making it the
default gateway for its respective site.
citrix.com 56
5. With the Interfaces defined, click add (+) to add WAN Links.
In the Add Site Setting pop-up window, enter the following detail for the Internet Link.
 WAN Link: LON_MPLS

 Virtual Interfaces: E1E2Vlan0
 LAN to WAN Physical rate: 2
 WAN to LAN Physical rate: 2
Click Add to accept the changes.
citrix.com 57
6. The LON office is flat network, so no Static Routes are required, so that can be skipped
for this site.
The basic configuration is complete, however you may notice one Audit warning at the
bottom of the screen.
This is to be expected since we are utilizing a virtual appliance with no bridge pair, and it
is typically not intended to be used in inline mode deployment. This can be correct by
enabling Source MAC learning as the message indicates.
Navigate to Advanced mode in the Configuration Editor and drill down to Sites > LON >
Interface Groups, to see where the Audit warning is occurring.
The Configuration Editor is built with configuration checking intelligence that will
help validate proper configuration. Orange indicators alert to potential
configuration mistakes, but will not stop an admin from pushing the configuration. Red
indicators however are more significant and need to be addressed before the MCN will
allow the configuration to be pushed out into the network.
The web interface is interactive. Hovering your mouse cursor over any of the fields,
including the audit errors will supply a pop-up with more detail to the warning.
citrix.com 58
7. Based on the audit error, lets first add the interfaces as Bridge Pairs.
Click the add (+) icon and select interface 1 and 2 in the drop downs.
1
2
Click Apply to save the changes.

8. Navigate to Basic Settings under the LON node, and click the pen icon to edit. Then
Enable Source MAC Learning and click Apply to save the setting.
2
3
citrix.com 59
9. Next, we are going to build the configuraion for the first WAN link of the SJC branch
office.
Navigate back to Basic mode, then click the + Site button to add the next site, and to
continue building the configuration.
citrix.com 60
10. In the Add Site pop-up window, populate the following:
 Site Name: SJC

 Model: CBVPX
 Model: client (default)
 Enable Dynamic Virtual Paths: unchecked
citrix.com 61
11. As per the design topology, we will deploy the SD-WAN in Edge mode as the edge
device. Ethernet 2 will connect to the MPLS WAN link.
With the new SJC site selected, click the add (+) button to add Interfaces:
 VLAN ID: 0
citrix.com 62
12. With the Interface defined, click add (+) to add WAN Links.
In the Add Site Setting pop-up window, enter the following detail for the Internet Link.
 WAN Link: SJC_MPLS

 Virtual Interfaces: E2Vlan0
 LAN to WAN Physical rate: 2
 WAN to LAN Physical rate: 2
citrix.com 63
13. Lastly, since the SJC_SDWAN is deployed in Edge mode, we can add another Interface
group to associated with the LAN.
With the new SJC site selected, click the add (+) button to add Interfaces:
 VLAN ID: 0
14. With the three sites build out, with one WAN link each defined, and the Configuration
Editor is alerting of zero audit warnings, the basic configuration is complete.
Click Save As, and save the new configuration file as “Exercise 5”
Exercise Summary
In this exercise, we utilized the configuration editor to build the configuration specific to the
desired deployment of the branch SD-WAN devices for LON and SJC.
citrix.com 64
Module 2 (Basic): SD-WAN
Provisioning and Change
Management
citrix.com 65
Module 2 Overview
This module will continue the deployment process for Citrix SD-WAN solution. It also highlights
that the MCN is used to build configuration and software packages for each newly added SD-
WAN device in the network, and with Virtual Paths available how that MCN can be used to
centrally push updates to configuration down to client nodes without having to directly make
configuration changes to each client node in the network.
It is critically important that before configuring the Citrix SD-WAN solution for any environment,
you complete the following pre-requisites:
1. Create the desired network topology

2. Identify the deployment mode and obtain all IP address for both the management plane
and the data plane
citrix.com 66
Exercise 6: Provisioning on the MCN (NYC)
Overview
Now that the basic configuration file with one link defined for each site is complete and there are
zero audit warnings, it is time to apply this configuration first to the MCN and then manually
apply the configuration to the remote branch in the next exercise.

 Provision the configuration using Change Management

Step Action
1. While still logged in to the MCN (NYC_SDWAN_SE1) Configuration Editor, we will
export the configuration.
First, make sure you have saved the configuration, then click the Export….button at the
top of the Configuration Editor.
2. Make sure Change Management Inbox is selected in the Destination drop-down list
and then click Export.
The “File download” option gives the ability to save the file to a local drive. This is
useful to backup saved configuration, where the Import button can be used on any
MCN system to import saved configurations.
The configuration file is now ready and waiting in the Change Management page of the
MCN for provisioning.
citrix.com 67
3. Navigate to Configuration > Virtual WAN > Change Management to start the MCN
provisioning process.
4. The Change Management page is specially designed for large scale SD-WAN
configuration and software update workflow.
The left pane outlines the workflow of each step:
 Step 1 “Change Preparation” is where you select the new desired configuration
file exported from the previous step, as well as upload of any new software
packages specific to appliances that are selected in the configuration.
 Step 2 “Appliance Staging” allows for a systematic push of the configuration to

the SD-WAN appliances already connected to the MCN node for single click
update of configuration and software (The table at the bottom of the page
outlines connected client nodes to the SD-WAN environment. Sites listed as
“Note Connected” require local web interface access to manually upload the
config/software packages for the first time).
 Step 3 “Activation” is where all the nodes are ready with their pushed
configuration and simultaneously be failed over from the old config/software to
the new config/software for minimal disruption to the network.
To begin Step 1 “Change Preparation” process, click the Begin button.
citrix.com 68
5. Notice that the Configuration drop-down will automatically have the config file we
exported in the previous exercise. If it doesn’t, you will need to manaully select that
specific configuration from the drop-down or revisit the exercise to export again.
There is the option at this point to upload and/or upgrade software for the SD-WAN-SE
system.
Software has already been download from the citrix.com/download portal and saved to
the desktop in the folder named “SD-WAN Software”.
Click the Choose Files button to upload the latest 9.3 software packages. Take note
that Software: is listed as “current”.
citrix.com 69
6. After clicking Choose File in the Change Preparation stage, you should encounter a file
browser upload window. Navigate to Desktop > SD-WAN Software > 9.3 > Platforms
directory, and then select the ns-sdw-sw-9.3.3.21.zip file.
Some detail regarding the software package name.
 ns-sdw-sw-9.3.3.21.zip – “ns-sdw-sw” denotes this software package is

specifically for the SD-WAN Standard Edition & Enterprise Edition (Not the
WANopt Edition, or the SD-WAN Center)
 ns-sdw-sw-9.3.3.21.zip – “9.3.3.21” denotes the software release/build number.

This zip file is ~720MB bundle of software and components that are used by Standard
and Enterprise Edition devices. In our lab, we are only making use of the VPX model,
but in a production environment, this single bundled file accounts for all the variations of
models, hypervisors, and OS components that are possible in an SD-WAN overlay. We
will cover the details of this single bundle file in a later exercise.
Once the file has been selected, click Open.
citrix.com 70
7. In the Change Preparation page, click Upload button once the software package has
been selected.
The 720MB software package may take only a minute to upload and process, but
this is due to the direct connection between the Student Desktop and the MCN
device. This process could be significantly delayed if the connectivity to the MCN web
browser is over a poor quality link. Local connectivity and upload of the large software
package is recommended.
Do not click the verify button before the software package is fully uploaded and
processed.
8. After the file is done uploading, the page will display the message; “Upload complete…”
and the Software box will populate with “Model(s): CBVPX”. You may need to wait an
extra minute or so as the system is “processing uploaded file”.
Click Stage Appliances to proceed.

Only the CBVPX model was populated because the configuration in the inbox only
has the two sites, each with VPX as the model. In production, each site may be a
unique appliance model and in that case each model will be listed based on that
configuration file.
citrix.com 71
9. Accept the Citrix License Agreement pop-up and click Ok.
10. You will now proceed “Appliance Staging” process. Click Next after the Transfer
Progress shows 100%.
Take note of table at the bottom of this page. It will populate the sites that are in
the configuration file we exported.
Click Next to proceed.
citrix.com 72
11. Next is the Activation stage, click the Activate Staged button to activate the changes
that have been made to this datacenter MCN device.
12. A warning box appears to let you know that this appliance has no running configuration.
Click OK to confirm that the package we created will be used for that purpose.
13. The web interface will automatically redirect to the Configuration > System
Maintenance > Local Change Management page of the MCN. Click Activate Staged
to accept the change management process for this local device.
citrix.com 73
14. In the pop-up warning banner, click OK to switch the Active software/config to the one
on the staged area.
15. The appliance activates within a 180 second count down.
16. When activation is complete, click the Done button.

You will be returned to the Dashboard screen, with a warning message that “the Citrix
Virtual WAN Service is currently disabled.”
citrix.com 74
17. In a production environment, we typically don’t enable the Virtual WAN service
until we have a partner appliance up and running with the appropriate
configuration to start having the SD-WAN device process traffic, but in this case it’s
a lab network and we are okay to proceed. Also this MCN site is deployed Virtual Inline
mode with no redirection of traffic to it, so for that reason we are also okay to proceed.
On the NYC_SDWAN_SE1 web interface we can enable the Virtual WAN Service, by
navigating to the Configuration > Virtual WAN > Enable/Disable/Purge Flows page.
Click the Enable button to enable the Virtual WAN service.
1
2
18. Click OK to confirm enabling the service.
19. After a few seconds, refresh the page and it should update indicating that “the Citrix
Virtual WAN Service is currenlty enabled.”
Take note that the service can also be disabled from here and you can uncheck
the diagnostic dump option if you want to quickly disable without having to wait for
the dump file to be generated.
citrix.com 75
20. Navigate back to the Change Management page (Configuration > Virtual WAN >
Change Management) and validate using the table at the bottom of the page that the
NYC Site VPX is populated with Currently Active Software and Config.
Note that the branch LON and SJC sites are listed as “Not Connected”. This is
because the first time the appliance is brought online, you must manaully upload
the active configuration/software file. The next exercise will walk you through that
process.
Exercise Summary
In this exercise, you applied the configuration and software and enabled the Virtual WAN
Service for the MCN appliance.
citrix.com 76
Exercise 7: Applying the Configuration to the Remote
Appliances (LON & SJC)
Overview
We have already saved and exported the configuration file to the headend MCN device, and we
have applied and uploaded the required software packages for that device to identify the SD-
WAN environment. Now from the MCN Change Management you will download the software
and configuration package that is intended for the remote site SD-WAN devices and upload
manually to each devices Local Change Management. After this first manual procedure,
subsequent software and configuration changes can be done through the MCNs communication
to remote branches over the Virtual Path and the below steps can be skipped in subsequent
changes to configuration and update software, if the data paths (Virtual Path) is up and
connected to the remote site.

 Download and apply the configuration/software package to the remote SD-WAN devices
 Enable the Virtual WAN Service on the remote SD-WAN devices

Step Action
1. While still logged on to the MCN (NYC_SDWAN_SE1) device, navigate to the Virtual
WAN > Change Management page.
citrix.com 77
Step Action
2. In the table at the bottom of this page, find the LON-LON-CBVPX and the SJC-SJC-
CBVPX sites, and click the active hyperlink for each at the right most column for that
row. This will download the packages for the remote sites.
Make sure to select “active” link that belongs to the correct site. Selecting any
other active configuration will put the wrong identify to the target device and you
will have duplicate identity devices on the SD-WAN network. Each active configuraiton
is specific to the site node built in the Configuration Editor.
Make sure the two packages were successfully downloaded:
3. On the XenCenter tool, right-click and Start the LON_SDWAN_SE and

SJC_SDWAN_SE virtual machines.
citrix.com 78
Step Action
4. Navigate to the Console tab of the LON_SDWAN_SE virtual machine and wait for the
VM to fully boot.
In the “cbvw login:” log in with credentials admin/password
And management IP address the SD-WAN using the “management_IP” command.
 mangement_ip
 set interface 192.168.10.27 255.255.255.0 192.168.10.1
 apply
 y
citrix.com 79
Step Action
5. Navigate to the Console tab of the SJC_SDWAN_SE virtual machine and wait for the
VM to fully boot.
In the “cbvw login:” log in with credentials admin/password
And management IP address the SD-WAN using the “management_IP” command.
 mangement_ip
 set interface 192.168.10.30 255.255.255.0 192.168.10.1
 apply
 y
6. Now open the web interface in the Student Desktop Chrome browser to
LON_SDWAN_SE in a new tab.
If you get a “This Connection is not secure” message in the browser, proceed by clicking
the “Advanced” button and “Add Exception”, then confirm security exception.
citrix.com 80
Step Action
7. Log in with admin/password credentials.
We will first run through some common administrative tasks.

8. Navigate to Configuration > System Maintenance > Date/Time Settings. In the
Timezone Settings pane select the Europe/London time zone specific to this site, and
then click the Change Timezone button. Verify the Date/Time after the updated
change.
Click OK on the pop-up window.

9. Next, navigate to the Configuration > Appliance Setting > Licensing page and select
the Remote license radio button, then configure the IP address (10.0.76.37) and port
(27000) of the lab license server. Also, select the desired license file to be pulled down
from the license server (V50VW). Then click Apply Settings.
citrix.com 81
Step Action
10. With administrative tasks complete, we can now upload the configuration/software
package for this site.
Navigate to Configuration > System Maintenance > Local Change Management.
Click the Choose File button to select the recently downloaded file.
11. Click Browse and upload the recently saved remote device software/configuration
package in the Student Desktop downloads directory. Then click Open.
Again, take special care to select the correct file. The site name will be in the file
name.
2
1
citrix.com 82
Step Action
12. Click Upload, then Next.
13. Click the Activate Staged button to complete the configuration of the LON SD-WAN.
14. Click OK to switch the Active software/config to the one on the staged area.
15. The appliance activates within of a 180 second countdown.
Click Done when complete.
citrix.com 83
Step Action
16. The Dashboard will indicate the status with the warning that the “The Citrix Virtual WAN
Service is currently disabled.”
17. Navigate to Configuration > Virtual WAN > Enable/Disable/Purge Flows of the LON
SD-WAN web interface and click the Enable button.
18. Click OK to confirm enabling the service.
citrix.com 84
Step Action
19. Once the service is enabled, navigate to Monitoring > Statistics page and you will
observe that the “Path (Summary)” as the default table displayed. This table provides
the status of each WAN path available between the repective branch SD-WAN and the
datacenter MCN SD-WAN device. The “Good” Path State indicates that there is IP
connectivity on the underlay network between Virtual IP (VIP) addresses of the two
systems, and that the latency, loss, and jitter characteristics are suitable for Virtual WAN
usage. As long as you have one active upload and download path, then your overall
Virtual Path Service State will also indicate a “Good” state.
There may be a few things that may prevent paths from being marked as “Good”.
 There is no IP connectivity between the SD-WAN VIPs on the respective paths

 If all paths are down, one of the devices may not be configured appropriatly with
license
 The Virtual WAN Service may still be disabled on one of the SD-WAN devices
 The configuration was not properly exported to the remote SD-WAN, or the
configuration is incorrect
20. Now open the web interface in the Student Desktop Chrome browser to
SJC_SDWAN_SE in a new tab.
If you get a “This Connection is not secure” message in the browser, proceed by clicking
the “Advanced” button and “Add Exception”, then confirm security exception.
Then log in with admin/password credentials.
citrix.com 85
Step Action
21. Navigate to Configuration > System Maintenance > Date/Time Settings. In the
Timezone Settings pane select the US/Pacific time zone specific to this site, and then
click the Change Timezone button. Verify the Date/Time after the updated change.

22. Click Done when complete, then navigate to the Configuration > Appliance Settings >
Licensing page and enable Remote licensing, and configured IP address (10.0.76.37)
and port (27000) of the lab license server. Also, select the desired license file to be
pulled down from the license server (V50VW). Then click Apply Settings.
citrix.com 86
Step Action
23. Navigate to the Configuration > System Maintenance > Local Change Management
page. Click Choose File.
24. In the pop-up window, make sure to select the SJC package from the downloads
direction, then click Open.
25. With the package uploaded, click Upload.
citrix.com 87
Step Action
26. Click Next and then Activate Staged.
27. With the license in place navigate to the Configuration > Virtual WAN >
Enable/Disable/Purge Flows page and click the Enable button to enable the Virtual
WAN service.
Click OK on the pop-up window to accept the setting.
citrix.com 88
Step Action
28. Navigating to the Monitoring > Statistics page of the SJC SD-WAN web interface, you
will see that the path state is DEAD.
This is to be expected because the MPLS underlay network is not aware of the new SJC
site and can not deliver the SD-WAN Virtual Path packets between the sites.
29. In order to address the dead path issue for the SJC site, we need to investigate the
underlay routing as to why the SJC SD-WAN VIP is not able to reach the NYC SD-WAN
VIP over the MPLS link. We will investigate in detail the underlay network in later
exercise, for now we will quickly address the problem, by opening XenCenter and
viewing the Console for the NYC_Core_Rtr.
citrix.com 89
Step Action
30. Log in with vyatta/Password1 credentials. Then issue the following commands to add a
static route to the SJC_SDWAN MPLS VIP address on the NYC_Core_Rtr.
configure
set protocols static route 169.15.90.2/32 next-hop 172.16.20.1 distance 1
commit
save
exit
31. After a minute, refresh the browser for the SJC_SDWAN Monitoring > Statistics >
Paths (Summary) and the WAN paths should indicate Good path state after the change
to the underlay network.
citrix.com 90
Step Action
32. If the Path State are not indicated as green in either of the LON or the SJC SD-WAN
web interface, then likely the incorrect config file has been uploaded to the local change
management. The next series of steps can be skipped if the path state is good.
The name displayed in the web browser tab when navigating to the Dashboard of each
device can indicate if the incorrect configuration package was applied. In this example,
the SJC configuration package was incorrectly uploaded to the LON site.
citrix.com 91
Step Action
33. (Skip this step if SJC SD-WAN paths are green)
In order to correct the mistake, we will reset the configuration and upload the correct file.
1. Navigate to Configuration > Virtual WAN > Enable/Disable/Purge Flows.
2. Uncheck “Perform a diagnostic dump…”, then click the Disable button.
1
3 2
3. Click OK in the pop-up window to accept the change.

4. Navigate to Configuration > System Mainenance > Configuration Reset.
5. Click the Configuration Reset button (reboot not required).
1
6. After a 60 second count down, the SD-WAN device will clear of configuration
and the upload of the correct package can be applied again.
7. The NYC (MCN) should indicate healthy paths to both the LON and SJC SD-
WANs.
Exercise Summary
In this exercise, you downloaded a configuration/software packages from the MCN and
manually uploaded it to the remote SD-WAN devices using each device’s local user interface.
citrix.com 92
You also enabled the service and validated expected path state between each remote branch
device and the central MCN.
citrix.com 93
Exercise 8: Adding additional WAN links (NYC, LON, SJC)
Overview
With our basic SD-WAN environment up and running configured for a single WAN link at each
site we can increase resiliency and available bandwidth of the SD-WAN overlay network (also
known as the Virtual Path) by introducing additional WAN links at each site. This increases
resiliency of the Virtual Path by providing alternative paths between SD-WAN sites.

 Add a second Internet WAN link to each site
citrix.com 94
Step Action
1. In the Student Desktop, in the browser open the web interface of the MCN
(NYC_SDWAN_SE1). Navigate to the Configuration > Virtual WAN > Configuration
Editor page.
From the NYC_SDWAN_SE1 VPX web interface, navigate to Configuration > Virtual
WAN > Configuration Editor:
The configuration from the previous exercise should already be open, if it is not, click the
Open button and open the last saved configuration file.
Select the Advanced tab.
Basic mode allows for very quick build out of configuration, but advanced mode
exposes more of the detail in the configuration editor that may be needed for some
aspects of the deployment. Once the initial path connectivity is available, like we have
been NYC and LON devices, basic mode no longer needs to be used.
citrix.com 95
Step Action
2. For the NYC site, since deployed in Virtual Inline Mode, we will make use of the existing
Virtual Interface (E1Vlan0) which was already created when building the first WAN link.
In the Sites tile, navigate down to NYC site node and begin by adding a new Virtual IP
Address.
 IP Address/Prefix: 172.16.40.3/24
 Virtual Interface: ElVlan0
Click Apply to accept the change.
Take note that these VIPs are going to be used in indentifying the unique WAN
links available for this site. The virtual IP addresses can reside on the same
subnet because of the deployment mode of this site, but we will be dependent on the
underlay network to make sure the traffic sourcing from these IP address are delivered
to the expected WAN link. We will dive in deeper on this subject in a later exercise.
3. For the NYC site, next we will add a new Internet WAN link.
Navidate to the WAN Link node for the NYC site, and click the add (+) icon to add a new
WAN link, in the pop-up window name the link NYC_INET and for Access Type select
Public Internet.
citrix.com 96
Step Action
4. For the newly added WAN link, expand the Settings node and click the edit icon.
Configure 20000 kbps for both LAN to WAN and WAN to LAN Physical Rate.
2 3
Click Apply to accept the changes.
5. Next, for the same NYC_INET WAN link expand the Access Interfaces node and click
the add (+) icon to add an interface for this WAN link. Again, we will be using the same
virtual interface (E1Vlan0) that we used for the other WAN link because of the Virtual
Inline deployment mode.

 IP adress: 172.16.40.3
 Gateway IP: 172.16.40.1
Click Apply to accept the settings.
citrix.com 97
Step Action
6. We are done adding the WAN link for the NYC site. Next we will add an internet WAN
link for the LON site
In the Sites tile in advanced mode, navigate to the LON > Interface Groups node.
Click the add (+) icon to add a new Interface Group.
 Security: Untrusted
For the LON site, we are required to add a new Interface Group because we are
introducing a new link to this site that did not exist in the underlay network. We
could have connected to the MPLS router and used the same interface group on SD-
WAN but instead we decided to directly connect the new internet link directly into the
SD-WAN device. Each and every configuration can be unique based on the design, the
configuration editor is designed to provide flexability. Links are configured as Untrusted
for public internet links that do not have an external firewall that can protect the network.
citrix.com 98
Step Action
7. Next for the LON site we will add a second Virtual IP Address to handle the new link.
Navigate to LON > Virtual IP Address, click the add (+) icon to add a new VIP.

8. For the LON site, next we will add a new Internet WAN link.
Click the add (+) icon to add a new WAN link, in the pop-up window name the link
LON_INET and for Access Type select Public Internet.
citrix.com 99
Step Action
Configure 2000 kbps for LAN to WAN and 10000 kbps WAN to LAN Physical Rate.
10. Next, for the same LON_INET WAN link expand the Access Interfaces node and click
the add (+) icon to add an interface for this WAN link.

 IP adress: 169.15.71.3
 Gateway IP: 169.15.71.1
citrix.com 100
Step Action
11. With the new internet link added to both the NYC and LON sites, we can confirm the
path association on these new links between sites.
Under the Connections tile, navigate to NYC > Virtual Paths > NYC-LON > Paths.
The path relationship was automatically created by the configuration editor. It

results in a simple 1-to-1 relationship when a single private intranet link and a
single public internet exist at each site, but paths can automatically be created to cross-
connect when multiple public internet links are available for a sites. This can result in
large number of path relationships that are auto created, which could potentially be a
limitation problem, so keeping the list in condenced is recommended.
Expanding each individual path provides an option to “Convert to Static Path” which
would allow you to delete the auto created path if desired. For large scale deployments
making use of auto path groups to limit the full meshing of internet links or make use of
WAN link templates and service provider identification features to help control the auto
path creation.
citrix.com 101
Step Action
12. Next, we will add an internet WAN link for the SJC site.
In the Sites tile in advanced mode, navigate to the SJC > Interface Groups node.
Click the add (+) icon to add a new Interface Group.
 Ethernet Interface: 3
13. Next for the SJC site we will add a second Virtual IP Address to handle the new link.
Navigate to SJC > Virtual IP Address, click the add (+) icon to add a new VIP.

citrix.com 102
Step Action
14. For the SJC site, next we will add a new Internet WAN link.
Click the add (+) icon to add a new WAN link, in the pop-up window name the link
SJC_INET and for Access Type select Public Internet.
Configure 2000 kbps for LAN to WAN and 10000 kbps WAN to LAN Physical Rate.
citrix.com 103
Step Action
16. Next, for the same SJC_INET WAN link expand the Access Interfaces node and click
the add (+) icon to add an interface for this WAN link.

 IP adress: 169.15.91.2
 Gateway IP: 169.15.91.1
17. With the new internet link added to the SJC site, we can confirm the new path
association between sites.
Under the Connections tile, navigate to NYC > Virtual Paths > NYC-SJC > Paths.
18. With zero audit warnings in our configuration editor, it is a good indicator that our
configuration is complete. Save As the new configuration with filename Exercise8.
citrix.com 104
Step Action
19. With the new configuration saved, click Export, and select the Change Management
Inbox.
20. Navigate to Change Management and as we have done before go through the change
process workflow.
a) Click Begin
b) We do not need to upload the software file again, it was only required the first
time, and only again be required if we upgrade the software
c) Make sure the Configuration file is in the inbox
d) Click Stage Appliances
e) Since the MCN has connectivity to both the LON and SJC SD-WAN, we should
see 100% with 3/3 appliances finished. Click Next to continue.
f) Click Activate Staged

g) After the process is complete, click Done
Change Management should indicate that the Active configuration filename is the one
we exported last, the staged configuration was the previous configuration before the
change management.
Because we have active virtual path between the MCN and a remote branch (SJC
and LON), we do not need to manually upload the configuration package to those
branches. It is done through the change management process and configuration and
software is automatically feed through a control channel. This becomes significant when
an environment scales to hundreds of nodes. The change mangement process time
increases as the number of sites increases, however it gives granular control to the
Admin for change successful change management.
citrix.com 105
Step Action
21. Now that the new configuration is confirmed to be active, we can take a look at the
health of the paths by navigating to the Monitoring > Statistics page of the MCN node
(NYC_SDWAN_SE1).
The path from LON to NYC across the Internet link is marked as Dead! Also for the
SJC site. For this lab this is expected outcome and we will address it by troubleshooting
in the next exercise.
Paths coming up as Dead during initial deployments is common occurrence in

production deployments, especially across public internet wan link types.
Generally this is due to misrouting of VIP to VIP communication between two SD-WAN
devices. For public internet links the communication typically goes through a Firewall or
two and NAT is required to allow the UDP tunnel to be established.
Note for every MCN internet WAN link defined, we are required to have a static
public IP which is typically NAT’d by the Firewall with a 1-to-1 static NAT to the
private VIP address of the the MCN, and also port forwarding of UDP 4980 configured to
allow the probe through for the tunnel establishment. For branch SD-WAN devices, the
“autodect public IP” can be enabled on the internet WAN links, eliminating the need for a
static public IP for each branch site, and also we don’t need to port forward 4980 since
the probe will originate from the branch and an NAT entry would automatically be
generated if a firewall/modem is in path of the connection attempt at the branch
internet/4G link.
Exercise Summary
In this exercise, we added a second WAN link to each site and pushed the configuration
centrally from the MCN’s change management.
citrix.com 106
Module 3 (Basic): Troubleshooting
and Validating SD-WAN Overlay
citrix.com 107
Module 3 Overview
This module will highlight commonly used troubleshooting techniques to help root cause Virtual
Path establishment between SD-WAN devices and walk through validating proper functionality.
It is critically important that before configuring the Citrix SD-WAN solution for any environment,
you complete the following pre-requisites:

and the data plane
citrix.com 108
Exercise 9: Troubleshooting path state
Overview
When deploying SD-WAN devices in any network, you may encounter issues where not all
available paths are reporting active state. It is a common occurrence on public wan links that
deal with external firewalls to have paths come up as dead, requiring further troubleshooting.
Troubleshooting involves identifying the root cause and typically once it can quickly be rectified
with the help of the networking team management the underlay.

 Be introduced to diagnostics tools to help troubleshoot network issues
 Root cause and fix dead path state

Step Action
1. A good starting point in identifying bad path state is to to navigate to the Monitoring >
Statistics page on each appliance and identify the issue paths. Select the more
advanced view of the paths by selecting to Show: Path (Detailed).
citrix.com 109
Step Action
2. You may have to look at both end devices of the tunnel to help further clarify the
connectivity issue of the path. Open the detailed path statistics page of each SD-WANs
web interface.
NYC_SDWAN_SE1 LON_SDWAN_SE
The output of these pages help identify that both devices are reporting that the path from
LON to NYC across the Internet link is reporting to have a problem. All other paths
between these sites, including the returning path across the same internet path are fine.
By taking a closer look at the kbps column on the dead path, notice that the LON SD-
WAN is pushing 12 kbps, while the NYC SD-WAN is reporting 0 kbps.
There is no production traffic in this lab network so we can quickly assess that the
reported kbps is probe traffic for the UDP tunnel creation. Branch office devices
typically initiate the probe, generally to punch through their local firewall and to target the
Static Public IP that is configured the MCN internet WAN link. The MCN will not initiate
a probe because it is programmed to wait for the incoming probes in order to
dynamically learn the public IP being advertised by the each unique branch SD-WAN.
citrix.com 110
Step Action
3. Best practice would involve starting at Layer 1 and double-checking cabling and proper
connectivity to the device interfaces. We can also check the Ethernet table for expected
interface connectivity, and proper speed/duplex negociation.
Configuration > Appliance Setting > Network Adaptors > Ethernet tab.
Keep in mind, this is a VPX, but the physically appliances will have ability to hard-
set desired speed and duplex. The green dot indicates the interface is active, and
you can match the reported MAC address to the MAC address reported on the
Hypervisor to validate the interface is properly matching between topology,
configuration, hypervisor, and SD-WAN web interface layout of the interface number.
Reference the design topology to get a better understanding of the setup.
Ethernet detail can also be found on the Monitoring > Statistics > Show: Ethernet
table. This gives us a picture of the link state status and that the device is trying to
communicate on each respective interface, and also if there are any errors that need
further investigation.
There are no Errors reported. If there were, speed/duplex settings would need to be
investigated further.
citrix.com 111
Step Action
4. Next we will investigate Layer 2.
We can begin by checking the ARP tables, which validates that each WAN link is at
least able to communciate with the configured default gateway and is able to obtain the
MAC address of the WAN side devices (Router, Firewall, Modem). In this lab exercise
you should observe the State of each reading “Ready_Active”, which indicates
successful communication. Any other reported state would require further investigation
of the connected interface and the availability of the configured gateway on that WAN
link.
Up to this point in the investigation the information tells us the issue of down INET path
is not with the configuration or connectivity of SD-WAN and neighboring devices.
citrix.com 112
Step Action
5. Layer 3 needs to be investigated next, and generally its an issue with the underlay
network not enabling the VIP to VIP communication from working properly. Commonly
used networking tools such as ping and traceroute are primarily used to validate IP
connectivity between WAN link VIPs between the two SD-WANs. These tools along
with others are directly available on the web interface of each SD-WAN device.
Packet capturing is also available and can help root cause connectivity issues. We can
start by doing a capture on both SD-WAN devices on the WAN interface to the issue
link.
Navigate to the Configuration > System Maintenance > Diagnostics > Packet
Capture page for each device.
For the NYC device select interface 1, For the LON device select interface and 3
since both LAN and WAN traffic share for INET connected interface. Leave
this interface. Leave everything else everything else default and click Capture.
default and click Capture.
1
1
2
2
3
3
Click OK on the pop-up window to accept
Click OK on the pop-up window to accept the capture.
the capture.
citrix.com 113
Step Action
6. After the capture, SD-WAN will provide a link to download the capture on your local PC,
but it also displays the capture right in the web interface for quicker analysis.
What we want to look for is the UDP probes that are being delivered. For working paths
you will see both incoming and outgoing. For dead paths probes in one or both direction
may be missing, which are highlied in red text.
Non-working INET paths (NYC <-> LON) Non-working INET paths (NYC <-> LON)
Packet Sent UDP 4980: Packet Sent UDP 4980:

 SRC: NYC INET VIP (172.16.40.3)  SRC: LON INET VIP (169.15.71.3)
 DEST: LON INET VIP (169.15.71.3)  DEST: NYC INET VIP (172.16.40.3)
Packet Received UDP 4980 (MISSING): Packet Received UDP 4980:
 SRC: LON INET VIP (169.15.71.3)  SRC: NYC INET VIP (172.16.40.3)
 DEST: NYC INET VIP (172.16.40.3)  DEST: LON INET VIP (169.15.71.3)
For comparison, we can also investigate For comparison, we can also investigate
the working MPLS path communication. the working MPLS path communication
but for this device will require a recapture
on interface 1.
Working MPLS paths (NYC <-> LON)
Working MPLS paths (NYC <-> LON)
Packet Sent UDP 4980:

 SRC: NYC INET VIP (172.16.40.2) Packet Sent UDP 4980:
 DEST: LON INET VIP (172.70.1.27)  SRC: LON INET VIP (172.70.1.27)
Packet Received UDP 4980:  DEST: NYC INET VIP (172.16.40.2)
 SRC: LON INET VIP (172.70.1.27) Packet Received UDP 4980:
 DEST: NYC INET VIP (172.16.40.2)  SRC: NYC INET VIP (172.16.40.2)
 DEST: LON INET VIP (172.70.1.27)
Focusing investigation on the non-working internet path, based on the results of the
trace capture, we can easily decipher that the LON SD-WAN receives the UDP 4980
packet from the NYC device (172.16.40.3 -> 169.15.71.3), however the return traffic
sent by LON SD-WAN (169.15.71.3 -> 172.16.40.3) is not received by the NYC SD-
WAN. It is getting lost in transit in the underlay network.
citrix.com 114
7. At this point we can easily conclude that the underlay network is preventing the packet
sent from the LON site to successfully be received by the NYC site for the INET VIP.
Traceroute can help further distinguish how far the packet gets in the network and
possibly identify if the packet fails due to route or packet being dropped by an
intermediate device.
We can first run traceroute on the working MPLS path to show a working sample, then
run traceroute on the non-working path to analyze the results.
Configuration > System Maintenance > Diagnostics > Traceroute tab
Reference the network topology to identify that the proper hops are seen in each
direction.
Select NYC_MPLS->LON_MPLS path Select LON_MPLS->NYC_MPLS path
from the drop-down and click Trace. from the drop-down and click Trace.
1. 172.16.40.1 – NYC_Core 1. 172.70.1.1 – LON_CE_Rtr

2. 172.16.20.1 – NYC_MPLS_Rtr 2. 169.15.70.1 – PE_MPLS_Rtr
3. 169.15.50.1 – PE_MPLS_Rtr 6. 169.15.50.2 – NYC_MPLS_Rtr
4. 169.15.70.3 – LON_CE_Rtr 3. 169.15.20.2 – NYC_Core
5. 172.70.1.27 – LON MPLS VIP 4. 172.16.40.2 – NYC MPLS VIP
Select NYC_INET->LON_INET path from Select LON_INET->NYC_INET path from
the drop-down and click Trace. the drop-down and click Trace.
citrix.com 115
Step Action
1. 172.16.40.1 – NYC_Core 1. 169.15.71.1 – PE_INET_Rtr

2. 172.16.30.3 – NYC_INET_Rtr 2. 192.168.10.1 – Lab internet
3. …. gateway….
From the MPLS traceroute results we can see that the expected hops are properly being
displayed, and successful connectivity to the respective MPLS VIP addresss, thus why
the MPLS path is being reported as GOOD in SD-WAN path statistics.
From the INET traceroute results we can see that between the NYC_INET_Rtr and the
PE_INET_Rtr there is a problem with routing that is preventing the VIP to VIP
communication resulting in DEAD path in the reporting.
citrix.com 116
8. Ping is the next tool we can use to help further diagnose the problem.
Configuration > System Maintenance > Diagnostics > Ping tab
Ping has two options, either ping from the entire system or ping from a specific Interface,
we will utilize the “Ping Interface” tool to troubleshoot this particular problem on specific
WAN paths.
Reference the topology for better understanding of the IP addresses in the network.
1. We should start by pinging the closest 1. We should start by pinging the closest
gateway (Core 172.16.40.1). From gateway (PE_INET_Rtr 169.15.71.1).
the Diagnostics > Ping Interface tool From the Diagnostics > Ping Interface
enter 172.16.40.1 in the IP Address tool enter 169.15.71.1 in the IP
field and make sure to select Address field and make sure to select
EVlan0:172.16.40.3/24 as the source, E3Vlan0:169.15.71.3/24 as the
then click the Ping Interface button. source, then click the Ping Interface
button.
1 2
3 1 2
3
You should observe successful ping with
this test: You should observe successful ping with
this test:
2. Next ping down the INET path to the 2. Next ping down the INET path to the
inside NYC_INET_Rtr address outside NYC_INET_Rtr address
172.16.30.3, from the same 169.15.60.2, from the same
EVlan0:172.16.40.3/24 interface E3Vlan0:169.15.71.3/24 interface
You should observe successful ping with You should observe successful ping with
this test: this test:
citrix.com 117
Step Action
3. Next ping down the INET path to the 3. Next ping down the INET path to the
inside PE_INET_Rtr address NYC_Core_Rtr address 172.16.30.3,
169.15.60.1, from the same from the same
There is the failure point for the NYC to There is a failure point from the LON to
LON INET path. NYC INET path.
4. Lastly ping down the INET path to the 4. Next ping down the INET path to the
partner SD-WAN INET VIP address partner SD-WAN INET VIP address
169.15.71.3, from the same 172.16.40.3, from the same
There is the failure point for the NYC to There is a failure point from the LON to
LON INET path. NYC INET path.
citrix.com 118
Step Action
9. Using ping from outside of the SD-WAN is also very valuable.
From the XenCenter application, we can log into the individual machines that make up
the underlay network.
On XenCenter, log in to the NYC_Core_Rtr VM using vyatta/Password1 credentials.
From NYC_Core_Rtr we can issue a ping command to the NYC SD-WAN INET VIP
172.16.40.3.
 ping 172.16.40.3
Once proven successful, Ctrl+C to stop.
citrix.com 119
Step Action
10. Similarly, we can log into the NYC_INET_Rtr virtual machine, and perform the same
ping test to the NYC SD-WAN INET VIP address.
On XenCenter, log in to the NYC_INET_Rtr VM using vyatta/Password1 credentials.
From NYC_INET_Rtr we can issue a ping command to the same NYC SD-WAN INET
VIP 172.16.40.3.
 ping 172.16.40.3
Pings fail to reach the VIP address from this device.
Ctrl+C to stop.
citrix.com 120
Step Action
11. It is important that this NYC edge device on the Internet be able to route traffic to the
virtual IP addresses used by SD-WAN. Tunnel packets that arrive at this router will have
the destination address of the NYC_SDWAN_SE INET VIP (172.16.40.3) and without a
proper route, this router may drop the packet. This is possibly why we seeing bad path
negociation between the SD-WAN partners, but we need to investigate further.
Issueing the “show ip route” command on the NYC_INET_Rtr, we can see that it truly
does have a route, and the packet is likely being delivered.
citrix.com 121
Step Action
12. Investigating further, we can next look at the NYC_SDWAN_SE1 devices route table by
navigating to the Monitoring > Statistics > Show Routes page. Here we notice
something interesting.
There is a hit count on the discard route.

This explains the ping failure sourced from the NYC_INET_Rtr (172.16.30.3).
SD-WAN being in Virtual Inline mode requires the routes to be defined for any
local subnets if communication is required to those subnets through the overlay
routing table.
We also do not see the subnet of the LON INET VIP (169.15.71.3), but we are okay in
that case, because partner VIP addresses are exclusions to the rule. SD-WAN delivering
to that subnet will hand it off to the configured gateway for the WAN link access interface
and rely on the underlay network to delivery it accordingly.
Adding the internal NYC_INET subnet to the SD-WAN route table would resolve the
local IP connectivity issue between the router and SD-WAN but will not address our
problem, so we need to continue investigating.
citrix.com 122
Step Action
13. Issuing the command show ip route on the NYC_INET_Rtr, will show the edge internet
router has means to get the packet with destination of the LON VIP (169.15.71.3) using
the default route.
Furthermore, ping to the LON INET VIP is successful.
citrix.com 123
Step Action
14. Issuing the same command show ip route on the NYC_Core_Rtr, show similar, that
the core router has means to get the packet with destination of the LON VIP
(169.15.71.3) using the default route. It hands it off to the NYC_INET_Rtr, which we
have already assessed to have the needed route.
However, ping 169.15.71.3 fails.
Traceroute 169.15.71.3 further identifies that the packet is getting lost after the
NYC_INET_Rtr hands it off to the PE_INET_Rtr router.
citrix.com 124
Step Action
15. Next, we can log in to the PE_INET_Rtr through the XenCenter console and using the
credentials vyatta/Password1.
16. On the PE_INET_Rtr, looking at output of show ip route, we can see that there is no
route to the NYC INET VIP subnet. Default route would send UDP probe packets from
the LON SD-WAN to the management network of the lab (192.168.10.1).
This is the root cause of the dead path issue we have been troubleshooting.
citrix.com 125
Step Action
17. In order for this to be addressed, we need PE_INET_Rtr to learn the route to the NYC
SD-WAN subnet through eBGP.
On the NYC_INET_Rtr, we can see what is shared with neighbors through the following
BGP command, which is nothing.
show ip bgp neighbors 169.15.60.1 advertised-routes
We will need to configure a static route on the NYC_INET_Rtr with the following
commands to advertise the route to the SD-WAN VIP through BGP.
configure
set protocols bgp 64511 redistribute static metric 2
commit
save
exit
On the NYC_INET_Rtr, running the command again will confirm the change.
show ip bgp neighbors 169.15.60.1 advertised-routes
citrix.com 126
Step Action
18. With that simple change in the underlay network, now we can see on the SD-WAN path
statistics that the INET path has successful connectivity and that we address the issue
we were seeing earlier. The same issue was for the LON and SJC INET paths to the
NYC INET SDWAN VIP, fixing one path also fixed the other path.
citrix.com 127
Step Action
19. Generally, when paths come up successfully, it is still recommended to run traceroute
from SD-WAN VIPs to make sure the unique paths that make up the Virtual Path do not
take the same exact underlay routes in the network, otherwise the solution will not
function properly. The paths can be confirmed from the Configuration > System
Maintenance > Diagnostics page:
NYC_INET -> LON _INET
NYC_MPLS -> LON_MPLS
citrix.com 128
Step Action
20. Similarly the same can be done from the LON_SDWAN_SE web interface
Configuration > System Maintenance > Diagnostics:
LON_INET -> NYC_INET
LON_MPLS -> NYC_MPLS
Both directions show unique path delivery across the expected hops in the underlay
network. Our SD-WAN is configured properly and we can proceed.
Exercise Summary
In this exercise, we used the available tools on SD-WAN diagnostics to troubleshoot WAN path
connectivity, root caused the problem, then resolved it by correcting the part of the network that
was causing the issue.
citrix.com 129
Exercise 10: Redirect traffic using OSPF to Virtual Inline
Mode SD-WAN (NYC)
Overview
In the previous exercise we validated successful path establishment between SD-WAN devices.
Now we need to make sure that hosts connections at each SDWAN and non-SDWAN sites are
communicating across the SD-WAN overlay network when appropriate. SD-WAN can be up
and active in the network, but without proper traffic coordination SD-WAN overlay may go
unutilized.

 Configure proper routing through the SD-WAN overlay
citrix.com 130
Step Action
1. In the last exercise we verified that SD-WAN to SD-WAN communication was taking the
appropriate hops per WAN path that was configured. In this exercise, we need to run
through the same test but using the end host machines at each site.
If the hops do not indicate that the SD-WAN tunnel is being utilized then the delivery
across SD-WAN overlay is not being utilized.
First, we will need to put the LON_Client behind the newly installed LON_SDWAN_SE
VM. Begin, by shutting down the LON_Client VM.
Right-click the LON_Client VM and select Shut down.
2. With the VM shutdown, navigate to the Networking tab, and select Device 1. Then
click Properties button.
From the Virtual Interface Properties pop-up window, select LON_LAN from the Network
drop down and click OK to accept the new setting.
1
3
5
citrix.com 131
Step Action
3. With the correct Virtual Interface selected for the LON_Client, start the VM with a right-
click, then clicking Start.
4. With the VM started, navigate to the Console tab and click the Send Ctl+Alt+Del button
to sign into the VM with Administrator/Password1 credentials.
citrix.com 132
Step Action
5. While logged in to the LON_Client VM, open the command prompt.
 From windows search, type “command”

 Select Command Prompt to open the application
6. From the LON_Client VM, issue a ping command to the NYC_Server (172.16.10.12).
 ping 172.16.10.12
 tracert 172.16.10.12
We can see from the ping and traceroute results that the traffic is successfully being
delivered across the SD-WAN overlay. The traceroute hops indicate delivery across the
SD-WAN overlay with NYC_SDWAN (172.16.40.2) as a the first hop before being
delivered throught the core router (172.16.40.1) to the end host (172.16.10.12).
citrix.com 133
Step Action
7. Next we need to check the delivery path for the reverse direction. Log in to the
NYC_Server and navigate to the Console tab on XenCenter to log in using
Administrator/Password1 as the credentials. The Send Ctrl+Atl+Del will provide the
login prompt.
8. On the NYC_Server VM, open the command prompt.
citrix.com 134
Step Action
9. In the NYC_Server Command Prompt, issue ping and traceroute commands to the
LON_Client VM (172.70.1.28).
 ping 172.70.1.28
 tracert 172.70.1.28
The results of the ping and traceroute test indicate that connectivity is accomplished
through the underlay MPLS network and not through the SD-WAN overlay.
Following the traceroute flow, the test first hits the NYC_Core_Rtr (172.16.10.199), then
the NYC_MPLS_Rtr (172.16.20.1), then the PE_MPLS_Rtr (169.15.50.1), then the
LON_CE_Rtr (169.15.70.3) and finally the end host (172.70.1.28).
This is because the NYC_SDWAN is deployed in Virtual Inline mode and traffic
must purposely be redirected to the SD-WAN device for delivery using the overlay.
Redirection of traffic can be accomplished using policy based routing, or dynamic routing
protocols (OSPF, BGP).
citrix.com 135
Step Action
10. Since the NYC_SDWAN is hanging off the NYC_Core router, we can run the show ip
route command to identify the current route table that delivers traffic destined for the
LON site (172.70.1.0/24) through the MPLS.
The following route is an OSPF learned route from the underlay network, which delivers
traffic destined to the LON network to be delivered to the NYC_MPLS router as a next
hop.
O>* 172.70.1.0/24 [110/20] via 172.16.20.1
11. We can make use of OSPF dynamic routing feature on the SD-WAN to inject routes to
the underlay at a lower administrative distance, so that when SD-WAN neighborship is
active, the NYC_Core_Rtr will redirect traffic to SD-WAN for overlay delivery.
Per the topology, OSPF Area 3 has already been designed to serve this purpose.
citrix.com 136
Step Action
12. Let’s begin by configuring OSPF for the NYC_SDWAN site.
Open the Configuration Edtior to the NYC_SWAN_SE web interface. Navigate to the
Configuration Editor, and open the latest config file. Then navigate down to the
Connections tile, and open the NYC site node, then Route Learning > OSPF.
Click the pen icon for Basic Settings.
citrix.com 137
Step Action
13. Check the Enable option, along with the Advertise NetScaler SD-WAN Routes. Then
input the desired loopback IP address to be advertised by the SD-WAN (20.0.1.14). For
Export OSPF Route Type, select Type 1 Intra Area from the drop down menu.

SD-WAN can advertise routes as intra-area routes (LSA Type 1) to get preference
as per its route cost using the OSPF path selection algorithm. The route cost can
be configured and advertised to the neighbor router. This allows for deployment of SD-
WAN devices in Virtual Inline mode deployment.
14. Next expand the Area node, and click the add (+) button to add a new OSPF Area ID.
Input 3 for the ID field, then expand the node, to input detail about the virtual interface.
Click the add (+) button, then select E1Vlan0 as the virtual interface, and select
Authentication Type as Plain Text. Then enter the Password into the Password field.
1
3 4
5
You may notice a warning indicator that Import Filters are needed to dynamically
learn routes from the underlay network, but this can be ignored for now.
citrix.com 138
Step Action
15. The configuration file can be saved as Exercise10 and Export to Change Management.
16. Complete the full Change Management process:

 Change Preparation (upload of software not required)
 Appliance Staging. With the Appliance Staging completing 100%, then click
Next.
 Activation (Activate Staged)
Since all the devices in the network have connectivity to the MCN, the “ignore
incomplete” option should not be used and manual upload of software through
local change management should not be required since we have already done that in
our first Change Mangement push. Complete Change Management for Exercise10
configuration file.
citrix.com 139
Step Action
17. On the Console tab of the NYC_Core_Rtr VM, issue the following command to confirm
that OSPF configuration has not yet been done on the interface connected to the
NYC_SDWAN_SE device:
 show ip ospf neighbor eth3
Begin configuration OSPF on the Core router by issuing the following commands:
configure
set protocols ospf area 0.0.0.3 area-type normal
set interfaces ethernet eth3 ip ospf network broadcast
set interfaces ethernet eth3 ip ospf authentication plaintext-password Password
set protocols ospf area 0.0.0.3 network 172.16.40.0/24
set protocols ospf neighbor 20.0.1.14
commit
save
exit
After the commands are issued, run show ip ospf neighbor eth3 again to determine
successful neighbor establishment.
citrix.com 140
Step Action
18. On the NYC_Core_Rtr, run the show ip route command to see the updated results. It
may take a minute for the route table to converge and update.
The highlighted routes were injected by the SD-WAN to the NYC_Core_Rtr and provides
redirection to the NYC_SDWAN.
With these new routes, we also have to consider the UDP traffic that SD-WAN
wants to deliver to the same subnet. The LON_SDWAN MPLS VIP 172.70.1.27
would get reflected back to the NYC_SDWAN with this new route in place. We can
see that based on the down path (NYC_MPLS -> LON_MPLS) that results with this
change.
citrix.com 141
Step Action
19. We can quickly address the down path problem by issuing a static route on the
NYC_Core_Rtr to prevent the UDP traffic from being reflected back.
configure
commit
save
exit
The static route should address the path issue.
20. In regard to which routes should be advertised by the NYC_SDWAN_SE1 device to the
underlay network (NYC_Core_Rtr) through OSPF should be limited to only remote sites
that have SD-WAN in place.
They are listed here:
O>* 172.70.1.0/24 [110/15] via 172.16.40.2
O>* 172.90.1.0/24 [110/15] via 172.16.40.2
The following routes that also being advertised are not needed and should be filtered
out.
O>* 169.15.90.0/24 [110/15] via 172.16.40.2
O>* 172.16.11.0/24 [110/15] via 172.16.40.2
O>* 172.16.12.0/24 [110/15] via 172.16.40.2
citrix.com 142
Step Action
21. We can create export filters to control what routes SD-WAN shares.
In the Configuration Editor navigate to Connections > NYC > Route Learning > Export
Filters.
1. Click the add (+) icon
2. Leave the default Any/* filters and make sure Include is not enabled
1
2
3. Click the add (+) icon again

4. Input Network Address: 172.70.1.0/24 and select to enable Include
1
2 3
5. Click the add (+) icon again

6. Input Network Address: 172.90.1.0/24 and select to enable Include
2 3
4
citrix.com 143
Step Action
22. With the new Export Filter in place, we can complete the full Change Management
process:
 Appliance Staging. With the Appliance Staging completing 100%, then click
Next.
23. With the new route export filter in place, run the show ip route command on the
NYC_Core_Rtr to validate that only the desired routes are being advertised to redirect
traffic to the NYC_SDWAN_SE1 (172.16.40.2).
O>* 172.70.1.0/24 [110/15] via 172.16.40.2

O>* 172.90.1.0/24 [110/15] via 172.16.40.2
citrix.com 144
Step Action
24. The dynamic route advertisement of SD-WAN routes to the underlay routers using
OSPF addresses the host to host communication being redirected to the SD-WAN
overlay. Traceroute on the NYC_Server to the LON host IP address 172.70.1.28 will
indicate NYC_SDWAN and LON_SDWAN as hops.
From the LON_Client we can issue traceroute to the NYC_Server to make sure the hops
still indicate the SD-WAN overlay.
The traceroute from LON may look a little strange because there are no local
London hops seen, the first hop is directly one of the SD-WAN VIPs (172.16.40.3
or 172.16.40.2)) in NYC. This is expected due to the deployment of the LON SD-WAN.
It is deployed inline mode across the existing flat network. SD-WAN intercepts the traffic
and delivers it encapsulated in a UDP envelope across the overlay network. The first
hop seen is the end of the tunnel.
citrix.com 145
25. The last bit of confirmation of correct configuration would be to run traceroute on the SD-
WAN devices to make sure the correct underlay paths are still working with the updated
route changes.
Here are the traceroute diagnostics for NYC_SDWAN:
Here are the traceroute diagnostics for LON_SDWAN:
citrix.com 146
Step Action
Here are the traceroute diagnostics for SJC_SDWAN:
Exercise Summary
In this exercise, we made use of dynamic routing protocols on SD-WAN to inject overlay routes
to the underlay network for traffic redirect and made use of export filters to control which routes
actually get injected.
citrix.com 147
Exercise 11: Dynamic Virtual Path between LON and SJC
Overview
In the previous exercise we made use of dynamic route learning to redirect traffic to the SD-
WAN deployed in Virtual Inline Mode for communication to SD-WAN deployed at remote
branches. In this exercise we will take a look at how branch nodes can directly communicate
with one another utilizing Dynamic Virtual Paths.

 Configure dynamic virtual paths for routing between LON and SJC site

Step Action
1. On XenCenter right-click and start the SJC_Client VM.
2. Navigate to the Console tab of XenCenter for the SJC_Client VM and log in with
credentials Administrator/Password1. Click the Send Ctrl+Alt+Del button to get the
login prompt.
citrix.com 148
Step Action
3. Once logged in, open the command prompt by right clicking the Windows start menu.
4. From the SJC_Client, run the following traceroute command to the LON_Client
(172.70.1.28) to see the hops taken by the connection attempt.
 tracert 172.70.1.28
As the traceroute output indicates, the packet is first delivered to the SD-WAN LAN VIP
(170.90.1.1) but the packet fails beyond there.
citrix.com 149
Step Action
5. We need to investigate by looking at the SJC_SDWAN route table.
Configuration > Statistics > Show (Routes):
From the route table, we can see that the SJC_SDWAN knows about the subnets of the
NYC site (172.16.10.0, 172.16.11.0, 172.16.12.0, 172.16.40.0), but is unaware of the
subnets of the LON site. So essentially, any packets destined to that subnet are getting
dropped by this SD-WAN due to the lack of route.
citrix.com 150
Step Action
6. We need to navigate back to the MCN Configuration Editor to change this behavior.
Open the latest config file in the Config Editor Advanced Mode and in the Connections
tile navigate to the NYC nodes WAN-to-WAN Forwarding.
Enable WAN-to-WAN Forwarding (Route Export) and Enable Site as Intermediate
Node.
Click Apply to accept the new setting.
3
4
WAN-to-WAN Forwarding (W2WF) allows the Site to act as a proxy between two
adjacent branch sites allowing for host to host communication through this site.
Enabling this feature allows this site to export SD-WAN overlay routes to partner devices
in the same WAN-to-WAN Forwarding group.
Enabling Site as Intermediate Node give the ability for the site to act as a mediator
for the creation and destruction of Dynamic Virtual Paths between two client node
SD-WAN devices.
citrix.com 151
Step Action
7. The Route Export option in W2WF addresses our connectivity between SJC and LON,
however the communication is relayed through the NYC_SDWAN. If we want direct
branch to branch communication without the added latency of having to boomerang
through the MCN, we need to further configure the option for Dynamic Virtual Paths
making use of the Intermediate Node setting on the NYC node.
In the same configuration file in the Connections tile, navigate down to the LON site >
Virtual Paths > Dynamic Virtual Paths.
Click the pen icon and select the Enable Dynamic Virtual Paths option.
Set the Maxium Dynamic Virtual Paths option to 1.
SD-WAN sets aside 80kbps for each Virtual Path Service and 100kbps for each
Intranet/Internet service created. Meaning if the available WAN links for the site
are limited, create Static Virtual Path and Dynamic Virtual Paths sparingly because you
can easily over subscribe the usage of the bandwidth with little room for data usage.
citrix.com 152
Step Action
8. Under the Connections tile, navigate down to LON > WAN Links > Virtual Paths and
configure an autopath group.
Click the edit icon, then select Default_Group from the autopath group setting.
Autopath groups are defined globally and are a good way to organize your WAN
links. Autopath groups automatically generate Paths between WAN links using
preset parameters. When a pair of local and remote WAN links of the same Access
Type reference the same autopath group, a path is created in both directions between
the links using the globally defined settings for that autopath group. The settings provide
options for outer UDP frame tagging, path encryption, path state sensitivity.
citrix.com 153
Step Action
9. In the Connections tile, navigate to the SJC site node and enable the same dynamic
virtual path option and autopath group setting.
SJC site > Virtual Paths > Dynamic Virtual Paths
 Check Enable Dynamic Virtual Paths
 Set Maximum Dynamic Virtual Paths to 1
SJC site > WAN Links > Virtual Paths

 Autopath Group setting for <DYNAMIC> Virtual Path Service to Default_Group
citrix.com 154
Step Action
10. Navigate to the Global tile > Default Set > Dynamic Virtual Path Default Sets. A new
default set was automatically created when dynamic virtual paths were enabled. The
default set allows for customization of the SD-WAN behavior at a global level. Site
specific customization can be done locally at each node, which will override these global
settings.
The Basic Settings outline the thresholds, which can be customized, that define when to
create or destroy the dynamic virtual path.
For this lab, lets bring down the creation limitions to 300 kbps and 20 pps. Click the edit
icon and adjust those settings.
Click Apply to accept the new settings.

Adjusting the creation limits can allow the dynamic virtual path to come up faster or
slower depending on the kbps or pps thresholds configured. Likewise the removal
limits help determine when the dynamic virtual path should be taken down and traffic
flow should go back to the Static Virtual Path to the MCN.
citrix.com 155
Step Action
11. Save As the new configuration as Exercise11 and Export to change mangement.

 Appliance Staging
13. With the new configuration actively running on the SD-WAN overlay, open the console
window on XenCenter to the SJC_Client VM.
Let’s first make sure that our traceroute connectivity to the LON site has been resolved.
Open a command prompt and issue the “tracert 172.70.1.28” command.
As we can see from the displayed hops, the traffic is first being sent through the Static
Virtual Path to the NYC_SDWAN (172.16.40.3) then to the LON_SDWAN (169.15.71.3)
and finally the destination (172.70.1.28).
With the route export option being enabled, we are now giving the SJC_SDWAN route
options to communicate to the LON subnet through the SD-WAN overlay.
citrix.com 156
Step Action
14. We will need to generate some traffic to meet the threshold requirements for dynamic
virtual path creation. We can do that using iperf.
InXenCenter, navigate to the console window for the LON_Client VM, launch the jPerf
application and configure jPerf for server mode, then click the Run IPerf! button.
3
2
15. In XenCenter, navigate back to the SJC_Client VM and open JPerf in Client mode. Set
the Server address to the LON_Client (172.70.1.28) and set transmit to 1000 before
clicking Run IPerf!.
3 5
2
1
4
citrix.com 157
Step Action
16. As iperf traffic is generated, navigate to the command prompt for the SJC_Client VM,
and run traceroute again to the LON site.
 tracert 172.70.1.28
As we can see from the results. The NYC_SDWAN is no longer a hop. The local
SJC_SDWAN (172.90.1.1) is seen as a hop because it is deployed in edge mode. Then
the next hop is the end of the Dynamic Virtual Path tunnel to the LON_SDWAN
(169.15.71.3) before reaching the destination host (172.70.1.28).
17. Further confirmation of the Dynamic Virtual Path tunnel creation can be done on the
local web interface of either the LON or SJC SD-WAN. Navigate to the Monitoring >
Statistics > Show: Path (Summary) page and refresh for the latest path availability.
You will notice SJC to LON paths listed as Dynamic types in addition to the SJC to
NYC paths that were already there.
Note that the MCN does not display these paths on its local web interface, and the
paths are only available for the two sites that have a dynamic virtual path
available. Also, the dynamic virtual paths will go away after the iperf traffic is eliminated
between the two sites.
citrix.com 158
Step Action
18. Prep the environment for the next exercise by disabling IPerf on both the LON_Client
and SJC_Client VMs.
Exercise Summary
In this exercise, we made use of dynamic routing protocols on SD-WAN to join a device
deployed in Edge mode to the overlay network and allow for direct traffic flow between two client
SD-WAN nodes.
citrix.com 159
Module 4 (Basic): Deployment
Features and Services
citrix.com 160
Module 4 Overview
This module highlights some additional deployment modes commonly seen in SD-WAN
deployments as well as configuration of different SD-WAN services that are available to properly
account for traffic flow.
It is important to have a clear understanding of your deployed SD-WAN environment.
Specifically, what IPs are being utilized for both management and data path communication,
and the appliance deployment modes. The following are the prerequisites:
1. SD-WAN deployed network topology

2. Identify the deployment mode
3. Identify all IP address for both the management plane and the data plane
citrix.com
161
Exercise 12: Configuration of BGP and Intranet Service to
DAL (LON and SJC)
Overview
With the SD-WAN overlay working properly for both Static and Dynamic Virtual Paths, the next
step is to make sure that connectivity between SD-WAN and non-SDWAN sites operates as
well.

 Set up BGP dynamic route learning on SJC_SDWAN
 Configure Intranet Service

Step Action
1. On XenCenter, right-click and start the DAL_CE_Rtr and the DAL_Client VMs.
2. When the DAL site VMs have fully loaded, from the XenCenter Console window on the
SJC_Client VM we can attempt to run a traceroute command to the DAL subnet by
issuing the following command:
 tracert 172.80.1.29
What this tells us is that the SJC SD-WAN acting as the edge device is not learning
routes from the underlay network and only has capability of routing on the overlay. We
can change this behavior by enabling dynamic route learning on the SJC SD-WAN.
citrix.com
162
Step Action
3. In the configuration editor on the MCN (NYC_SDWAN_SE1), open the latest
configuration file and navigate to the Connections tile > SJC > Route Learning > BGP
> Basic Settings.
4. For the SJC node, click the edit icon for BGP > Basic Settings and select Enable as
well as Advertise NetScaler SD-WAN Routes.
Input 169.15.90.2 as the SD-WANs Router ID, and 64590 as its Local Autonomous
System number. Then Apply the new settings.
citrix.com
163
Step Action
5. Navigate down to BGP > Neighbors node and click add (+) to add a neighbor.
Select E2Vlan0 for the Virtual Interface, input 169.15.90.1 as the Neighbor IP. Update
the Neighbor AS to 23457 to reflect what is provided in the topology. Input Password1
for the Password, then click Apply.
You should notice a new warning icon for import filters.
citrix.com
164
Step Action
6. We encountered the same warning icon about import filters when configuring OSPF on
the NYC_SDWAN. We chose to ignore the warning since the SD-WAN was deployed in
Virtual Inline Mode. For the case of the SJC_SDWAN, it is deployed in Edge mode so in
order for the SD-WAN to know how to route to Intranet sites on the MPLS network we
need to enable route learning from the underlay with import filters.
Navigate to SJC > Route Learning > Import Filters and click the add (+) icon.
You will notice that there is an “auto” filter already defined. We just want to import our
intranet DAL sites so input 172.80.1.0/24 in the Destination field.
Scroll the window to the right, and notice that the newly added filter is set to Any (*) for
all the fields but importantly it is set to Include, unlike the default one.
Click Apply to save the new filter.

Use caution if importing all learned BGP routes. It may yield undesired SD-WAN overlay
route tables.
citrix.com
165
Step Action

Since all the devices in the network have connectivity to the MCN, the “ignore
incomplete option should not be used and manual upload of software through local
change management should not be required.
9. With the new configuration file pushed out and active, from the XenCenter Console
window on the SJC_Client VM we can attempt to run a traceroute command again to the
DAL subnet by issuing the following command:
 tracert 172.80.1.29
This is made possible because the configuration for route filters is to learn any underlay
routes being advertised through the enabled dynamic route learning.
citrix.com
166
Step Action
10. Likewise, from the XenCenter Console window on the DAL_Client VM, we can run
traceroute to check connectivity to the SJC branch.
 tracert 172.90.1.31
This was made possible because we enabled the SJC_SDWAN to advertise it’s local
routes to the underlay using the BGP.
11. With SJC_SDWAN delivering traffic to the DAL site, that traffic is considered as
passthrough traffic and the SD-WAN Virtual Path services will need to contend for WAN
link bandwidth against that traffic. Meaning if the MPLS link gets saturated with
passthrough traffic there is no bandwidth available for virtual path usage.
This is where defining traffic that is not being delivered across the Virtual Path service as
either Internet or Intranet service becomes invaluable. SD-WAN can then account for
that traffic, provision bandwidth accordingly and make efficient use of the available WAN
link bandwidth. We should also mention, that by classifying the traffic as Internet or
Intranet service, traffic delivered on those services will also be reported against for
analytics, where passthrough traffic will not.
On the MCN (NYC_SDWAN_SE1) open the latest configuration file and navigate to
Connections > SJC > Intranet Service.
Click the add (+) icon to add a new Intranet Service.
citrix.com
167
Step Action
12. For the newly created Intranet Service, navigate to the WAN links and click the pen icon
to edit. Then select the SJC_MPLS link for Use.
2
3
13. Next we can navigate to the Global tile > Default Sets > Intranet Default Sets. Click
the add (+) icon to add a new Intranet Default Set.
14. In the newly created default set, expand the Rules node and click the add (+) icon to a
add a new filter rule.
Here you can globally define rules to override the Intranet service to either passthrough
or discard traffic as needed. For our use, we don’t need any filters, since we are just
looking to route traffic through our intranet service.
citrix.com
168
Step Action
15. We want to go to the SJC site where we configured an Intranet Service to start using the
default set we created earlier.
Navigate to the Connections tile > SJC > Intranet Services > New_Intranet_Service
> Basic Settings, edit and select “New_Intranet_Default_Set” from the drop down.
Then click Apply to accept the settings.
16. Lastly, we need to create a route on the SJC site to direct the traffic destined for the DAL
subnet to be delivered through the Intranet Service.
Navigate to Connections > SJC > Routes and click add (+) to add a new route.
Input 172.80.1.0/24 for the Network IP Address, select Intranet from the Service Type
drop-down and select New_Intranet_Service from the Intranet Service drop-down.
1
2 3
citrix.com
169
Step Action
17. Save the new configuration as Exercise12 and Export to change mangement.

19. After the new configuration has been pushed to the SD-WAN environment. We can
configure the DAL_Client Jperf application in Server mode.
citrix.com
170
Step Action
20. Configure the SJC_Client in Client Mode and point to 172.80.1.29 as the server with
1000 seconds transmit time. Then click Run IPerf!
3 5
2
21. After the iperf traffic is running, confirm that the SD-WAN is delivering it directly out to
the MPLS WAN link.
This can be done by issuing tracert 172.80.1.29 to the DAL_Client.
citrix.com
171
Step Action
22. Additional confirmation can be done the SJC_SDWAN_SE web interface.
Navigate to Monitoring > Statistics > Show Intranet.
23. SD-WAN will also start logging Intranet flows on the Monitoring > Flows table.
Other reporting pages will include:

 Monitoring > Firewall > Statistics: Connections
 Monitoring > Statistics > Show: WAN Links
 Monitoring > Usage Reports
citrix.com
172
Step Action
24. We can make use of the same New_Intranet_Default_Set to create an Intranet Service
for the LON site as well.
In the Configuration Editor navigate to Connections > LON > Inranet Service. Click
the add (+) icon to add a new Intranet Service for this site.
Under Basic Settings, edit the Default set to use New_Intranet_Default_Set from the
drop-down. Then click Apply.
Under WAN Links, edit and enable the usage of LON_MPLS for Intranet connectivity to
remote sites with no SD-WAN. Click Apply to accept the new settings.
citrix.com
173
Step Action
25. Lastly, nagivate LON > Routes and add (+) an Intranet route for the DAL subnet.
Input 172.80.1.0/24 for the Network IP Address field, select Intranet from the Service
Type drop-down and select New_Intranet_Service from the Intranet Service drop-
down. Click Add to accept the new route.
2 3
1
4
26. Save the new configuration as Exercise12 and Export to change mangement.

citrix.com
174
Step Action
28. With the new configuration running, we can perform the same Iperf communication
between the LON and DAL site and monitoring for successful Intranet classification in
the LON_SDWAN_SE reporting.
29. Once that is complete, make sure to stop iperf before next exercise.
Exercise Summary
In this exercise, we stepped through the setup of Intranet Service for both the SJC and LON site
allowing for underlay communication between SDWAN and non-SDWAN sites.
citrix.com
175
Exercise 13: Internet Service (LON)
Overview
In this exercise, we will introduce you to the Internet Service. Internet Service configuration on
the SD-WAN involves creating default routes where internet traffic can be delivered and
accounted for in WAN link bandwidth usage. Internet Service allows for direct breakout of
internet traffic at any site node.

 Configure Internet Service for the LON site

Step Action
1. On XenCenter, log in to the Console of the LON_Client VM using
Administrator/Password1 credentials.
Open a Command Prompt in Administrator mode and issue the following command:
 tracert 8.8.8.8
Notice that the traceroute goes first to the Management network gateway
(192.168.10.1), this is by design and we will adjust the behavior in the next step.
citrix.com
176
Step Action
2. In the command prompt run the following command to create a default route making
sure that all traffic is exported through the data network of the lab.
route delete 0.0.0.0
route add -p 0.0.0.0 mask 0.0.0.0 172.70.1.1 metric 1
Confirm that the setting was accepted by running traceroute again:
 tracert 8.8.8.8
Notice now that the traffic is being delivered to the data network gateway (172.70.1.1)
for the LON site.
citrix.com
177
Step Action
3. In order for web site URLs (FQDN) to be resolved to a public IP address we will need to
configure DNS on the LON_Client VM. We will use google DNS 8.8.8.8 address.
 Click the Network Connections link available on the desktop.

 Right-click on the interface labeled Data_LonLan and select Internet Protocols
Version 4 (TCP/IPv4), then Properties.
 Manually input 8.8.8.8 as the Preferred DNS-Server.
Click OK to accept the new settings.
4. Now that the LON_Client is configured to route internet traffic through the data network,
we can update the Configuration Editor to enable direct internet breakout from this site.
Open the current configuration file and navigate to Connections > LON > Internet
Services. Click the add (+) icon to create an internet service, then navigate to the WAN
links and select to Use LON_INET for internet access. Click Apply to accept the
settings.
citrix.com
178
Step Action
5. When creating an Internet Service, the system will automatically create an Internet route
for that site. To view this, navigate to Connections > LON > Routes. Instead of
passing through the internet traffic to the underlay network, SD-WAN will now make use
of the Internet Service route.
citrix.com
179
Step Action
6. With traffic directly breaking out to the internet, the system will automatically create a
Dynamic NAT policy to protect the internal IP address. Navigate to Connections >
LON > Firewall > Dynamic NAT Policies to view the update.
Note also that when Dynamic NAT is configured, the firewall policies are also
automatically loaded to allow traffic in and out of the network. To view this, navigate to
Connections > Firewall > Policies.
citrix.com
180
Step Action
7. Lastly, when NAT is configured on the SD-WAN devices it is recommended to enable
Track, which will allow the device to track the NAT connections for longer and not drop
the connections.
Navigate to the Advanced tile of LON > Firewall > Settings > Advanced tile and select
Track from the Default Connection State Tracking drop-down. Click Apply to accept
the changes.
8. We will now specifically stipulate that the default routes are not exported at the LON site.
Otherwise the route would be advertised, and peer SD-WAN devices may route their
internet traffic to this site.
Navigate to LON > Internet Service > Internet > Basic Settings, click the edit icon
and uncheck the Export Default Routes option. Click Apply to accept the new setting.
2
3
citrix.com
181
Step Action

11. In XenCenter, navigate back to the LON_Client Console and open the Chrome web
browser and open a webpage to test for internet connectivity.
citrix.com
182
Step Action
12. In the web browser to the LON SD-WAN we can navigate to the Monitoring > Firewall
page and select NAT Policies from the Statistics drop-down to see that our Dynamic
NAT rule is being utilized.
1 2
13. For this lab PE_INET_Rtr has been pre-configured to perform NAT for traffic destined
for the public internet. We can further confirm NAT functionality on the underlay network
by navigating to the PE_INET_Rtr VM in XenCenter. Run the following command:
 show nat statistics

 show nat translations
citrix.com
183
Step Action
14. Now if we run traceroute from the LON_Client VM it will show broken due the security
feature that has been implemented into the firewall to further protect visibility to the
private IP address. So this is expected.
Exercise Summary
In this exercise, we stepped through the setup of Internet Service for the LON site allowing for
internet traffic to directly break out from the branch office.
citrix.com
184
Exercise 14: Stateful Firewall (LON)
Overview
Citrix SD-WAN 9.2 release introduced a Stateful Firewall integrated into the SD-WAN
technology. The firewall allows for policy creation between SD-WAN services and firewall
zones and supports Network Address Translation.
With the integrated Firewall, Citrix SD-WAN enables device consolidation and simplifies
deployment for the WAN Edge. The Firewall feature also enables secure direct internet access
at the branch with application centric firewall policies.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/stateful-firewall-

nat-support.html

 Investigate some of the firewall configuration options
citrix.com
185
Step Action
1. Firewall policies can be created at the Global configuration level if policies are planned
to be uniform across all sites, or firewall policies can be local to each site and override
the global configurations. Firewall policies can be defined using Zones and Firewall
Policy Templates. The firewall capabilities allow for implementation of consistent
security policies across the entire SD-WAN network and allow consideration for direct
internet access for branch sites.
Open the web interface to the NYC_SDWAN_SE1 VM and navigate to Configuration >
Virtual WAN > Configuration Editor. Open the latest configuration file.
1
2
2. Select the Advanced tab to get the detailed options for building a configuration.
citrix.com
186
Step Action
3. Expand the Global tile, to view the newly available Firewall node.
Expand the Firewall node. There are two sub-categories for firewall.
 Zones
 Firewall Policy Templates
Firewall zones define flow for traffic entering or leaving the system for policy definition.
Citrix SD-WAN creates the following default zones:
 Default_LAN_Zone – applies to traffic to or from an object with a configurable

zone, where the zone has not been set
 Internet_Zones – applies to traffic to or from the Internet services using Trusted
Interfaces
 Untrusted_Internet_Zones – applies to traffic to or from the Internet Service
using an Untrusted Interfaces
citrix.com
187
Step Action
4. Click the add (+) icon to add a new Firewall Zone.
Name it Intranet_Zone, then click Apply.
An admin can create their own zones and assign them to the following types of objects:
• Virtual Network Interfaces (VNI)
• Intranet Services
• GRE Tunnels
• LAN IPsec Tunnels
The source zone of a packet is determined by the service or virtual network interface a
packet is received on. The exception to this is the virtual path service. When traffic
enters a virtual path, packets are marked with the zone that originated the traffic and
that source zone is carried through the virtual path. This allows the receiving end of the
virtual path to make a policy decision based on the original source zone before it
entered the virtual path.
citrix.com
188
Step Action
5. Navigate to the Sites tab and expand the Interface Groups node for the LON site.
We can make use of the new Intranet_Zone and identify the Virtual Interface
E1E2Vlan0 as using that defined zone. Click Apply to accept the settings.
1
2
Note: The Internet facing Interface Group (E3Vlan0) is default configured for the
Untrusted_Interent_Zone because of its Untrusted security type configuration.
citrix.com
189
Step Action
6. Firewall Policies provide the ability to allow, deny, reject, or log specific traffic flow.
Applying these policies individually to each individual site would be difficult depending
on the scale of the deployment. To resolve this issue, groups of firewall filters can be
created with a Firewall Policy Template.
Navigate back up to the Global > Firewall node, click the Firewall Policy Templates
and click the add (+) icon to add a new template.
A Firewall Policy Template can be applied to all sites in the network or only to specific
set of sites. These policies are ordered as either Pre-Policies or Post-Policies.
Both network-wide Pre-Policies and Post-Policies are configured at the Global level.
Local policies are configured at the site level under the Connections tile and apply only
to that specific site.
Pre-Policies are applied before any local site policies. Local site policies are applied
next, followed by Post-Policies. The goal is to simplify the configuration process by
allowing site specific policies that will override the globally defined defaults.
citrix.com
190
Step Action
7. Let’s proceed by creating a Pre-Policy. Expand the Pre-Policies node and click the add
(+) icon to add a new policy.
In the Edit Firewall Policy window, configure the following:
 Priority: 100 (default)

 From Zone: Any
 To Zone: Intranet_Zone
 Action: Drop
 Match Type: Application
 Application: Internet Control Messaging Protocol(ICMP)
 Enable Reverse Also
Click Add to accept the new pre-policy template.
3 4
citrix.com
191
Step Action
8. We can now navigate to the individual site nodes and enable to template for the site.
Navigate to Connections tile, select LON > Firewall > Settings. Click the + Add
button to add a template. Select New_Policy_Template from the drop-down and
select Apply.
9. Navigate to the Policies node for the LON site and expand the Policies node.
The Pre-Appliance Template Policies were inherited by the Global template (Drop ICMP
traffic). The Local Policies were created in an earlier exercise when defining Dynamic
NAT for this site.
citrix.com
192
Step Action
10. We can further define Local Policies for the LON site to control which traffic is allowed
or not allowed from this specific site. Expand the Firewall > Policies > Local Policies.
Click the +Add button to add a new policy.
citrix.com
193
Step Action
11. In the Add Firewall Policy window, configure the following:
 Priority: 100 (default)

 From Zone: Any
 To Zone: Untrusted_Internet_Zone
 Action: Reject
 Match Type: Application
 Application: Facebook(facebook)
Click Add to accept the new pre-policy template.
3 4
12. In the Configuration Editor, click the Save As button and save the new configuration as
Exercise14. Then, click Save.
citrix.com
194
Step Action
13. Click Export and export the new configuration to Change Management Inbox.

citrix.com
195
Step Action
15. After the configuration, has been pushed to the SD-WAN devices open XenCenter
Console to the LON_Client virtual machine.
Open command prompt window and ping the Intranet site (DAL_Client 172.80.1.29).
ping 172.80.1.29
1
3
With the Local LON firewall policy created to drop ICMP traffic to the Intranet Zone, the
ping traffic will fail.
16. We can further confirm the firewall policy has been enacted by navigating to the
LON_SDWAN_SE web interface and navigating to the Monitoring > Firewall >
Statistics: Filter Policies.
2 3
citrix.com
196
Step Action
17. The table for the firewall policies will indicate that the ICMP application is being seen by
the DPI engine and that the policy in place is dropping the traffic.
18. In the LON_Client VM, open the Chrome browser and attempt to navigate to
https://facebook.com.
1 2
citrix.com
197
Step Action
19. Navigate to the Firewall Statistics page for the LON_SDWAN_SE web interface and
further confirm that the firewall policy was rejecting the connection attempt.
Exercise Summary
In this exercise, we configured the integrated Firewall to block ICMP traffic across the Intranet
Zone and Reject Facebook traffic to the Untrusted Internet Zone.
citrix.com
198
Exercise 15: Routing Domains for Internet Service (LON)
Overview
Citrix SD-WAN introduced Virtual Routing and Forwarding (VRF) to allow SD-WAN routing
support for LAN segmentation across the SD-WAN Overlay. SD-WAN 9.1 introduced Virtual
Routing and Forwarding capabilities, to allow for segmentation across the SD-WAN Overlay. In
release 9.2 the feature was extended to provide VRF Firewall segmentation to give multiple
routing domains access to the internet through a common interface, with each domain’s traffic
isolated from that of the others. For example, employees and guests can access the internet
through the same interface, without any access to each other’s traffic.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/virtual-routing-

and-forwarding-sd-wan/how-to-configure-firewall-segmentation.html

 Configure Firewall Segmentation for internet access

Step Action
1. The goal of this exercise will be to segregate employees and guest internet access
through the same interface, with the aim to not compromise the employee traffic.
The prerequisite to configure multiple routing domains for internet service include:
 Internet Service with assigned WAN link

 Firewall NAT with correct policies
 Routing policies defined globally
From our previous exercise, for our LON site we already created local internet breakout
from that site by defining an Internet Service to use the available local internet WAN
link. Which also created the needed dynamic NAT and firewall policies to allow the
traffic out to the internet.
citrix.com
199
Step Action
2. First, we will need to globally define the routing domains. On the NYC_SDWAN_SE1
web interface, open the latest configuration file, and navigate to Global > Routing
Domains.
Click add (+) to add a new Routing Domain.
Enter “Guest” for the new field and rename the Default_RoutingDomain to “Employee”.
Not that the Employee routing domain will be default.
Click Apply to save the changes.
3. The new Guest routing domain will automatically be added to the MCN site. Since we
are only intending to use this routing domain at the LON site, we can disable the usage
of it at other sites.
Navigate to Sites > NYC > Routing Domains. Click the edit icon and uncheck the
enable field for the Guest routing domain.
Then click Disable on the pop-up window.
citrix.com
200
Step Action
4. We will be using the new routing domain at the LON site, so navigate to the Sites >
LON > Routing Domains, click the edit icon and enable the Guest routing domain by
checking the enable option.
2
3
5. With the routing domain made available, we can now navigate to the Sites > LON >
WAN Links > LON_INET > Access Interfaces and enable the option for Internet
Access for All Routing Domains.
Click the edit icon to make the change, then click the Apply button to accept the
changes.
1
2
3
Selecting this option allows SD-WAN to use this access interface for internet service on
all routing domains.
citrix.com
201
Step Action
6. Lastly, we need to define LAN interfaces where the VLAN for the Guest traffic is
expected to enter the box.
Navigate to Sites > LON > Interface Groups and expand the E1E2Vlan0 Interface
Group.
Click the edit icon and edit the E1E2Vlan0 Interface group and click add (+) to add a
new Virtual Interface to handle our incoming Guest Wifi traffic.
Name it E1E2Vlan10, select Guest for the Routing Domain, and enter 10 for the VLAN
ID.
2
3

Routing domains require some physical barrier to separate the traffic. Virtual Interface
is that barrier and is are the starting point for the routing domains. A Virtual Interface
refers to the paring of a VLAN and Ethernet Interfaces. The object is unique to the
routing domain, but one can create multiple of them in different domains. A Virtual IP
address is specific to a Virtual Interface, and Virtual Interfaces cannot exist in two
routing domains. However, the same VIP can be defined in multiple domains.
citrix.com
202
Step Action
7. On the LAN side, we will need a Virtual IP Address for each routing domain. We
already defined a new VLAN 10 for the Guest Wifi and creating a new Virtual Interface
to capture that VLAN traffic across our Interface Group with consists of interface eth1
and eth2.
Navigate to Sites > LON > Virtual IP Addresses and click add (+) add a new Virtual IP
Address. Enter an IP address (172.70.10.27/24) to be used in the configured VLAN,
select Guest as the Routing Domain, and select the Virtual Interface configured
specifically for the new VLAN (E1E2Vlan10).
1
2
Click Apply to accept the new settings.

The NAT and Firewall policies created when configuring the Internet Service will
translate connections to proper routing domains.
Routing domains can also be configured for traffic that traverses the Virtual Path. In
this scenario you don’t need to define access points in all domains. You still only need
one access interface. The routing domain information is carried with the packet across
the Virtual Path to maintain separation at the remote end. The routing domain
information is carried with the packet inside our encapsulation for Virtual Paths.
citrix.com
203
Step Action

10. With the new configuration running. We validate the two routing tables available for the
individual Routing Domains.
Navigate to Monitoring > Statistics > Show (Routes) for the LON SD-WAN web
interface.
citrix.com
204
Step Action
11. Navigating to the Monitoring > Firewall > Statistics (NAT Policies), we can see that
we can monitor for any NAT traffic using Routing Domains as one of the filters.
Exercise Summary
In this exercise, we stepped through configuring a new Guest routing domain to segregate Wifi
internet traffic from employee traffic for local internet service from a branch office.
citrix.com
205
Exercise 16: High Availability Deployment (NYC)
Overview
Citrix SD-WAN deployments can include high-availability to increase availability of the SD-WAN
Overlay during failure scenarios of SD-WAN hardware or software.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/ha-deployment-

modes.html

 Configure high availability for the NYC SD-WAN deployment

Step Action
1. The NYC_SDWAN_SE1 is deployed in Virtual Inline mode. Generally, the headend
SD-WAN is recommended to be deployed as a High Availability (HA) pair of devices.
On the XenCenter app, right-click and Start the NYC_SDWAN_SE2 VM.
You may have to shut down some of the other VMs if resource limitation is preventing
the NYC_SDWAN_SE2 from starting. It would be recommended to shut down
DAL_CE_Rtr and DAL_Client.
citrix.com
206
Step Action
2. In XenCenter, select the Console tab for the NYC_SDWAN_SE2 VM and log in with
admin/password credentials. Assign a unique management IP address by issuing the
following commands:
 management_ip
 set interface 192.168.10.24 255.255.255.0 192.168.10.1
 apply, y
3. Before opening the web interface of the NYC_SDWAN_SE2 VM. We will need to build
the configuration for this secondary HA unit.
Navigate to the NYC_SDWAN_SE1 web interface and open the latest configuration file
in the Configuration Editor. In the Advanced tab, navigate to the Sites > NYC > High
Availability node.
citrix.com
207
Step Action
4. In the High Availability node for the NYC site. Click the edit icon and Enable High
Availability, name it as NYC-CBVPX-HA
Click the add (+) button and configure the following HA IP Interface:
 Virtual Interface: select E1Vlan0(0)

 Primary: 172.16.40.23
 Secondary: 172.16.40.24
1
2
5. Save As the configuration as Exercise16. Then Export to Change Management.
citrix.com
208
Step Action
Activation (Activate Staged)
7. After the Change Management is complete, download the active package from the
NYC-NYC-CBVPX-HA site.
It is important to select the correct package for the HA unit. If the same package was
uploaded to two SD-WANs in an environment. They HA systems would have the same
identify and would not function properly.
citrix.com
209
Step Action
8. Open web interface to NYC_SDWAN_SE2 VM. Navigate to Configuration > System
Maintenance > Local Change Management. Click Choose File and select the HA
package downloaded earlier and Upload.
Click Next and Activate Staged.
9. Navigate to the NYC_SDWAN_SE2 VM Configuration > Virtual WAN >

Enable/Disable/Purge Flows. Click the Enable button to enable the SD-WAN service
on the HA unit.
citrix.com
210
Step Action
10.
1
2
Navigate to the Monitoring > Statistics page of the HA unit to confirm it has the HA
identify.
11. In the XenCenter application, open the Console tab to the LON_Client VM, and
enable the iperf application in Server Mode, then click Run Iperf!
1
3
2
citrix.com
211
Step Action
12. On the XenCenter app, navigate to the Console tab of the NYC_Server VM and run
iperf in Client Mode, set the Transmit time for 1000 seconds, the Server address as
172.70.1.28, and click the Run Iperf button.
2 3 5
1 4
13. As the iperf traffic is running lets initiate the HA failover by shutting down the
NYC_SDWAN_SE1 VM. Right-click and select Shut Down.
citrix.com
212
Step Action
14. Navigate back to the Console tab of the LON_Client VM to monitor the health of the
iperf session as the HA secondary unit takes up responsibility of delivery across the
Virtual Path.
15. We can confirm that the secondary HA unit has picked up responsibility of the overlay
by refreshing the browser to NYC_SDWAN_SE2 (https://192.168.10.24).
citrix.com
213
Step Action
16. Restore the NYC_SDWAN_SE1 as the primary SD-WAN by restarting it and shutting
down the NYC_SDWAN_SE2.
Exercise Summary
In this exercise we configured a secondary high availability unit for the NYC_SDWAN_SE unit
deployed in Virtual Inline mode.
citrix.com
214
Exercise 17: Standby Links (LON)
Overview
In this exercise, we will introduce Metered and Standby Links which is a feature that provides
business logic to conserve bandwidth on links that are billed based on usage. The feature
lowers it path assessment frequency and provides ability to email alert on meeting/exceeding
user defined byte usage count thresholds.
Links defined as Standby disallows usage of the link unless all other non-metered links are in
down or degraded state. Standby links also have two sub-features:
 On-demand: a standby link that becomes active when bandwidth thresholds on the
Virtual Path are meet.
 Last-resort: a standby link that becomes active only when all non-standby links and on-
demand standby links are dead or disabled.
Reference (USA hyperlink): https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/standby-wan-
links.html

 Introduce a new 4G/LTE WAN link to the LON site
 Enable Standby links for the LON 4G/LTE WAN link
citrix.com
215
Step Action
1. Open the web interface to the NYC_SDWAN_SE1 VM, then log in with default
credentials (admin/password).
Open the latest configuration file and navigate to Site > LON > Interface Groups and
click add (+). Create a new interface group using the following settings:
Click Apply to accept the setting.
3
4
5
6
citrix.com
216
Step Action
2. Navigate to LON > Virtual IP Addresses. Click add (+) to create a new IP address and
enter the following settings:

 Routing Domain: Employee
3. Navigate to LON > WAN Links. Click the add (+) to add a new WAN link to the LON
site. In the Add WAN Link window, enter LON_LTE as the name and select Public
Internet as the Access Type. Click Add to save the setting.
citrix.com
217
Step Action
4. For the newly created WAN link, open the Settings node then click edit. For the Basic
Settings, configured the LAN to WAN (upload) rate to 2000 Kbps and configured the
WAN to LAN (download) to 10000 Kbps.
Navigate down and open the Metered/Standby Link for the LON_LTE WAN link, then
configure the following:
 Enable Metering: check

 Data Cap (MB): 1000
 Billing Cycle: Monthly
 Starting From: 01/01/2018
 Standby Mode: Last-Resort
 Standby Heartbeat Interval: 1 second
2
3
citrix.com
218
Step Action
5. Navigate to the Access Interface node of the new WAN link and add (+) the following:
 Routing Domain: Employee

 IP Address: 169.15.72.3
 Gateway: 169.15.72.1
2
3
6. With our configuration complete, Save As Exercise17 and Export to Change

Management.
1 4
7. Follow the Change Management process, which you should already be familiar with
from previous exercises.
citrix.com
219
Step Action
8. After completing the activation process, open the web interface to the LON_SDWAN_SE
VM and navigate to the Monitoring > Usage Reports page to see the report of usage
on your metered link.
citrix.com
220
Step Action
9. Another key thing to notice is that if you navigate to the Monitoring > Statistics page.
The LON_LTE links is denoted as (standby-m) and the usage of the LTE WAN path is
lower compared to the other WAN links. Currently there is no traffic flowing through the
SD-WAN so all that is being reported in the kbps column is the heartbeat going back
between the sites. When there is no traffic between sites, the SD-WAN systems are
forced to send heartbeat packets between sites to determines the state (latency, loss,
jitter) of the links in each direction, in the event applications start to flow.
In the configuration of the standby heartbeat interval we selected 1s, but we could have
selected 10s which lessens the frequency and significantly lessens the bandwidth being
consumed by synthetic traffic. Available options were: default
1s/2s/3s/4s/5s/6s/7s/8s/9s/10s/disabled.
2
3
citrix.com
221
10. We will now test the functionality of the standby LON_LTE link.
We will need to push traffic onto this link to further see the functionality.
1. On XenCenter, log in to the NYC_Server VM (Administrator\Password1)
2. Open the JPerf application, configure it for Server Mode and click the Run Iperf!
button.
3
2
3. Log in to the LON_Client VM, launch the JPerf app and configure for Client
Mode, transmit time 1000, and IP Address 172.16.10.12, then click Run iPerf!
2 4 5
4. Navigate to the LON_SDWAN_SE GUI Monitoring > Statistics page to see the
usage of the links. Enable Auto Refresh and click the Start button. We can
make note of the following:
 Traffic is delivered across the LON_MPLS and LON_INET links. This

can be seen by looking at the kbps column of the Path Statistics.
 The LON_LTE link is still only sending probe traffic to check the health of
the link.
citrix.com
222
1
5. Open a new tab to the WANem web interface. We will induce 100% loss on both
the MPLS and INET WAN link to see the behavior of traffic delivery. Start by
selecting eth1 and click Start. Enter 100 for Loss % and click Apply Settings.
3
1
4
2
6. With the MPLS path no longer available, SD-WAN will be forced to use the other
available paths, leaving only the INET path available since LTE is set as a link of
last resort (notice the LTE path is still reporting < 1 kbps consisting of only
heartbeat traffic). We can see on the JPerf graph that utilization drops from 4
Mbps down to 2 Mbps.
citrix.com
223
7. Now we can bring down the INET link by selecting eth3 and apply 100% loss on
settings.
1
3
8. We will now force the LTE link to be used, by also taking down the last remaining
non-metered path, and we can notice a momentary dip in throughput.
9. We should now be able to observe that data is flowing across the last remaining
LON_LTE path with both MPLS and INET links down.
citrix.com
224
10. On the WANem, we can bring back the INET link by eliminating the loss (set loss
% to 0) on the eth3, and we can notice the SD-WAN moving traffic off of the LTE
link.
11. SD-WAN quickly merges traffic onto the non-metered link, and the traffic dip is
not noticed.
12. We can further confirm that SD-WAN is no longer using the LON_LTE link for
delivery of traffic across the Virtual Path.
citrix.com
225
Step Action
11. Reset the environment for the next exercise by bringing up the MPLS path also on the
WAN emulator by setting the Loss % to 0 on eth1.
Exercise Summary
In this exercise, we introduced standby links, forced failures on all other non-metered links to
force usage of the link of last resort and the investigated the effect on traffic throughput across
the Virtual Path.
citrix.com
226
Module 5 (Advanced): Pre-stage
Environment
citrix.com
227
Module 5 Overview
This module provides instructions to start with a newly provisioned lab environment with pre-
staged configuration to allow the student to be introduced with the latest Citrix SD-WAN
features.
It is critically important that you have basic understanding of the lab environment, you must
complete the following pre-requisites:
1. Familiarize yourself with underlay network details

2. Identify the SD-WAN deployment mode for each site
3. Understand all the needed IP address for both the management plane and the data
plane
4. HAVE REFERENCE TO A COMPLETED SD-WAN NETWORK TOPOLOGY
citrix.com
228
Exercise A: Import pre-staged configuration to SD-WAN
devices
Overview
In this exercise, we provide the steps needed to stand up the SD-WAN environment with pre-
staged configuration to begin exercises of newly introduced features of Citrix SD-WAN.
After performing the steps in this exercise, you may advance forward to any of the advanced
modules that focus on features that were added to the SD-WAN solution through new software
releases.
NOTE: If you have reached this exercise after first performing the basic lab modules and
manually configuring the SD-WAN devices, you can skip this first exercise of importing
of pre-staged packages.

 Power on virtual machines
 Promote the MCN node
 Stand up the client nodes
 Upload pre-staged config/software packages

Step Action
1. From the lab portal, log in to the lab session by launching your Student Desktop with the
Launch Lab button. You will need the latest Citrix Receiver installed (USA hyperlink -
https://www.citrix.com/go/receiver.html )
If you encounter the Microsoft Windows pop-up, click the Restart Later option.
citrix.com
229
2. In the student desktop the XenCenter application should automatically start. If it does
not, launch the application from the Start menu. In the XenCenter infrastructure pane,
right-click on the “XenCenter” node, and select Add…
3. Log in using the “XenServer” credentials supplied via the lab web portal. Make sure
that you use the user name “admin”, and use the password listed in the lab portal.
Then click Add.
Close the Health Check Overview pop-up window.
citrix.com
230
4. The environment will begin with some virtual machines (VMs) powered on and others
powered off. We will first power ON the following virtual machine by right-clicking the
selected VM, then selecting Start:
 NYC_SDWAN_SE1
While that VM starts, proceed to the next steps.
5. Before starting the LON_Client VM, update the interfaces setting.

With the VM shutdown, navigate to the Networking tab, and select Device 1. Then
click Properties button.
From the Virtual Interface Properties pop-up window, select LON_LAN from the
Network drop down and click OK to accept the new setting.
2
1
3
4
6
After the interface has been updated right-click and Start the LON_Client VM.
citrix.com
231
6. Once the LON_Client VM has started, navigate to the Console tab, click the Send
Ctrl+Al+Del button. Log in with Administrator/Password1 credentials
Open a Command Prompt and run the following in the command prompt.
 route delete 0.0.0.0
 route add -p 0.0.0.0 mask 0.0.0.0 172.70.1.1 metric 1
citrix.com
232
7. Allow the NYC_SDWAN_SE1 VM some time to fully boot (2-3 mins). Once the
following VMs have started, we will assign a management IP address to the SD-WAN.
On the XenCenter application, first select the NYC_SDWAN_SE1 VM, then navigate to
the Console tab, and press the Enter key to get into the login prompt.
 Log in with credential admin/password
 Issue the management_ip command
 Issue an IP address using the set interface command as outlined in the table
below for each individual SD-WAN VM
set interface 192.168.10.23 255.255.255.0 192.168.10.1
 Issue the Apply or y command to accept the new setting
citrix.com
233
8. Next, right-click and Start the LON_SDWAN_SE VM.
9. Allow the LON_SDWAN_SE VM some time to fully boot (2-3 mins). Once the following
VMs have started, we will assign a management IP address to the SD-WAN.
On the XenCenter application, first select the LON_SDWAN_SE VM, then navigate to
the Console tab, and press the Enter key to get into the login prompt.
 Log in with credential admin/password
 Issue the management_ip command
 Issue an IP address using the set interface command as outlined in the table
below for each individual SD-WAN VM
set interface 192.168.10.27 255.255.255.0 192.168.10.1
 Issue the Apply or y command to accept the new setting
citrix.com
234
10. If the IP addresses were assigned properly, you should be able to open the web
browser to each virtual machine management IP address.
On the Student Desktop, open a Chrome browser then navigate to each VM’s web
interface in a separate browser tab. A bookmark link to each VM is available.
3
2 4
1
Click the Advanced and Proceed option to advanced past the browser cert error.
citrix.com
235
11. On the NYC_SDWAN_SE1 VM, login with admin/password credentials.
We will begin with the NYC_SDWAN_SE1 (192.168.10.23) VM on the first tab.
 Navigate to the Configuration > Appliance Settings > Administrator

Interface > Miscellaneous tab
 Click Switch Console button
1
2
Click OK in the pop-up to accept the changes, then log back into the NYC_SDWAN
web interface with admin/password credentials.
Here you can also update the Web Console timeout setting so that your connection
does not time out while implementing the configuration changes in the exercises.
citrix.com
236
12. On the NYC_SDWAN_SE1, navigate to the Configuration > Virtual WAN >
Configuration Editor page.
Click the Import button, then Browse.
2
3
13. In the explorer pop-up window, navigate to Desktop > SD-WAN Saved
Configurations, and select the Exercise13.cfg file, then click Open.
Click Import after the file is selected.
14. With the file successfully imported, click the Save button, then Export.
Select Change Management inbox for the destination, then click Export.
1 2
citrix.com
237
15. Navigate to the Change Management page and run through the steps to push the
configuration. Click Begin to start the process.
16. Click the Choose File button to upload the 9.3.3 software package which is available in
on the Desktop > SD-WAN Software > 9.3 > Platforms directory.
citrix.com
238
17. Click the Upload button after the software package has been selected, then step
through the remainder of the change management process.
a) Make sure the Configuration file is in the inbox (Exercise13.cfg)
b) Click Stage Appliances
c) Accept the End User License Aggreement, and click Ok
d) Since the MCN does not have connectivity to both the LON and SJC SD-WAN,
we should see 100% completion, however LON and SJC SD-WAN must
manually be configured via local change management if they are to join the SD-
WAN Overlay.
Click Next to continue.
e) Click Activate Staged

f) Approve the activation by clicking OK in the pop-up window.
g) Wait for the GUI should automatically forward to the Local Change Mangement
page for the NYC_SDWAN_SE1 and then click Activate Statged.
h) Click Ok to accept the change in the pop-up window.
citrix.com
239
18. After the local change management is complete, click Done, which redirects the web
interface to the Dashboard page, it should indicate that the Virtual WAN Service needs
to be enabled. Click the hyperlink to navigate to the Configuration > Virtual WAN >
Enable/Disable/Purge Flows page to do so.
Click the Enable button to enable the service on the MCN.
Click OK on the pop-up window. The system will take a few minutes to successfully
enable.
citrix.com
240
19. Navigate to the Configuration > System Maintenance > Date/Time Settings and
update to timezone setting for each device.
LON_SDWAN_SE Time Zone:
Europe/London 1
NYC_SDWAN_SE1 Time Zone: US/Eastern

1
citrix.com
241
20. With the Service enabled on the NYC_SDWAN_SE1 (MCN), navigate to the Change
Management page to download the package for the remote site (LON).
Make sure to click the active link specific to the LON site row.
Make sure the file is fully downloaded.
citrix.com
242
21. On the browser tab that has the LON_SDWAN_SE (192.168.10.27) web interface. Log
in and navigate to the Configuration > System Maintenance > Local Change
Management. Select Choose File to upload the recently downloaded LON package,
then click Open.
1
2
4
Then click Upload.
22. Click Next, then Activate Staged.
Click OK in the pop-up window to accept the changes.

After the activation completes, click the Done button.
citrix.com
243
23. Lastly, navigate to the Configuration > Virtual WAN > Enable/Disable/Purge Flows
page and click the Enable button to enable the service.
Click OK on the pop-up window to accept the change.
Click OK in the pop-up window to accept the change.
24. Navigating to the Monitoring > Statistics page on the LON_SDWAN, and view the
current state of the WAN links, with the LON_INET to NYC_INET path being marked as
down, which is expected.
In the previous module labs we covered troubleshooting dead path state, in the next
step we will get right to the solution.
citrix.com
244
25. On XenCenter, navigate to the NYC_INET_Rtr VM. Log in with vyatta/Password1
credentials, then issue the following commands.
configure
set protocols bgp 64511 redistribute static metric 2
commit
save
exit
26. The down path on the SD-WAN should come up into a good state, if auto refresh is
enabled. Refresh the page if not until the path state changes.
citrix.com
245
27. In previous lab modules we also configured a redirection method for the virtual inline
deployed SD-WAN at the NYC site, we will skip the detail here and just issue the
needed commonds to enable dynamic route learning.
On XenCenter navigate to the NYC_Core_Rtr VM, log in with vyatta/Password1

credentials and run the following commands:
configure
set protocols ospf area 0.0.0.3 area-type normal
set interfaces ethernet eth3 ip ospf network broadcast
set interfaces ethernet eth3 ip ospf authentication plaintext-password Password
set protocols ospf area 0.0.0.3 network 172.16.40.0/24
set protocols ospf neighbor 20.0.1.14
commit
save
exit
citrix.com
246
28. Next we will test the network to make sure it is using the SD-WAN overlay network by
testing from the end host machines.
On the XenCenter app, right-click and Start the NYC_Server VM.
29. With Good path state, we can validate proper packet flow by running traceroute from
NYC_Server host to LON_Client host (172.70.1.28) to confirm delivery using the SD-
WAN Virtual Path tunnel as the taken hops.
Log in to the NYC_Server VM by clicking the Send Ctrl+Alt+Del button, then using
Administrator/Password1 credentials.
Launch Command Prompt, by right-clicking the Start menu, then selecting Command
Prompt. Issue the following command.
 tracert 172.70.1.28
1
2
citrix.com
247
30. You can also run traceroute in the reverse direction from the LON_Client and
SJC_Client VMs to the NYC_Server (17.16.10.12). Log in to the LON_Client VM by
logging in with Administrator/Password1.
 tracert 172.16.10.12
Exercise Summary
In this exercise, we loaded a pre-staged configuration and perform some underlay network
changes to quickly restore the SD-WAN lab environment to a working state.
citrix.com
248
Module 6 (Advanced): Central
Management with SD-WAN
Center
citrix.com
249
Module 6 Overview
This module introduces SD-WAN Center to the deployment to provide a central management
tool to address the need for better analytics and other centralized management functions as the
deployment increases in the number of active nodes.
It is critically important that before configuring the NetScaler SD-WAN solution for any
environment, you complete the following pre-requisites:

and the data plane
citrix.com
250
Exercise B: Setup SD-WAN Center
Overview
This lab exercise will introduce you to SD-WAN Center, which serves as a central controller for
the SD-WAN environment. We will walk through the basic installation and cover how SD-WAN
Center interoperates with the different devices in the SD-WAN environment.
For further detail about SD-WAN Center please reference the docs: (USA hyperlink)
https://docs.citrix.com/en-us/netscaler-sd-wan-center/9-3.html

 Install SD-WAN Center
 Integrate SD-WAN Center in to the running SD-WAN environment

Step Action
1. On the XenCenter app, right-click and Start the NYC_SDWAN_Center VM.
citrix.com
251
Step Action
2. Navigate to the Console tab and hit the Enter key on your keyboard and log in with
admin/password credentials.
3. Issue the command management_ip and press Enter. Then issue the following
command to configurate a management IP address:
set interface 192.168.10.25 255.255.255.0 192.168.10.1
Press the Enter key, then y to accept the changes.
citrix.com
252
Step Action
4. In the web browser, open a new tab to the SD-WAN Center (192.168.10.25) web
interface, log in to SD-WAN Center with the default credentials admin/password after
proceeding past the browser certification error.
5. Navigate to Administration > Global Settings > TimeZone and select the desired
timezone location (e.g. America > New_York) and click Apply.
2 3
citrix.com
253
6. We can now begin the task of establishing the communication between SD-WAN
Center and the SD-WAN that is the assigned as MCN. Typically, these two
components reside at the same location, but just need IP connectivity to one another on
the management interfaces.
a) On the SD-WAN Center web interface navigate to Configuration > Network
Discovery > SSL Certificates, then click Download Certificate.
1
2
3
b) The HTTPS Certificate (WWCSSLCert.pem) will automatically save to the

Download direction of the Student Desktop.
c) Navigate to the NYC_SDWAN_SE1 (MCN) web interface in the browser and
navigate to the Configuration > Virtual WAN > SD-WAN Center Certificates
page and click the Choose File button to upload the cert we just downloaded
from SD-WAN Center.
d) Locate the WWCSSLCert.pem file in the Downloads directory, and click Open
citrix.com
254
e) Click the Upload and Install button, then click Continue after the message
“Upload and install of the new Virtual WAN SSL Certificate successful.”
1 2
f) Next navigate back the SD-WAN Center user interface and go to the
Configuration > Network Discovery > Discovery Settings tab, and enter the
IP address (192.168.10.23) of the Master Control node and click the Test
button.
2
3
g) Since the test indicates successful communication with the management IP of

the MCN, you can click Discover to establish the relationship.
h) In the Polling Configuration, set “Polling Interval (mins)” to 2 and click Apply.
This denotes the frequency at which data is collected from the SD-WAN
appliances. 2 minutes is the minimum, so keep this in mind when you run traffic
citrix.com
255
Step Action
through the appliance, and don’t observe SD-WAN Center reporting on it
immediately. This is because SD-WAN Center is to be used as a historical data
collector and live reporting and troubleshooting should be done local to the
appliance.
1
2
i) Confirm the addition of all SD-WAN environment appliances by navigating to the

Configuration > Network Discovery > Inventory and Status page. Select the
options for polling, and lick Apply.
citrix.com
256
Step Action
7. Navigating to another tab or page, you may notice a warning icon banner indicating to
“add a separate disk to store statistics…”
This is expected on each SD-WAN Center installation, since the default storage on the
VM when created is 8GB and depending on your SD-WAN environment the storage
requirements will be variable: https://docs.citrix.com/en-us/netscaler-sd-wan-center/9-
3/before-you-begin.html
citrix.com
257
8. We next need to add some additional storage that what was initially added during the
import of this VM. Since this is a lab deployment, we will simply add an additional
20GB of storage for this VM on our hypervisor.
a) From the Student Desktop, Shut Down the NYC_SDWAN_Center VM from
XenCenter
b) On the Storage tab for this VM, click Add. In the “Add Virtual Disk” pop-up
window, enter a Name: SDWC, and size (e.g. 20GB), then click Add. This
additional storage is highly dependent on the number of sites this SD-WAN
Center is expected to handle, it could be several terabytes of storage of the
Center is expected to handle 512 SD-WAN nodes with 30 days of historical
data.
1
3
citrix.com
258
c) Start the VM, then navigate back to the browser user interface to switch the
Active Storage to the new data store.
d) After the SD-WAN Center VM has fully booted, log back in to the web
management interface with admin/password credentials
e) Navigate to the Administration > Storage Maintenance page, Activate the
newly added storage host and click Apply
3
4
f) Confirm by clicking Switch in the pop-up window to delete all existing files,
confirm by clicking Switch again to switch the active storage system.
Note that this procedure is the same when the storage size needs to be
increased. The Migrate Data option is automatically enabled, and the data from
the older smaller storage will be sync’d across to the new larger storage system.
g) SD-WAN Center will “enter maintenance mode” to perform this operation, when
complete click the Continue button.
citrix.com
259
Step Action
9. With SD-WAN devices successfully linked to SD-WAN Center, those appliances will
systematically send information to SD-WAN Center as a central collector. Since SD-
WAN Center provides a historical view of the SD-WAN environment instead of a real-
time view, the Reporting page will take a few minutes before populating with useful
data after the configuration has been imported. Note that the reporting provides views
for the last Hour, Day, Week, and Month.
Exercise Summary
In this exercise, you were introduced to SD-WAN Center, and setup communication with the
MCN to poll data from all devices in this SD-WAN environment.
citrix.com
260
Exercise C: SD-WAN Center Network Configuration
Overview
With SD-WAN Center in the network, we can make use of it as a central controller for
configuration changes, as well as utilize it for central monitoring of the SD-WAN network. SD-
WAN Center also has additional monitoring ability of application, which are not available on the
local appliances GUI like being able to track the Mean Opinion Score (MOS) of targeted
applications.
https://docs.citrix.com/en-us/netscaler-sd-wan-center/9-3/configuration-editor.html

 Import active configuration into SD-WAN Center
 Enable an application for calculation of MOS score using SD-WAN Center

Step Action
a) Once SD-WAN Center is connected to the active MCN, the running configuration can be
imported. Even though the MCN still has the Configuration Editor, it is better to utilized
SD-WAN Center’s Network Configuration tool to perform the same operation.
Open the web interface to SD-WAN Center and import the running configuration.
Navigate to Configuration > Network Configuration, then click the Import button.
Make sure “Active MCN” is displayed in the From Network drop-down, then click
Import.
2
3
citrix.com
261
Step Action
b) Click the Show/hide Network Map button to give more room for the Network
Configuration, then click the Advanced tab.
2
1
c) Select the Advanced tab of the Configuration Editor. In the Global tile navigate to the
Rule Groups node. In prior releases Rule Groups were called Applications. This
change in terminology occurred in version 9.2.
This group name provides the ability to enable MOS calculations.
Find IPERF in the list, and click the edit icon, then enable Estimate MOS. Click Apply
to accept the changes.
The “Estimate MOS” setting will cause SD-WAN Center to calculate MOS passively for
existing traffic that passes through the Virtual Path. The MOS calculation is a quality
assessment of the targeted traffic. The statistical data is only visible from the SD-WAN
Center.
citrix.com
262
Step Action
d) Navigate to Global > Default Set > Virtual Path Default Set and click the add (+) icon
to add a new default set.
e) In the new default set, expand the Rules node and create a new IP rule to filter for
IPERF traffic.
Click the add (+) icon to add a new rule. Select IPERF for the Rule Group Name, then
select IPERF from the Protocols drop-down.
2 3
citrix.com
263
Step Action
f) Expand the new rule and click the Initialize Properties Using Protocol button. This
will populate all the advanced fields based on the select protocol.
We can also adjust the Transmit Mode here. Select Load Balance Paths, which is
typically the default for most applications. Also enable the Track Performance option,
this is required for the MOS calculation to be tracked.
1
2
Before clicking Apply, open the LAN to WAN tile for the rule and select the Class 14
(interactive_very_low_class).
Click Apply to accept the changes which is located at the very bottom of the rule.
citrix.com
264
Step Action
g) Next, we can look at another Application Rule filter available called Application QoS.
This feature is similar to the IP Rule feature we configured above but is more advanced
to be able to define filter rules based on Application name using the built-in QOSMOS
Deep Packet Inspection (DPI) engine.
Navigate to Default Set > Virtual Path Default Sets > New_Virtual_Path_Default_Set
> Application QoS and click the add (+) icon to add a new rule.
citrix.com
265
Step Action
h) In the Add Application QoS pop-up window, select Application from the Match Type
drop-down window, then search the database by typing in “office365”. Then select
“Microsoft Office 365” from the menu.
1
2
Here also the Transmit Mode as well as the LAN to WAN Class can be selected.
Let’s select Persistent Mode and Class 11 (interactive_high_class). Then click Add
to accept the new QoS rule.
Release 9.3 introduces Application QoS rules as an addition to the traditional

Rules, which were already available but were limited in defining filters using 7-
tuple (src/dst IP, src/dst port, Protocol, DSCP, VLAN). Whilst one can create rules using
just the IP settings, you can now take advantage of the integrated application DPI
engine to identify specific applications or groups of applications as defined by custom
application objects. The IP Rules are still processed first and the Application QoS rules
are processed second, but the same settings and filters still apply. One important note
is that the Application QoS rules can be used to override the default IP rules. For new
installations, the recommendation is to make use of Application QoS to define
customized policies and leave the default IP Rules intact to perform default out of the
box operation.
citrix.com
266
Step Action
i) With a global default set of rules defined, we need to make sure that the sites will used
the new defined rules. Navigate to Connections > NYC > Virtual Paths > NYC-LON >
Local Site > Basic Settings. Click the edit icon and from the drop-down select
New_Virtual_Path_Default_Set. Then click Apply.
Perform the same operation on the Virtual Path > NYC-SJC.
citrix.com
267
Step Action
j) If we navigate to Connections > LON > Virtual Paths > NYC-LON > Local Site >
Basic Settings, we can see that the default set is being called here also.
We do have the option to override the globally defined rules in the default set and define
ones local to a site. Both the iperf rule and the office 365 rule can be found in the
respective Rules and Application QoS nodes.
k) From the SD-WAN Center, click the Save As button to save the new configuration with
name ExerciseC.
2
3
Use the Export button and run through the Change Management to export the new
configuration to the SD-WAN environment.
citrix.com
268
Step Action
l) Navigate to Configuration > Change Management.
Click on the here link to navigate to the MCN.
m) This should forward you to the MCN web management interface to run through the
Change Management.
Navigate to the Configuration > Virtual WAN > Change Management page of the
MCN and run through the Change Management procedure.
citrix.com
269
Step Action
n) Complete the Change Management process, which you should already be familiar with
a) Click Begin
b) Make sure the Configuration file is in the inbox (ExerciseC.cfg)
c) Click Stage Appliances
d) Since the MCN does not have connectivity to the SJC SD-WAN, we should see
66% completion, select the “ignore incomplete” then click Next to continue.
1 2

f) Click Done when complete.
g) After the configuration has been pushed to SD-WAN environment, you can now run the
test traffic.
On XenCenter, log in to the NYC_Server VM and launch the Iperf app from the desktop.
Configure it in Server mode and click Run Iperf!.
1 3
citrix.com
270
Step Action
h) In XenCenter, navigate to the LON_Client VM and launch the Iperf tool found on the
desktop, but set to Client mode, with 172.16.10.12 as the server address. Configure
1000 for the Transmit time, and then click the Run Iperf button.
1 2
4 6
3
citrix.com
271
Step Action
i) Navigate to the NYC_SDWAN_SE1 user interface, Monitoring > Flows to observe the
Iperf application flow on port 5001.
One line captures the traffic flowing from the datacenter down to the branch site, and a
separate line captures the traffic flowing from the branch back up to the datacenter.
Scrolling the view to the right columns in the columns, we can see that the iperf session
is being delivered across the NYC_INET-> LON_INET path.
citrix.com
272
Step Action
j) Navigate back to the SD-WAN Center user interface and navigate to Reporting > MOS
Score tab. You can view the Average and Lowest MOS score for each application that
MOS was enabled for in the configuration. With the historical capability of SD-WAN
Center, you can now use this tool to obtain the default value of MOS per targeted
application, and then re-measure MOS after features like Packet Duplication are
enabled for that applications leveraging the default set and rules which was introduced
in previous exercise. This provides a useful numerical value to coincide with reported
user experience improvements when conducting proof of concept for the SD-WAN
solution.
Note, the default polling interface for SD-WAN Center is set to 5 mins, which we lowered
to 2 mins, so there data may take a few minutes to appear as diplayed in the screenshot
below. Give it a few minutes it does not. Also note that this is a limited lab
environment, and for production deployments that have active running traffic the
graphical data will be more robust and provide mode valuable data points.
citrix.com
273
Step Action
k) Clicking on any of the graphs links in the table will redirect to UI to the Monitoring page
of the SD-WAN Center UI and output the stored data regarding the applications.
These reports are useful for drilling down in to applications and data collection
capabilities of SD-WAN Center from a historical point of view.
Exercise Summary
In this exercise, you leverage SD-WAN Center to identify the Mean Opinion Score of a targeted
application.
citrix.com
274
Exercise D: Zero Touch Deployment
Overview
Citrix SD-WAN introduced Zero Touch Deployment (ZTD) Service in the maintenance release of
9.1.1. This release contains code changes specific to enabling zero touch deployment
capabilities.
SD-WAN appliances shipped from factory with a special agent that attempts to connect to the
Citrix Cloud Service upon boot.
https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/zero-touch-deployment-service.html

 Stepped through the process involving zero touch deployment

Step Action
1. We will first need to log in to the SD-WAN Center web management interface.
In the Student Desktop, open the Chrome browser and click on the SD-WAN Center
link in the bookmarks toolbar.
Log in using admin/password credentials.
citrix.com
275
Step Action
2. The infrastucture for SD-WAN to support support Zero Touch Deployment is available
starting in the 9.1.1 release. The two main components SD-WAN Center, and the SD-
WAN appliances ship from factory will have zero touch agents which will handle the
processes needed to accomplish zero-touch install of the remote appliance
configuraiton to join the SD-WAN environment. The zero touch agent software and the
SD-WAN release software are not coupled, the agent software has a shorter release
cycle and it is automatically delivered by the Citrix Cloud Service. The self-updating
agent interacts with the cloud service, downloads the latest/compatible agent and
installs the same.
Navigate to the Configuration > Zero Touch Deployment page and click the Login to
Citrix Workspace Cloud button.
3. After clicking the Login to Citrix Workspace Cloud button, you should encounter the
following error.
This is due to the inability for the SD-WAN Center management IP to communicate to
the Citrix Cloud service.
citrix.com
276
Step Action
4. Navigate to Administration > Global Settings > Management Interface page.
Enter DNS IP address. Primary DNS: 192.168.10.1, Secondary DNS 8.8.8.8.
2 3
5. Navigate back to Configuration > Zero Touch Deployment. Click on the Login to
Citrix Workspace Cloud again.
This type notice the browser blocking any pop-up windows.

Click the hyperperlink to allow the pop-up window.
citrix.com
277
Step Action
6. Log in with your Citrix account credentials.
If you do not have an account, click “Sign up and try it for free”.
With successful login, the window will update with the “Citrix Cloud Login
Successful” message, and generally the window will automatically close and the SD-
WAN Center browser refresh with available Zero Touch Deployment sites.
7. If the browser doesn’t automatically refresh, manually refresh the page to display the
table of available sites from the configuration, which can be deployed using the zero
touch deployment procedure.
8. Select the SJC site and click Provision and Deploy button.
1
2
citrix.com
278
Step Action
9. The two available sites, SJC and LON, both point to virtual appliances, and because of
the model selected being VPX, the system assumes these are virtual instance in either
AWS or Azure Cloud and the only provides the two cloud environments as options for
provision types.
10. Since zero touch is not an option for the VPX models we are using in this lab we have
to end this exercise at this point. Depending on the physical or virtual cloud instance,
the procedure is documented here if you are interested in learning more:
 Appliance Deployment with Zero Touch: https://docs.citrix.com/en-us/netscaler-

sd-wan/9-3/zero-touch-deployment-service/appliance.html
 AWS Deployment with Zero Touch: https://docs.citrix.com/en-us/netscaler-sd-

wan/9-3/zero-touch-deployment-service/aws.html
 Azure Deployment with Zero Touch: https://docs.citrix.com/en-us/netscaler-sd-

wan/9-3/zero-touch-deployment-service/azure.html
Exercise Summary
In this exercise, we stepped through the zero-touch deployment process made available to
speed up the deployment process of SD-WAN devices and cloud instances.
citrix.com
279
Module 7 (Advanced): Features
Introduced in SD-WAN 9.3.0
citrix.com
280
Module 7 Overview
This module will lead you through some of the more advanced features of the Citrix SD-WAN
solution. Specifically, some of the feature enhancements that were introduced with the release
9.3.
It is important to have a clear understanding of your deployed SD-WAN environment. What IPs
are being utilized for both management and data path communication, and how the appliances
are deployed. The following are prerequisites:
 SD-WAN deployed network topology

 Identify the deployment mode and obtain all IP address for both the management plane
and the data plane
citrix.com
281
Exercise E: Single-Step Upgrade bundle
Overview
Citrix SD-WAN 9.3 release introduced a consolidation of individual, model specific software
packages as well as the operating system upgrade bundle. The previously separated software
packages have now been bundled into a single *.zip file that can be pushed to all devices using
the change management process. The bundled file includes:
 <model>.<release>.<build>.tar.gz
 <release>.<buld>.upg
There is also a central console introduced where an admin can monitor the status of the
upgrades.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/9-3/single-step-
upgrade-for-standard-edition-appliances.html

 Be introduced to single bundle upgrade

Step Action
1. On the StudentDesktop, log in to the web interface of NYC_SDWAN_SE1
(https://192.168.10.23) VM.
citrix.com
282
Step Action
2. Navigate to Configuration > Virtual WAN > Change Management to start the MCN
config/software staging process.
Click Begin to advance to Change Preparation.
3. Change Preparation allows the upload of software in addition to verifying the config.
Once an SD-WAN environment is operating on release 9.3, an admin can ignore the
individual model .tar.gz and instead make use of a single bundle software .zip file,
which bundles all the individual software files into one.
Click Choose Files button to upload the single bundle software .zip file.
Older SD-WAN software releases required the upload of individual *.tar.gz files.
An individual file was made available for every available Standard and Enterprise
model of SD-WAN and depending on the configuration file in the change management
inbox the individual files had to be uploaded one by one.
citrix.com
283
Step Action
4. In the file upload window, browse to Desktop > SD-WAN Software > 9.3 > Platform
and select and Open the ns-sdw-sw-9.3.3.21.zip file.
Take note that the file is ~700MB in size. If we unzip the file, we can see that the
contents is more then just the individual model .tar.gz files (which are bundled into
cb-vw-all.9.3.3.21.tar.gz), but also include Service Management VM, XenServer and
WAN optimization software upgrade components, which are typically found in the .upg
file which previously was upgraded seperately.
5. After the file has been chosen, click the Upload button. This will compare against the
Configuration file and selectively update the Model(s) list with the required models
found in the configuration.
citrix.com
284
Step Action
6. With the software upload and the desired configuration file in the inbox, click Stage
Appliance to continue.
7. In the License pop-up window, check the box to I accept the End User Aggrement
and click Ok.
1
2
8. Depending on the Virtual Path connectivity from the MCN to all SD-WAN nodes, you
may have to select “Ignore Incomplete” in order to be able to click Next and advance
forward.
1 2
citrix.com
285
Step Action
9. Click the Activate Staged button to activate the uploaded software to the nodes in the
SD-WAN lab environment.
With release 9.3 there is a new feature to “Revert on Error”. This Configuration
Rollback feature allows the Change Management system to detect and recover
from certain software / configuration errors by reverting to the previously active
software/configuration. Not all failure modes will result in a rollback. This feature can
detect network outage and appliance crash after upgrade and in those scenarios
automatically roll back the SD-WAN devices.
10. When activation is complete, click the Done button.
citrix.com
286
Step Action
11. After upgrade, navigate to Configuration > Change Management Settings page of
the NYC_SDWAN_SE1 (MCN). After upgrade to release 9.3 a new page named
Change Management Setting is made available to track the complete upgrade of the
model specific .tar.gz file and also the component (SVM, XS, WANOP) specific
software upgrades for hypervisor-based platforms.
Click the pen icon to edit scheduling for upgrade. By default the MCN schedules
disruptive installations at 21:20:00 with the expectation to be disruptive during off-peak
hours. These settings can be customized to sync up with maintenance windows.
Updating the Maintenance Window (hours) to “0” will allow the update to take place
immediately if network disruption is not a concern.
It is very important to utilize the Change Management Settings feature when

dealing with Enterprise Edition SD-WAN devices. The system will perform audit
checks to make sure that the hypervisor and WANOP components have properly been
upgraded and are ready for testing.
Exercise Summary
In this exercise, we upgraded the SD-WAN environment using the single bundle upgrade file
and identified where to monitor the success of the upgrade for all the individual devices in the
network.
citrix.com
287
Exercise F: Application Preferred WAN link
Overview
In this exercise, we will continue with creating service policies to provide more customization of
application control through the Preferred WAN link feature introduced in release 9.3.1.

 Create a preferred WAN link for the iperf test traffic

Step Action
1. From previous exercise iperf traffic was delivered through the network and as
referenced in the flow table, the NYC_INET-> LON_INET path was the path primarily
used in its delivery across the SD-WAN overlay.
In this exercise we will configure a preferred path as the MPLS path to provide some
granular control as to what path the session should take.
On the web interface for SD-WAN Center, open the running configuration in the Network
Configuration Console.
Navigate to the Connections tile and select NYC > Virtual Paths > NYC-LON >
Application QoS.
Here you will find some existing QoS rules auto-populated by the default set.
citrix.com
288
Step Action
2. We can override the policies from the default set by creating site specific Application
QoS rules. When creating site specific Application QoS rules, Preferred WAN Links
becomes an option to create preference to one WAN link over another.
Click add (+) to add a new Application QoS policy and add the following filter criteria:
 Match Type = Application
 Application = iperf(iperf)
 Transmit Mode = Persistent Path
 Class = 14 (interactive_very_low_class)
Lastly, select NYC_MPLS as the Preferred WAN Link. Then click the Add button to
accept the settings.
1 2
3 4
Notice that the Preferred WAN Link option is only available if the Persistent Path
transmit mode is selected.
3. Save As the new configuration and name it ExerciseF, and Export to Change
Management.
citrix.com
289
Step Action
4. Complete the Change Management process, which you should already be familiar with
a) Click Begin
b) Make sure the Configuration file is in the inbox (ExerciseF.cfg)
c) Click Stage Appliances
d) Since the MCN does not have connectivity to the SJC SD-WAN, we should see
66% completion, select the “ignore incomplete” then click Next to continue.
1 2

f) Click Done when complete.
citrix.com
290
Step Action
5. After completing the Change Management process, test the new configuration by
running iperf traffic between two host nodes.
Using XenCenter console for the NYC_Server virtual machine, configure the jperf
application in Server mode. Then Run Iperf.
3
2
citrix.com
291
Step Action
6. Using XenCenter console for the LON_Client virtual machine, configure the jperf
application in Client mode, pointing to the NYC_Server (172.16.10.12) IP as the server
address, and increase the Transmit time to 1000 seconds. Then Run Iperf.
1
5
2 3
citrix.com
292
Step Action
7. With the iperf traffic flow, we can look to see in the SD-WAN reporting that the
Application QoS rule we defined in our Local Site is working as configured.
On the NYC_SDWAN_SE1 web interface, navigate to Monitoring > Flows page.
Find the port 5001 (iperf) flow in the table, and confirm that the session is successfully
being associated to the Application QoS policy that was created without the use of any
port or ip address and that the flow is being delivered using the defined Class Type as
Realtime and the Transmission Type as Persistent, but specifically using the
NYC_MPLS -> LON_MPLS path configured.
3 4
It is important to understand that the Citrix SD-WAN by design does its absolute
best to keep applications alive and to make full use of all available paths to do so.
Even though, we may prefer the traffic to flow across the MPLS link if that link becomes
unavailable SD-WAN is forced to use other more viable links.
8. As we monitor the Jperf graphs, we can see that iperf test traffic is limited to just the
MPLS link bandwidth, maxing out at 1.7Mbps. In previous test we have seen this same
application use the load balanced transmit mode and make use of the full virtual path
(MPLS+INET).
citrix.com
293
Step Action
9. To identify the behavior of SD-WAN on the iperf application set to Persistent Mode with
Preferred WAN links, we are going to purposely bring down the MPLS WAN link.
Open a new tab to the WANem web interface. Select eth2 from the drop-down and
click Start.
1
2
With eth1 selected, configure Loss(%) to 100 and click Apply Settings to bring down
the MPLS link.
citrix.com
294
Step Action
10. First, we can confirm the health of the MPLS WAN link by navigating to the
NYC_SDWAN_SE1 web interface Monitoring > Statistics page. The NYC_MPLS-
>LON_MPLS and the reverse path are both marked as Dead.
11. Navigate to the Monitoring > Flows page we can further confirm that the iperf, port
5001, session is still alive but is being delivered on the remaining INET path. It is not
necessarily the preferred path, however SD-WAN is designed to keep the application
alive regardless of the preferences of path.
Exercise Summary
In this exercise, we controlled application preference of path using the Application QoS
Preferred Path setting. We also illustrated SD-WANs ability to keep a session active and alive
in the event of extreme outage conditions of the preferred WAN path.
citrix.com
295
Module 8 (Advanced): Features
Introduced in SD-WAN 10.0
citrix.com
296
Module 8 Overview
This module will lead you through some of the more advanced features of the Citrix SD-WAN
solution. Specifically, some of the feature enhancements that were introduced with release
10.0.
It is important to have a clear understanding of your deployed SD-WAN environment. What IPs
are being utilized for both management and data path communication, and how the appliances
are deployed. The following are prerequisites:
 SD-WAN deployed network topology

 Identify the deployment mode and obtain all IP address for both the management plane
and the data plane
Pre-Requisite
You must have completed the previous modules, or only module 5 & 6 to load the pre-staged
configuration.
citrix.com 297
Exercise G: Upgrade environment to release 10.0
Overview
Citrix SD-WAN release 10.0 introduced some key features to address the need for scaled SD-
WAN deployments:
 Scaled deployment (2,500 site support) with multi-region deployment

 Simplified Configuration Editor
 Centralized Licensing
 REST API enhancements
 Application based service selection
 Virtual Path Cost
 Multicast IGMP support
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/10.html

 Upgrade SD-WAN to the 10.0 release
 Be exposed to the new simplified Configuration Editor
 Upgrade SD-WAN Center to the 10.0 release
Estimated time to complete this exercise: 15-20 Minutes

Step Action
1. On the StudentDesktop, open the web interface to the MCN (NYC_SDWAN_SE1) and
log in with admin/password credentials.
citrix.com
298
Step Action
2. Navigate to the Configuration > Virtual WAN > Change Management page and click
Begin to start the Change Management process.
Click the Choose Files button to upload the release 10.0 software bundle. On the
desktop find the SD-WAN Software > 10.0 > Platforms directory and upload the ns-
sdw-sw-10.0.0.207.zip file.
1 4
Click the Upload button after the file has been chosen. The upload process may take a
minute to complete.
citrix.com
299
Step Action
3. The system will take a minute to process the uploaded file. When the software bundle
upload completes, click Stage Appliances and complete the SD-WAN change
management process.
There are significant changes with SD-WAN release 10.0, thus the change
management process may take longer than usual.
citrix.com
300
4. Step through the remainder of the change management process.
a) Accept the End User License Aggreement, and click Ok.
b) The MCN may not have connectivity to the SJC SD-WAN (if we started the lab
with Module 5), we should wait until we see 66% completion progress. Select
“Ignore Incomplete” option to proceed, and click Next to continue.
1 2
Note, the State % indicator in the table outlines the software packging
progress for each connected site. Sites that do not have Virtual Path
connectivity will be displayed as Not Connected.
c) Click Activate Staged at the Activation step. After reaching 0s count-down,

refresh the browser.
citrix.com
301
Step Action
After the activation count-down with the Change Management complete, you may need
to refresh the browser and log back into the SD-WAN web interface. Successful
upgraded version will be indicated on the banner.
citrix.com
302
Step Action
5. Next, we will update the SD-WAN Center to release 10.0.
In the Chrome browser, open a tab to the web interface of the SD-WAN Center
(NYC_SDWAN_Center) and log in with admin/password credentials.
Navigate to the Administration > Global Settings > Software Upgrade.
Click the Browse button, navigate to the Desktop > SD-WAN Software > 10.0 > SD-
WAN Center directory and upload the ns-sdwc-10.0.0.207.tar.gz software file.
Click Upload after the file has been selected, then Install.
2 4
3
Click Install to acknowledge the upgrade request.
Accept the EULA, and click Install.
citrix.com
303
Step Action
6. During the upgrade, SD-WAN Center will go into Maintenance Mode. After the
installation completes, click the Continue button.
The web interface will be refreshed, and R10.0 will be indicated in the top banner.
7. Navigating to the Dashboard of the SD-WAN Center, the first change you will notice is
the new Network Map view.
SD-WAN Center web interface can provide a quick view of multi-region summary in the
dashboard, providing an easy to read overview of the network health across the
configured regions. For each region, the SD-WAN administrator can view the total
number of sites defined in each region, and from that summary outline which sites are
in good, fair, or poor condition. Allowing for an admin to quickly converge on poor sites
to address SD-WAN overlay issues.
citrix.com
304
Step Action
8. Navigating down the page, we can Visualize the Virtual Path relationship between site
to identify why sites are reported in Poor condition.
Navigate down and select the NYC as the Origin Site, then select LON as the
Connected Site before clicking the Visualize button.
Hovering the mouse cursor over the LON-NYC Virtual Path, one can see the details of
the individual paths.
In this case, if we select SJC as the connected site and visualize that virtual path, we
will see why the NYC site is listed as a Poor site since at least one Virtual Path is
DOWN.
1 2
This high-level path assessment of the SD-WAN environment will become more
and more necessary when the deployment scales up to thousands of sites and
network health assessment is necessary. Being able to quickly gauge the health of the
SD-WAN environment and drill down to specific Virtual Path and individual WAN path
state will be very valuable for large deployments.
citrix.com
305
Step Action
9. Navigating to the Network Configuration page of the SD-WAN Center and click the
Import button to import the Active MCN configuration. Or open the last saved
configuration.
Allow Overwrite if needed and click Import.
citrix.com
306
Step Action
10. With the latest configuration file open, another notable change with release 10.0 is with
the new Configuration Editor tool and the removable of the hierarchical tree to build
configuration.
The configurable features have been collapsed and categorized into tabular form.
+
As you familiarize yourself with the new configuration editor, you will notice drop down
options to select the Region, and to select the Site. You will also notice that selected
features will open a pane to the right already in edit mode to quickly make changes and
apply. There are also drop-down options on the right pane for select features that have
sub-features. This will be the natural workflow of the new configuration editor to first
select the targeted reation, then the specific site and its features for edit.
1 4 5
2
Exercise Summary
In this exercise, we upgraded SD-WAN devices and SD-WAN Center to release 10.0 and briefly
looked at the noticeable changes in the web interface and configuration.
citrix.com
307
Exercise H: Application Route
Overview
With Citrix SD-WAN there are various ways to route targeted traffic through the SD-WAN
Overlay. In release 10.0 the Application Route feature introduced the ability to selectively route
traffic across the various Service Types (Virtual Path, Internet, Intranet, Local, GRE Tunnel,
LAN IPsec Tunnel) by defining global Application Objects and creating Application Routes at
desired sites.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-

wan/10/routing/application-routing.html

 Configure Application Route
citrix.com
308
Step Action
1. Before we can start configuring an Application Route, we need to first identify the target
application we want to selectively route and then create the required Application Object.
Generally, the usage of this feature is enabled when all the traffic from the branch
locations are backhauled for internet access through the DMZ in the Data Center and
the Application Route feature can be configured for selective trusted applications that
the admin feels is safe to route through other available Service Types which will yield
better user experience.
In our lab (assuming exercise13 configuration is loaded), the current configuration for
the LON site is to have local internet breakout. As we open a browser window in the
LON_Client VM, the default page is Citrix.com. This traffic is being NAT’d out on the
SD-WAN interface which is connected to the INET WAN link.
In the same browser, change the URL from www.citrix.com to www.salesforce.com.
citrix.com
309
Step Action
2. On the local LON_SDWAN_SE web interface, navigating to Monitoring > Flows, and
find our flow to salesforce. Scrolling the table to the right, we can see “salesforce”
defined as the application and the Service Type defined as INTERNET.
Since this is the behavior of the active running environment, let us change delivery of
Salesforce by defining a new Application Route.
citrix.com
310
Step Action
3. On the StudentDesktop, open the web interface to the SD-WAN Center
(192.168.10.25) and log in with admin/password credentials.
In the Network Configuration page, open active configuration or the pre-staged

configured from the desktop.
citrix.com
311
Step Action
4. In order to define an Application Route, we must first define an Application Object.
On the Global tab, select Applications, then in the Section drop-down select
Application Objects.
Click the add (+) icon to add a new Application Object.
4
2
citrix.com
312
Step Action
5. Enter “SaaS” in the Name field and check Enable Reporting to enable SD-WAN
Center to collect the statistics of this particular Application Object in the reporting.
Click the add (+) icon to add an Application Match Criteria.
Select Application as the Match Type and enter and select Salesforce (salesforce)
as the Application.
1 2
3
4 5
Before saving this, let us add another application to this Application Object. Click the
add (+) icon and select Application as the Match Type and Slack(slack) as the
Application.
6
7 8
citrix.com
313
Step Action
6. With the Application Object defined, we can now create our Application Route for our
LON_Client to send Salesforce and Slack SaaS applications another route.
Navigate to Connections tab, select LON for the View Site, and then select
Application Routes. On the right, click the add (+) icon to define a new Application
Route policy.
2 4
In the Add pop-up window, select SaaS from the Application Objects drop-down. Since
the current behavior is local internet breakout, we can backhaul just these applications.
Select Virtual Path from the Service Type drop-down and select NYC as the Next Hop
Site. Click the Add button to accept the new setting.
1 2
With this new configuration, selectively the SaaS defined Application Objects
(Salesforce and Slack) will be delivered through the Virtual Path to the NYC Data
Center SD-WAN.
citrix.com
314
Step Action
7. Save As the new configuration as ExerciseH and Export to change mangement.
1 4
2
3
8. Navigate to the NYC_SDWAN_SE1 web interface and complete the full Change
Management process:
 Appliance Staging (enable “Ignore Incomplete”, since SJC SD-WAN is not
currrently up)
9. With the new configuration in place, close the browser and navigate again to
www.salesforce.com.
citrix.com
315
Step Action
10. On the LON_SD-WAN (https://192.168.10.27) web interface. Navigate to the
Monitoring > Firewall and Connections from the Statistics drop-down.
Select Salesforce(salesforce) from the Application dropdown for filtering.
2
3
We can see here that Salesforceis still classified as Internet as the Service Type. This
is expected since it takes up to 16 packets to full define an application. Once the
application is defined in the local database of public IP addresses, then subsequent
connection attempts will start using the new policy.
citrix.com
316
Step Action
11. Navigate back to LON_Client and close the browser. Open a new browser, and
navigate to www.salesforce.com again.
This time the page will fail to load. This is expected since now the new Application
Route is in place to backhaul the traffic to the NYC_SDWAN and the DC currently does
not have any Internet Service in place to properly handle the traffic.
citrix.com
317
Step Action
12. Open the web browser to the SD-WAN Center web interface and navigate to the
Reporting tab. Click and drag to highlight portion of the time line in which you have run
the SaaS Application Route test.
If enough time has passed for the SD-WAN Center to collect the statistics (default 5
mins), then Salesforce will be seen in the the Applications table along with analytics of
outoging and incoming traffic throughput.
Exercise Summary
In this exercise, we configured Application Objects and Application Routes to selectively route SaaS
applications.
citrix.com
318
Exercise I: Scaled deployments with Multi-Region Mode
Overview
Citrix SD-WAN release 10.0 introduced a new multi-region architecture enabling scaled
deployments up to 2,500 nodes, where previously the limitation was 550 node
deployments.
In this new architecture, the MCN will have an added non-forwarding mode to
communicate with newly introduced Regional Control Nodes (RCNs). In this mode, the
MCN will have Virtual Paths only to the RCNs and will not be able to forward traffic
between RCNs and their directly connected client nodes.
The multi-region architecture still allows for clients to be directly connected to the MCN
as a default region, but for the most part scale is achieved by regionalizing the client
nodes. The 550-node limitation is handed down to the RCN nodes, which now require
the direct Virtual Path connectivity to each client node within the region.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/10/use-cases-sd-

wan-virtual-routing/multi_region_deployment.html

 Be introduced to Multi-Region Mode and Regional Control Nodes
citrix.com
319
Step Action
1. Regional Control Node or RCN is a new concept introduced in release 10.0 and the
feature addresses the need for large scale deployments.
In summary, MCN responsibility of direct communication with each client node has
been offloaded to RCNs. This frees up the MCN from having to manage connectivity to
each client node in the SD-WAN environment and unblocks the limit of the MCN
enabling scale beyond the limits of a single device. A few concepts to make note of:
 RCNs are essentially the same as current MCNs with regards to functionality
 RCNs can be deployed as high availability pair and also have a geographically
distributed “Secondary RCN”
 RCNs require a Virtual Path to each client node in that region
 The network is capable of having multiple RCNs but there is still one MCN
 RCNs require a Virtual Path to the MCN but the client nodes under the RCN
regions do not require a Virtual Path to the MCN
 The MCN functionality is still the same and can have client directly connected to
it as a default region
Taking those concepts into account, we can take our current lab as example, which is
limited to 550 nodes (if our NYC SD-WAN was a 5100-SE appliance as opposed to the
current VPX) and introduce the concept of regions and RCNs allowing us to scale
beyond the limit.
Current Lab Scaled
With the example Scaled deployment above, this would enable 550 nodes per RCN
(assuming we are using 5100-SE) and 550 nodes for the MCN (again assuming we are
using 5100-SE) totaling an SD-WAN deployment consisting of potentially 1,100 nodes.
Always reference the Citrix SD-WAN Data Sheet for the specs per device:
https://www.citrix.com/products/netscaler-sd-wan/netscaler-data-sheet.html
citrix.com
320
Step Action
2. In our lab we have implemented virtual appliance, VPX which supports up to 16 Virtual
Paths. Selecting VPXL would allow for 128 Virtual Paths but requires additional
resources on the host machine.
Using VPX model, the above adjustments in our architecture to promote the LON SD-
WAN to an RCN would yield 16 client nodes per RCN and 16 client/RCN nodes for the
default MCN region giving a total of potentially 32 nodes.
As you can see the selection of SD-WAN models assigned as the RCN and MCN can
yield smaller or larger deployments and the selection of mode is flexible depending on
the needs of the network and the regions.
citrix.com
321
Step Action
3. We will convert the LON client SD-WAN node to a RCN node, in order to build out a
new region and allow for more SD-WAN nodes.
On the StudentDesktop, open the web interface to the SD-WAN Center
(https://192.168.10.25) and log in with admin/password credentials.
In the Network Configuration page, import the active configuration or the pre-staged
configured from the desktop.
4. Before we can start configuring a multi-region deployment, we must first specify the
region names.
On the Global tab, select Regions, then add (+) to create new regions to add to the
existing Default_Region.
2 3
citrix.com
322
Step Action
5. In the Add pop-up window, input the region Name as EMEA. There are additional
options to help better organize the configuration and enable audit checking of subnets
used within the region.
 Force Internal VIP Matching: When enabled, all non-private Virtual IP

Addresses in the Region will be forced to match the configured subnets.
 Allow External VIP Matching: When enabled, non-private Virtual IP Addresses
from other regions will be allowed to match the configured subnets.
Enable Force Internal VIP Matching and click the add (+) icon to add a subnet, which
we expect to encompass all subnets that live within that region: 172.70.0.0/16.
Click the Add button when complete.
6. Navigate to the Sites tab, from the View Sites drop-down select LON.
With Basic Settings selected, select the new EMEA region from the drop-down. Update
the Mode from client to Primary RCN, then click Apply to accept the changes.
2 3
citrix.com
323
7. With the LON site SD-WAN promoted as RCN, we need to address a few changes from
the previous configuration.
1. Navigate the Connections tab and select SJC site from the Default Region.
Select Dynamic Virtual Paths and uncheck the “Enable Dynamic Virtual
Paths” option. Click Apply to accept the change.
2 4
3
5
2. Select NYC as the site, then select Virtual Paths, NYC-LON from the Virtual
Path to Site drop-down, and WAN links for the Section. Enable Use for
NYC_MPLS WAN link and scroll to the right-most column and select <Default>
Autopath Group (Use the scroll bar at the bottom of the page to scroll to the
right). Then Apply the setting.
3 4
1
2
5 7
8
3. Select Paths from the Section drop-down. To view the empty table.
citrix.com
324
Step Action
4. Select EMEA region, then LON site. Select WAN Links from the Sections drop-
down, then check LON_MPLS for Use and <Default> for the Autopath Group
before clicking Apply.
1 3
2
4 6
7
5
5. Select Paths from the Section drop-down to view the empty table.
8. With the latest changes, Save As the new configuration with name ExerciseI.
2
3
Click Export but Cancel the Change Management.
With R10.0 the Export button triggers the audit check of the configuration.
citrix.com
325
Step Action
9. With the Export canceled we can monitor for any audit warnings.
Nothing concerning, we are okay to proceed.

Also, selecting Paths for the LON site, we can see the auto-population of the paths for
the configured autopath group.
Similarly, we can confirm this for the NYC site. Select the Default_Region, NYC, and
NYC-LON for the virtual path to site.
1 3
2
citrix.com
326
10. With the configuration accepting the promotion of LON to an RCN, we can now add a
new client node to report under the LON RCN in the EMEA region. For each branch
site that is added, the appropriate Region must be selected, which determines which
RCN/MCN it is expected to communicate with.
We will add one branch node to report to the LON RCN we just promoted.
a) In the Sites tab, select EMEA as the region, then click + Site.
2
3
b) Name the site BEL, select CBVPX as the Model, make sure EMEA is selected
as the Region and Client as the Mode. Click Add to accept the new site.
2
3
c) Navigate to the Sites tab, select Basic Setting for the BEL site. Enable
Source MAC Learning, then click Apply.
3
4
d) With the new BEL site still select. Navigate to Interface Groups and click add
(+). Define Ethernet 1 & 2, Fail-to-Block and add E1E2Vlan0 as the Virtual
Interface Name. Click Apply to accept the changes.
citrix.com
327
2
1 3
e) Navigate to Virtual IP Addresses and click add (+). Input 172.70.2.2/24 as the
IP Address and select E1E2Vlan0 as the Virtual Interface. Click Apply to
accept the changes.
2
3
1 4
f) Select WAN Links and click + Add Link.
g) Input BEL_MPLS for the Name and Private Intranet for the Access Type, then
click Add.
citrix.com
328
h) Define 1500 Kbps for the Physical Rates. Click Apply to accept the changes.
1 2
3
i) For the created WAN link, the select Access Interfaces from the Section drop-
down and click the add (+) button. Select E1E2Vlan0 from the Virtual Interface
drop-down, input 172.70.2.2 for the IP Address and 172.70.2.1 as the Gateway.
j) Navigate to the Connection tab. Select EMEA region and BEL site. Select
Virtual Paths. From the Sections drop-down select WAN Links and check the
Use column for the BEL_MPLS WAN Link. Scroll to the right-most column and
select <Default> as the Autopath Group. Click Apply to accept the changes.
citrix.com
329
1
3
2 4 6
7
5
k) In the Connections tab, select now the LON site, Virtual Paths and WAN Links
should already be displayed. Enable the Use column for the LON_MPLS WAN
Link. Uncheck the use of LON_INET. Click Apply to accept the changes.
l) Change the Virtual Path to Site to LON-BEL and Section to WAN Links.
Disable the Use of LON_INET and enable the Use of LON_MPLS with
<Default> as the Autopath Group.
1 2
3
4
5
citrix.com
330
m) Navigate to WAN Links, select LON_INET and Virtual Paths from the
Sections drop-down. Enable usage for the NYC-LON and disable for BEL-
LON virtual path. Then click Apply.
3
1
4
n) Navigate to the Sites tab, select BEL site, Basic Settings, then Enable
Source MAC Learning and click Apply to accept the change.
3
4
o) Save the new configuration and click Export. There should be no more
concerning audit warning so it is safe to continue export.
citrix.com
331
Step Action
1 2
Enable Overwrite Change Management running configuration and click

Export.
citrix.com
332
Step Action
11. Open the web browser to the NYC_SDWAN_SE1 VM and step through the change
management process.
a) Navigate to Configuration > Virtual WAN > Change Management, click
Begin
b) In Change Preparation, make sure the Configuration file is in the inbox
(ExerciseI.cfg), click Stage Appliances
c) The table will indicate the progress of package creation for the defined sites.
LON may take some time to create due to the update from Client node to RCN.
Allow time for that process to fully complete.
d) You should notice a new table outlining the Global Multi-Region Summary
e) Since the MCN does not have connectivity to the newly created sites, we
should see 66% completion,
Select Ignore Incomplete and Click Next to continue.
1 2
f) Click Activate Staged
Approve the activation by clicking OK in the pop-up window.
citrix.com
333
Step Action
12. With the new configuration successfully pushed out. Navigate to the Configuration >
Change Management page again on the NYC_SDWAN_SE1 VM. One of the new
features here is the consolidate of sites based on Region selection. Click on the EMEA
region, and notice the table below populate with sites only from that region.
citrix.com
334
Step Action
13. Next, select the Default_Region and the below table will update to reflect nodes only
associated with that region, including connected RCNs of other regions.
14. On the table for the Default_Region Details click the active link for the LON site to
download the new software/configuration package.
citrix.com
335
15. Since the LON site was given a new RCN identify. It would be good to complete clear
its previous identify as a Client node.
1. On the LON SD-WAN web interface, navigate to Configuration > Virtual WAN
> Enable/Disable/Purge Flows. Uncheck “Perform a diagnostic dump”,
then click Disable.
4 3
2
2. Navigate to System Maintenance > Configuration Reset, click Configuration

Reset.
3. Log back into the LON SD-WAN web interface

4. Navigate to Configuration > System Maintenance > Local Change
Management. Click Choose File and Upload the recently download LON
active package found in the download directory.
citrix.com
336
Step Action
1
2
4
5. Complete the Local Change Management process and Activate Staged.
6. After logging back into the new LON SD-WAN, navigate to Configuration >
Virtual WAN > Enable/Disable/Purge Flows page and click the Enable button.
1
2
citrix.com
337
Step Action
16. Open the web interface to the LON_SDWAN_SE VM. Navigate to the Configuration >
Virtual WAN > Change Management page.
Some items to highlight here is that this newly configured LON SD-WAN RCN now has
Change Management and Change Management Settings options similar to the MCN.
The Change Management lists and is responsible for package creation and distribution
only for devices in this region. Note that the NYC and SJC site are not listed here.
Navigate to the Change Management Setting page. Again, this page only reflects the
software update progress of nodes directly reporting to this RCN.
citrix.com
338
Exercise Summary
In this exercise, we reconfigured an existing client node and promoted to a Regional Control Node (RCN)
and added a new site as a client node to that promoted RCN.
citrix.com
339
Exercise J: Enable Multi-Region on SD-WAN Center
Overview
Citrix SD-WAN Center R10.0 has been updated to support the new region-based architecture.
Multi-region SD-WAN Center architecture requires the addition of a Collector per region. The
Collectors are separate instance of SD-WAN Center configured to collect data from the RCN
and Client node of a particular region.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan-center/10/multi-

region-network-deployment.html

 Introduce a new SD-WAN Center as a Collector for the EMEA region
 Walkthrough the procedure of enabling multi-region on SD-WAN Center
citrix.com
340
Step Action
1. Like the deployment of SD-WAN devices, the deployment of multi-region SD-WAN
Center also requires the network to be laid out beforehand.
From the previous exercise, we created an EMEA region and denoted the LON site as
the RCN with one direct reporting client node (BEL). The Default Region has
connectivity to the RCN of the EMEA region but also contains the MCN and one directly
reporting client node (SJC). For this architecture, we need two SD-WAN Centers. One
we currently have, and it is associated with the MCN. We need to introduce a new SD-
WAN Center and associate it with the EMEA region’s RCN (LON).
Each SD-WAN Center (Headend and Collector) require to have IP connectivity to the
SD-WAN devices on the management plane and management interfaces. Connectivity
to the Client nodes can be accomplished through the Virtual Paths if required.
citrix.com
341
Step Action
2. Based on our new architecture, we need to install a second SD-WAN Center as the
Collector to connect to the LON RCN.
In the XenCenter application, locate the SDWAN_Center_9.3.1 template. Right-click
and select Quick Create.
3. Right-click the newly imported VM and select Properties.
citrix.com
342
Step Action
4. Change the name to LON_SDWAN_Center and click OK.
5. For the LON_SDWAN_Center VM, right-click and Force Shutdown the VM.
Select Yes to accept the shutdown request.
citrix.com
343
Step Action
6. Navigate to the Networking tab for the shutdown VM and click the Properties button.
Update the Virtual Interface Properties to select Internal as the Network. Then click
OK.
1
3
7. Right-click and Start the LON_SDWAN_Center.
8. Navigate to the Console tab and once the log in with admin/password credentials.
citrix.com
344
Step Action
9. Issue the command management_ip and press Enter. Then issue the following
command to configurate a management IP address:
set interface 192.168.10.70 255.255.255.0 192.168.10.1
Press the Enter key, then y to accept the changes.
citrix.com
345
Step Action
10. In the web browser, open a new tab to the LON_SDWAN_Center
(https://192.168.10.70) web interface, log in with the default credentials
admin/password after proceeding past the browser certification error.
11. Once logged in to the LON_SDWAN_Center, navigate to Administration > Global

Settings > TimeZone tab and select the Time Zone as Europe > London from the
drop-down and click Apply.
2 3
citrix.com
346
Step Action
12. Navigate to the Administration > Global Settings > Software Upgrade page and
browse and upload to install release 10.0 SD-WAN Center software (ns-sdwc-
10.0.0.207.tar.gz) found in the C:\Users\localuser\Desktop\SD-WAN Software\10.0\SD-
WAN Center directory.
4
5
13. In the web browser, navigate over to the original NYC_SDWAN_Center

(https://192.168.10.25) and navigate to the Configuration > Network Discovery >
Discovery Settings page, and check Enable Multi-region mode. Then click
Discover regions.
citrix.com
347
Step Action
14. The Collector Configuration table will populate with the region detail from the active
configuration. The Collector IP Address will be empty and needs to be configured.
Click the edit icon.
15. Edit the Collector IP Address by entering the IP address of the newly created
LON_SDWAN_Center VM (192.168.10.70). Then click the Save icon, to save the IP
address and push the certificates.
16. In the pop-up window, enter the credentials for the LON_SDWAN_Center VM,
admin/password. Then click the Push Certificate button.
citrix.com
348
Step Action
17. Select All the listed devices and click Discover.
18. Navigate to Inventory and Status tab, select Poll to select all nodes, then click Apply.
citrix.com
349
Step Action
19. For multi-region deployments, you can select specific regions to view events and
statistic reports on the default region SD-WAN Center. The events and statistics
reports data are fetched from the respective region’s collector.
20. Open the web interface to the LON_SDWAN_Center. From the header you will notice
that this SD-WAN Center is denoted as a “Collector” and the data in the various
reporting pages is limited to the nodes connected to the RCN for this region.
citrix.com
350
Step Action
21. Comparing the two SD-WAN Centers, one being the headend monitoring the Default
region, and the other being the collector monitoring the EMEA region. Comparing the
web interfaces of each we will see the differences in capabilities.
Exercise Summary
In this exercise, we configured the SD-WAN Center Collector for the EMEA region and connected it to the
multi-region enabled SD-WAN Center Headend.
citrix.com
351
Exercise K: Centralized Licensing with SD-WAN Center
Overview
Citrix SD-WAN has different methods to license devices. Local licensing enables the upload of
license files directly to each device using the local web interface. Remote licensing enabled the
SD-WAN devices to pull down their license file from a remote Licensing Server using the
management network, enabling centralization of all license files. In R10.0, SD-WAN introduced
a new method of centralized licensing using SD-WAN Center to replace the need for remote
license server, coupled with the ability to tie the license file into the configuration this new
feature gives more control as the licensed ability of each device.
Reference: (USA hyperlink) https://docs.citrix.com/en-us/netscaler-sd-wan/10/sd-wan-

licensing/centralized-licensing.html

 Walk through the new Centralized Licensing feature of SD-WAN Center
citrix.com
352
Step Action
1. Navigate to Configuration > Licensing page available on the NYC_SDWAN_Center
(https://192.168.10.25).
The SD-WAN Center headed will report on active/connected devices in the network. If
we have upgraded the devices from an existing environment, the existing local or
remote licensing configured will be ported over and reporting accordingly on this page.
citrix.com
353
Step Action
2. In the Network Configuration and open the latest configuration. Navigate to the Global
tab and select Centralized Licensing.
Enter the management IP address of the NYC_SDWAN_Center (192.168.10.25) and
click Apply.
The IP address of an external license server can also be used. If SD-WAN Center is
planned to be used as the License Server, port 27000 must be used.
3. To change the licensing mechanism of any new or existing site you can accomplish this
directly in the Configuration Editor.
Navigate to the Sites tab, and select EMEA region, and select the LON site. Then
select Centralized Licensing.
Since the previous configuration had “Local” licensing configured before the upgrade to
R10.0, this Configuration Editor preserves that functionality.
citrix.com
354
Step Action
4. From the License Server Location drop-down, select Central. Then Select License
Rate of 10. The License Server IP and port are automatically populated from the
Global configuration. Click Apply to save these settings.
5. Save As this new configuration as ExerciseK.
Export and send the file to Change Management.
citrix.com
355
6. Navigate to the NYC_SDWAN_SE1 web interface and complete the full Change
Management process:
 Click the Default_Region to see the progress of package creation
To refresh the status, click Default_Region periodically. It may take several

minutes for the package to be transferred to the region from the MCN.
Navigate to the LON_SDWAN_SE web interface, and view the Configuration > Virtual
WAN > Change Management page.
 There is no connected to the BEL site, enable “Ignore Incomplete” then click
Next
 Then Activate Staged
citrix.com
356
Step Action
7. The LON site was updated in the configuration for Central Licensing for a 10Mbps
license, but without an avaiable license the device goes into a no license mode.
With newer model devices, the SD-WAN in this scenario will go into a Grace License
mode, allowing for continued operation of SD-WAN until the license is sync through the
configured Central License Server.
citrix.com
357
Step Action
8. Open the web interface to the NYC_SDWAN_SE1 Change Management page. This
has now reached 66% with the LON SD-WAN manually advanced. Enable “Ignore
Incomplete” for the disconnected SJC site and click Next.
Click Activate Staged to proceed.
citrix.com
358
Step Action
9. With Change Management complete, open the web interface to NYC_SDWAN_Center,
and refresh the page for Configuration > Licensing > Network Summary.
We can see now that the LON site is configured for the remote license server. Without
the license file not being present on the SD-WAN, the site will conitnue to report
unlicensed.
10. Navigate to the File Management page to identify the License Server Host ID to use for
licensing SD-WAN devices.
citrix.com
359
Step Action
11. Open a new tab to the Citrix Eval store. Copy and past the following URL.
http://store.citrix.com/store/citrix/en_US/pd/productID.278335900/ThemeID.33753000
Select VPX-10-VW and 30 Days from the respective drop-down menus. Then click Add to Cart.
12. Log into the store with your Citrix credentials, submit the Eval license request and
submit the order.
Upon receiving the license details via email. Open a web browser window to the
License Management System URL: License Management System
Use the license code to allocate the requested license file.
Download the license file.
citrix.com
360
Step Action
13. Open the NYC_SDWAN_Center web interface.
Navigate to the Configuration > Licensing > File Management page.
Browse and upload the license file.
14. With a successfully uploaded license file, navigate to the License Detail tab.
Here we can monitor usage of available centralized license files.
citrix.com
361
Step Action
15. The LON SD-WAN will reach out to the SD-WAN Center’s central licensing and install
the 10Mbps license.
16. Further confirmation can be made on the LON_SDWAN_SE management interface that
the device is licensed and Virtual Path established.
Exercise Summary
In this exercise, we made use of the SD-WAN Center as the Centralized Licensing tool for the SD-WAN
environment.
citrix.com
362
Lab Guide Appendices
citrix.com
363
Appendix A: Additional Resources and Information
Lab Infrastructure Diagram Underlay (w/out SD-WAN): https://tinyurl.com/y7fe2s3z

Lab Infrastructure Diagram Overlay (w/ SD-WAN): https://tinyurl.com/y7zqutct
Deployment Mode: https://docs.citrix.com/en-us/netscaler-sd-wan/10/use-cases-sd-wan-virtual-
routing.html
Troubleshooting Guides: https://support.citrix.com/article/CTX226234
SD-WAN Routing Features: https://docs.citrix.com/en-us/netscaler-sd-wan/10/routing.html
SD-WAN Overlay Routing: https://docs.citrix.com/en-us/netscaler-sd-wan/10/routing/overlay-
routing.html
citrix.com
364
Corporate Headquarters Hong Kong, China
Fort Lauderdale, FL, USA India Development Center
Bangalore, India Latin America Headquarters
Silicon Valley Headquarters Coral Gables, FL, USA
Santa Clara, CA, USA Online Division Headquarters
Santa Barbara, CA, USA UK Development Center
EMEA Headquarters Chalfont, United Kingdom
Schaffhausen, Switzerland Pacific Headquarters
citrix.com
365
About Citrix
Citrix (NASDAQ:CTXS) is a leader in mobile workspaces, providing virtualization, mobility management, networking and cloud services
to enable new ways to work better. Citrix solutions power business mobility through secure, personal workspaces that provide people
with instant access to apps, desktops, data and communications on any device, over any network and cloud. This year Citrix is
celebrating 25 years of innovation, making IT simpler and people more productive. With annual revenue in 2013 of $2.9 billion, Citrix
solutions are in use at more than 330,000 organizations and by over 100 million users globally. Learn more at www.citrix.com.
Copyright © 2014 Citrix Systems, Inc. All rights reserved. [list Citrix trademarks (without ® or ™ symbols!) in document] are trademarks of
Citrix Systems, Inc. and/or one of its subsidiaries, and may be registered in the U.S. and other countries. Other product and company
names mentioned herein may be trademarks of their respective companies.

Citrix SD-WAN-R10.0 Lab Guide-Full-v4

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Citrix SD-WAN-R10.0 Lab Guide-Full-v4

Uploaded by

Copyright:

Available Formats

Citrix SD-WAN

Worldwide Product Readiness

Prepared by: Shoaib Yusuf

Revision Change Description Updated By Date

Lab Guide Conventions

Special note to offer advice or background information

Focuses attention on a part of the screen (R:255 G:102 B:0)

 SD-WAN Standard Edition (available as physical or virtual appliances)

If you would like to start the lab using a pre-built

1. Familiarize yourself with underlay network details

In this exercise, you will:

Estimated time to complete this exercise: 20 Minutes

Step by Step Guidance

10. Right click Command Prompt and select Run as Administrator.

In this exercise, you will:

Estimated time to complete this exercise: 15 Minutes

Step by Step Guidance

Deployment options include (recommended from top down):

 Dynamic Routing (OSPF, BGP)

 Policy Based Routing (PBR) with IP SLA

 The SD-WAN will be deployed in HA, so another set of IP addresses will be

 Each SD-WAN appliance will require a unique IP address on the management

 IP address for web browser access

Estimated time to complete this exercise: 15 Minutes

Step by Step Guidance

 set interface 192.168.10.23 255.255.255.0 192.168.10.1

The Release Notes capture supported browsers. It is best practice to use

Click OK to accept the change.

Verify the Date/Time after the updated change.

Your lab virtual machines may already be licensed locally.

Validate that the VPX is licenses before proceeding.

There should only be one device promoted to MCN in an SD-WAN environment.

Estimated time to complete this exercise: 10 Minutes

Step by Step Guidance

 Advanced – Advanced mode enables configuraiton of advanced features of SD-

Default AES-128 encryption is used in marketing the performance numbers on

 Site Name: NYC

In the Add Site Settings pop-up window, enter the following:

 WAN Link: NYC_MPLS

Click the Add button to accept the settings.

 Networking IP Address/Prefix: 172.16.10.0/24

 Networking IP Address/Prefix: 172.16.11.0/24

By defining these routes in the SD-WAN configuration, we are enabling remote

Estimated time to complete this exercise: 15 Minutes

Step by Step Guidance

3. In the Add Site pop-up window, populate the following:

 Site Name: LON

In the add interface pop-up window, enter the following detail:

 Ethernet Interfaces: 1 & 2

 WAN Link: LON_MPLS

Click Add to accept the changes.

Click Apply to save the changes.

 Site Name: SJC

In the add interface pop-up window, enter the following detail:

 WAN Link: SJC_MPLS

In the add interface pop-up window, enter the following detail:

1. Create the desired network topology

In this exercise, you will:

Estimated time to complete this exercise: 20 Minutes

Step by Step Guidance

 Step 2 “Appliance Staging” allows for a systematic push of the configuration to

 ns-sdw-sw-9.3.3.21.zip – “ns-sdw-sw” denotes this software package is

 ns-sdw-sw-9.3.3.21.zip – “9.3.3.21” denotes the software release/build number.

Click Stage Appliances to proceed.