DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING,

SECOND CYCLE, 30 CREDITS

STOCKHOLM, SWEDEN 2021

Performance Analysis
of an SD-WAN Infrastructure
Implemented Using Cisco
System Technologies

GIANLORENZO MOSER

KTH ROYAL INSTITUTE OF TECHNOLOGY

SCHOOL OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE
Performance Analysis
of an SD-WAN Infrastructure
Implemented Using Cisco
System Technologies

GIANLORENZO MOSER

Master’s Programme, ICT Innovation, 120 credits

Date: August 26, 2021

Supervisor: Sladana Josilo

Examiner: Dán György
School of Electrical Engineering and Computer Science
Host company: VEM Sistemi
© 2021 Gianlorenzo Moser
 Abstract | i

Abstract
Software-Defined Wide Area Networking (SD-WAN) is an emerging
technology that has the potential to satisfy the increasing demand for reliable
and efficient Wide Area Networks (WANs) in the enterprise-network market.
This thesis focuses on the main features of an SD-WAN network and on the
technical challenges facing the design and implementation of an SD-WAN
infrastructure. It also provides a detailed comparison between the SD-WAN
and the other WANs solutions such as MultiProtocol Label Switching (MPLS).
The thesis is based on the project that is about the migration of network
infrastructure that uses the MPLS technology to a network infrastructure that
uses the SD-WAN technology. The migration process includes many phases
such as the analysis of the existing MPLS based infrastructure, identification
of suitable appliances based on customer requests, and the design of the
SD-WAN infrastructure that can be implemented without disrupting the
network functioning during the transition stage. The thesis provides a detailed
description of these steps and it discusses the trade-offs that were made during
the design phase of the project.
The results presented in the thesis are obtained through on-site tests performed
for the new SD-WAN infrastructure. The tests were performed with the
objective to evaluate some of the main SD-WAN functionalities such as load
balancing, traffic shaping, and high availability. The obtained results show
the effective functioning of the network infrastructure and illustrate some of
the main advantages that the new SD-WAN infrastructure has over the old
MPLS infrastructure. Finally, this thesis could be of interest to network
professionals and employees who consider SD-WAN as a possible solution
for their company’s business.

Keywords
SD-WAN, SDN, MPLS, Load Balancing, Traffic Shaping
ii | Abstract
 Sammanfattning | iii

Sammanfattning
Software-Defined Wide Area Networking (SD-WAN) är en framväxande
teknik som har potential att tillgodose den ökande efterfrågan på tillförlitliga
och effektiva Wide Area Networks (WAN) på företagsnätverksmarknaden.
Denna avhandling fokuserar på huvudfunktionerna i ett SD-WAN-nätverk och
på de tekniska utmaningarna för design och implementering av en SD-WAN-
infrastruktur. Det ger också en detaljerad jämförelse mellan SD-WAN och
andra WAN-lösningar som MultiProtocol Label Switching (MPLS).
Avhandlingen
bygger på projektet som handlar om migrering av nätverksinfrastruktur
som använder MPLS-tekniken till en nätverksinfrastruktur som använder
SD-WAN-tekniken. Migreringsprocessen omfattar många faser, till exempel
analys av befintlig MPLS-baserad infrastruktur, identifiering av lämpliga
apparater baserat på kundförfrågningar och utformningen av SD-WAN-
infrastrukturen som kan implementeras utan att nätverket fungerar under
övergångssteget. Avhandlingen ger en detaljerad beskrivning av dessa steg och
diskuterar de avvägningar som gjordes under projektets designfas.
Resultaten som presenteras i avhandlingen erhålls genom test på plats för den
nya SD-WAN-infrastrukturen. Testerna utfördes i syfte att utvärdera några av
de viktigaste SD-WAN-funktionerna som lastbalansering, trafikformning och
hög tillgänglighet. De erhållna resultaten visar att nätinfrastrukturen fungerar
effektivt och illustrerar några av de största fördelarna som den nya SD-WAN-
infrastrukturen har jämfört med den gamla MPLS-infrastrukturen. Slutligen
kan denna avhandling vara av intresse för nätverkspersonal och anställda som
anser SD-WAN som en möjlig lösning för företagets verksamhet.

Nyckelord
SD-WAN, SDN, MPLS, Load Balancing, Traffic Shaping
iv | Sammanfattning
 Sommario | v

Sommario
Software-Defined Wide Area Networking (SD-WAN) è una tecnologia
emergente che ha il potenziale per soddisfare la crescente domanda di reti
geografiche (WAN) affidabili ed efficienti nel mercato delle reti aziendali.
Questa tesi si concentra sulle caratteristiche principali di una rete SD-WAN e
sulle sfide tecniche che devono affrontare la progettazione e l’implementazione
di un’infrastruttura SD-WAN. Fornisce inoltre un confronto dettagliato tra
la SD-WAN e le altre soluzioni WAN come MultiProtocol Label Switching
(MPLS).
La tesi si basa sul progetto che riguarda la migrazione di un’infrastruttura di
rete che utilizza la tecnologia MPLS ad un’infrastruttura di rete che utilizza
la tecnologia SD-WAN. Il processo di migrazione comprende molte fasi
come l’analisi dell’infrastruttura esistente basata su MPLS, l’identificazione
di dispositivi idonei in base alle richieste dei clienti e la progettazione
dell’infrastruttura SD-WAN che può essere implementata senza interrompere
il funzionamento della rete durante la fase di transizione. La tesi fornisce una
descrizione dettagliata di questi passaggi e discute i compromessi che sono
stati fatti durante la fase di progettazione del progetto.
I risultati presentati nella tesi sono ottenuti attraverso test eseguiti in loco per
la nuova infrastruttura SD-WAN. I test sono stati eseguiti con l’obiettivo di
valutare alcune delle principali funzionalità SD-WAN come load balancing,
traffic shaping, e high availability. I risultati ottenuti mostrano l’effettivo
funzionamento dell’infrastruttura di rete e illustrano alcuni dei principali
vantaggi che la nuova infrastruttura SD-WAN presenta rispetto alla vecchia
infrastruttura MPLS. Infine, questa tesi potrebbe interessare professionisti e
dipendenti di rete che considerano SD-WAN come una possibile soluzione
per il business della propria azienda.

Parole Chiave
SD-WAN, SDN, MPLS, Load Balancing, Traffic Shaping
vi | Sommario
 Acknowledgments | vii

Acknowledgments
I would first like to thank my thesis supervisor Sladana Josilo of the School of
Electrical Engineering and Computer Science at KTH. Without her assistance
and dedicated involvement in every step throughout the process, this thesis
would have never been accomplished. I would like to thank you very much
for your support and understanding over these past four months. I would also
like to show gratitude to my examiner György Dán of the School of Electrical
Engineering and Computer Science at KTH.
I would like to express my sincere gratitude to VEM Sistemi. Despite the
constraints of the COVID-19 pandemic, they welcomed me into their reality
and allowed me to perform my degree project. In particular, I thank Nicola
Gatto and all the colleagues of the Padua office.
I would like to offer my special thanks to EIT Digital Master School for the
opportunity to be part of the students that have participated in the double
degree master program. Moreover, a special thanks to all the fantastic people
from every part of the world that I met during these two years of my master
degree that made my experience unique.
Finally, I must express my very profound gratitude to my family for providing
me with unfailing support and continuous encouragement throughout my years
of study and through the process of researching and writing this thesis. This
accomplishment would not have been possible without them. Thank you.
Stockholm, August 2021
Gianlorenzo Moser
viii | Acknowledgments
 CONTENTS | ix

Contents

1 Introduction 1
1.1 Motivation and Challenges . . . . . . . . . . . . . . . . . . . 2
1.2 Research Methodology . . . . . . . . . . . . . . . . . . . . . 3
1.3 Delimitation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Sustainability and ethical concerns . . . . . . . . . . . . . . . 4

2 Background 7
2.1 Types of internet connection
technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Multiprotocol Label Switching . . . . . . . . . . . . . 8
2.1.2 Internet . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Software Defined Networking . . . . . . . . . . . . . . . . . 11
2.3 Software-Defined Access . . . . . . . . . . . . . . . . . . . . 13
2.3.1 Mobility . . . . . . . . . . . . . . . . . . . . . . . . 13
2.3.2 Segmentation . . . . . . . . . . . . . . . . . . . . . . 14
2.3.3 Management . . . . . . . . . . . . . . . . . . . . . . 14
2.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4 Software-Defined Wide Area Networking . . . . . . . . . . . 15
2.4.1 Network Appliances . . . . . . . . . . . . . . . . . . 16
2.4.2 Topologies . . . . . . . . . . . . . . . . . . . . . . . 17
2.4.3 The Essentials of SD-WAN Architecture . . . . . . . . 19
2.4.4 Economic advantages . . . . . . . . . . . . . . . . . . 23
2.4.5 Vendors comparison . . . . . . . . . . . . . . . . . . 24
2.5 Key Features of SD-WAN . . . . . . . . . . . . . . . . . . . . 26
2.5.1 Security . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.2 Segmentation . . . . . . . . . . . . . . . . . . . . . . 26
2.5.3 Cloud integration . . . . . . . . . . . . . . . . . . . . 28
x | Contents

2.5.4 Policy Framework . . . . . . . . . . . . . . . . . . . 29

2.5.5 Load balancing . . . . . . . . . . . . . . . . . . . . . 30
2.5.6 Traffic shaping . . . . . . . . . . . . . . . . . . . . . 31
2.5.7 Voice optimization . . . . . . . . . . . . . . . . . . . 31
2.5.8 High Availability . . . . . . . . . . . . . . . . . . . . 33

3 Methodology 39
3.1 Research Process . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2 Experimental design . . . . . . . . . . . . . . . . . . . . . . 41
3.2.1 Reliability and validity of the network
infrastructure . . . . . . . . . . . . . . . . . . . . . . 41

4 From MPLS based network to SD-WAN infrastructure 43

4.1 Overall Network Structure . . . . . . . . . . . . . . . . . . . 43
4.1.1 Headquarter Network Infrastructure . . . . . . . . . . 44
4.1.2 Branch Network Infrastructure . . . . . . . . . . . . . 46
4.2 Customer requirements and motivations . . . . . . . . . . . . 47
4.2.1 Motivation for changing the current network
infrastructure . . . . . . . . . . . . . . . . . . . . . . 47
4.2.2 Customer requirements . . . . . . . . . . . . . . . . . 47
4.3 Design of a new SD-WAN infrastructure . . . . . . . . . . . . 49
4.3.1 Network appliances . . . . . . . . . . . . . . . . . . . 50
4.3.2 SD-WAN Network Infrastructure . . . . . . . . . . . . 53
4.4 Implementation of key SD-WAN aspects . . . . . . . . . . . . 57

5 Results and Analysis 63

5.1 High Availability tests . . . . . . . . . . . . . . . . . . . . . . 63
5.1.1 Link failure . . . . . . . . . . . . . . . . . . . . . . . 63
5.1.2 Appliance failure . . . . . . . . . . . . . . . . . . . . 66
5.2 Load balancing tests . . . . . . . . . . . . . . . . . . . . . . 67
5.3 Traffic shaping test . . . . . . . . . . . . . . . . . . . . . . . 69

6 Discussion 73

7 Conclusions and Future work 75

7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
7.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.3 Reflections . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

References 79
 LIST OF FIGURES | xi

List of Figures

2.1 MPLS forwarding process . . . . . . . . . . . . . . . . . . . 8

2.2 Header MPLS . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 SDN architecture . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4 Basic SD-WAN Architecture . . . . . . . . . . . . . . . . . . 17
2.5 SD-WAN Topologies . . . . . . . . . . . . . . . . . . . . . . 18
2.6 SD-WAN communications between edges and controller . . . 20
2.7 Auto-VPN and Automatic NAT traversal with UDP hole
punching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.8 Micro-segmentation . . . . . . . . . . . . . . . . . . . . . . . 28
2.9 Direct access to internet for access to cloud services . . . . . . 29
2.10 Voice Optimization Strategies . . . . . . . . . . . . . . . . . 32
2.11 Virtual Router Redundancy Protocol . . . . . . . . . . . . . . 34

3.1 Representation of the procedural phases . . . . . . . . . . . . 40

4.1 AS-IS Network Infrastructure . . . . . . . . . . . . . . . . . . 45

4.2 Organization of the headquarters’ subnets . . . . . . . . . . . 46
4.3 Intermediate network infrastructure . . . . . . . . . . . . . . 54
4.4 Intermediate network infrastructure with VPN . . . . . . . . . 55
4.5 Final network infrastructure . . . . . . . . . . . . . . . . . . . 56
4.6 Meraki recommended design - switch stack . . . . . . . . . . 58
4.7 SD-WAN topology settings . . . . . . . . . . . . . . . . . . . 58
4.8 Load balancing settings . . . . . . . . . . . . . . . . . . . . . 60
4.9 Traffic shaping settings . . . . . . . . . . . . . . . . . . . . . 61

5.1 Link failure times . . . . . . . . . . . . . . . . . . . . . . . . 64

5.2 Internet traffic analysis in the link failure scenario . . . . . . . 65
5.3 Appliance failure times . . . . . . . . . . . . . . . . . . . . . 66
5.4 Uplink bandwidth consumption analysis with load balancing . 67
xii | LIST OF FIGURES

5.5 Internet traffic analysis with SD-WAN policies - bandwidth

usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.6 Internet traffic analysis with SD-WAN policies - latency values 69
5.7 Internet traffic segmentation at the application level . . . . . . 70
5.8 Internet traffic analysis with traffic shaping . . . . . . . . . . . 71
 LIST OF TABLES | xiii

List of Tables

2.1 Vendors comparison . . . . . . . . . . . . . . . . . . . . . . 25

4.1 Customer requirements . . . . . . . . . . . . . . . . . . . . . 48

4.2 Comparison between client requirements and proposed solution 50
xiv | LIST OF TABLES
 List of acronyms and abbreviations | xv

List of acronyms and abbreviations

5G Fifth-Generation

ACL Access Control List

AH Authentication Header
AMP Cisco Advanced Malware Protection
ATM Asynchronous Transfer Mode

BFD Bidirectional Forwarding Detection

BGP Border Gateway Protocol

CAGR Compound Annual Growth Rate

Cisco DNA Cisco Digital Network Architecture
CO2 Carbon Dioxide

DC Data Center
DHCP Dynamic Host Configuration Protocol
DMZ Demilitarized zone
DoS Denial of Service
DSCP Differentiated Services Code Point
DTLS Datagram Transport Layer Security

ECMP Equal-cost multi-path routing

ESP Encapsulating Security Payload

FEC Forward Correction Code

GbE Gigabit Ethernet

GF Greedy failover

HA High Availability
xvi | List of acronyms and abbreviations

HMAC Hash Message Authentication Code

HQ Headquarter
HSRP Hot Standby Router Protocol

IaaS Infrastructure as a Service

IDS Intrusion Detection System
IETF Internet Engineering Task Force
IKE Internet Key Exchange
IoT Internet of Things
IP Internet Protocol address
IPS Intrusion Prevention System
IPsec IP Security
ISE Identity Services Engine
ISP Internet Service Provider

LACP Link Aggregation Control Protocol

LAN Local Area Network
LISP Locator/Identifier Separation Protocol
LLPD Link Layer Discovery Protocol
LTE Long Term Evolution

MEF Metro Ethernet Forum

MPLS Multiprotocol Label Switching

NAT Network address translation

OMP Overlay Management Protocol

OSPF Open Shortest Path First

PaaS Platform as a Service

 List of acronyms and abbreviations | xvii

PPF Pre-partitioning failover

QoE Quality of Experience

QoS Quality of Service

RADIUS Remote Authentication Dial-In User Service

SA Security Association
SaaS Software as a Service
SD-Access Software-Defined Access
SD-WAN Software-Defined Wide Area Networking
SDN Software-Defined Networking
SFP small form-factor pluggable transceiver
SIG Secure Internet Gateway
SSL Secure Sockets Layer
STP Spanning Tree Protocol

TCP Transmission Control Protocol

TLS Transport Layer Security
TTL Time To Live

UI User Interface
USB Universal Serial Bus

VIP Virtual IP
VLAN Virtual Local Area Network
VN Virtual Network
VoIP Voice over IP
VPN Virtual Private Network
VRRP Virtual Router Redundancy Protocol
xviii | List of acronyms and abbreviations

VXLAN Virtual Extensible LAN

WAN Wide-Area Network

ZTP Zero-Touch Provisioning

 Introduction | 1

Chapter 1

Introduction

The demand for a variety of emerging cloud services is growing across every
line of businesses. Cloud services usually have high bandwidth and low
latency requirements that cannot be met in a traditional Wide-Area Network
(WAN) designed to provide the best-effort service [1]. A promising approach
to support these emerging cloud services is Software-Defined Wide Area
Networking (SD-WAN). Owing to its potential to improve the user experience
and provide fast, scalable, affordable, and flexible connectivity between
different network environments, SD-WAN-based solutions are continuously
being deployed. This aspect has been confirmed by a recent estimate by Cisco,
according to which the SD-WAN traffic is expected to grow at a Compound
Annual Growth Rate (CAGR) of 37% compared to 3% for traditional WAN in
the next few years [2].
SD-WAN based solutions have the potential to extend the capabilities of an
organization by leveraging the corporate WAN and multi-cloud connectivity.
One of the key benefits of SD-WAN is that it provides dynamic path selection
between connectivity options such as Multiprotocol Label Switching (MPLS),
Long Term Evolution (LTE) / Fifth-Generation (5G) and internet network
[3]. Moreover, SD-WAN with the traffic shaping features allows for the
inbound and outbound traffic segmentation. In addition, SD-WAN allows
for traffic prioritization based on both user/group policies and the types of
their applications. Consequently, SD-WAN solutions have the potential to
allow organizations to access business-critical cloud applications quickly and
easily, and thus to provide high-speed application performance along the
WAN perimeter of branch offices [1]. Finally, SD-WAN can increase the
2 | Introduction

uplink redundancy to improve the Quality of Experience (QoE) and Quality

of Service (QoS) perceived by users.
The objective of this thesis is to analyze and deepen various topics related to
Software-Defined Networking (SDN). The main focus is on how to plan and
design a SD-WAN, starting from pre-existing network infrastructure that is
based on the MPLS technology. In this context, the technical and economic
choices will be discussed with the objective to identify the vendor and the most
suitable product for the customer’s needs. Furthermore, the thesis will focus
on aspects related to reducing downtime during the migration, and propose
possible ad-hoc solutions to the downtime related issues. Finally, based on
the actual behaviour of the infrastructure, characteristic aspects of SD-WAN
such as High Availability (HA), load balancing, and traffic shaping will be
discussed.

1.1 Motivation and Challenges

The use of traditional networks for communication between offices located in
geographically different places is very often inefficient in terms of network
resource utilization. Moreover, this type of network organization requires
high-security standards to be applied to secure communications. To mitigate
these problems, the Internet Engineering Task Force (IETF) developed in the
late 90s a network management protocol, known as MPLS, intended to operate
between layers 2 and 3 of the ISO-OSI model [4]. Despite good performance,
easy resource reservation and prioritization, MPLS suffers from high costs and
low flexibility since it makes users strictly dependent on the providers.
SD-WAN addresses the cost and flexibility issues by allowing the parallel use
of multiple uplinks based on several technologies such as MPLS, LTE, and
5G. Furthermore, SD-WAN allows routing traffic on a specific uplink, based
on the link’s performance or based on the infrastructure manager’s discretion.
Despite this, there are still many aspects related to the HA, the efficiency
of load balancing, and traffic shaping that can be analyzed in a physical
implementation with the objective of verifying the advantages identified in
theory.
Some of the main challenges in the SD-WAN field that are of interest in this
thesis are:
• How to carry out a correct design and planning of an SD-WAN solution
starting from a solution already present on-site, and how to identify the
 Introduction | 3

most suitable product for the customer’s needs?

• How to carry out the migration between the two network infrastructures,
so that the interruptions during the transition phase are minimized?
• How to verify the correct functioning of the new infrastructure?
Inspired by the above challenges, this thesis has the goal to provide a
comprehensive analysis of an SD-WAN architecture. The thesis is based on the
implementation of a real SD-WAN architecture, which involved the following
three phases:
1. Planning and design of a new SD-WAN infrastructure with the objective
of replacing the existing MPLS based infrastructure. This phase must be
done according to the clients’ requirement. Moreover, coexistence and
normal operation must be ensured during the transition phase between
the two infrastructures.
2. Implementing policies, routes, and Virtual Local Area Network
(VLAN), verifying that the behavior of the new infrastructure is the
same as in the replaced infrastructure and that the load balancing and
the traffic shaping are enabled and capable to intervene when explicitly
requested.
3. Testing HA through the simulations of infrastructure failover and
observing whether the scheduled failovers produce the activation of
backup links or devices to continue delivering the services.

1.2 Research Methodology

The main research methodologies used in this thesis are:
1. An in-depth study and analysis of technical specifications, solutions
offered by various vendors, and related work with the objective of
understanding the characteristics of diverse products and challenges
facing the design of an SD-WAN network.
2. An in-depth analysis of the client needs and existing infrastructure
with the objective of understanding how the system works and which
architecture types are feasible within the current organization.
3. Design and implementation of a new architecture that uses SD-WAN,
and is capable of meeting all technical requirements while ensuring best
4 | Introduction

practice in terms of HA.

4. Tests and simulations to verify the real functioning of the infrastructure
(e.g., load balancing, traffic shaping, and failover tests). Moreover,
comparative analysis between the previous and the current infrastructure
in terms of the fault detection performance.

1.3 Delimitation
This thesis project will have as the main objective the realization of an SD-
WAN infrastructure, which will be able to satisfy the customer’s requirements
of HA and provide the availability of services during the transition to the
new infrastructure. Since the implementation is physically carried out with
components available on the market, it will not be possible to carry out an
in-depth analysis of algorithms and protocols as they are often proprietary
and therefore not subject to disclosure to third parties. Finally, since the
implementation is existent, sensitive information related to places, people,
specifications, and public Internet Protocol address (IP) will be omitted.

1.4 Thesis Structure

The structure of the thesis is as follows. Chapter 2 presents relevant
background information about internet access technologies and main aspects
related to SD-WAN, such as traffic shaping, load balancing and security
aspects. Chapter 3 presents the methodology used to solve the problem.
Chapter 4 discusses the steps and issues related to the actual design
and implementation of the new network infrastructure based on SD-WAN.
Chapter 5, presents the results that validate the correct functioning of the
new infrastructure and confirm the advantages provided by the SD-WAN
technology. Chapter 6 and Chapter 7 conclude the work and discuss potential
direction for future work.

1.5 Sustainability and ethical concerns

The technologies considered in this thesis may be related to various aspects
concerning social issues, sustainability and ethics. One of the main
social issues is related to the requirement that the SD-WAN based network
infrastructure needs to be highly resilient to failures in the case of services
 Introduction | 5

provided for the safety-critical domains such as healthcare, autonomous

driving, and automated industry. Furthermore, it is necessary to migrate the
infrastructure from MPLS to SD-WAN technology while ensuring the proper
and secure functioning of the network during the transition phase.
From the point of view of sustainability, the technologies discussed in this
thesis have the potential to reduce Carbon Dioxide (CO2) emission though
reducing electricity consumption, without reducing the number and the quality
of the services. In particular, the sustainability of SD-WAN technology can be
seen in its capability to allow sharing of cloud spaces and network connections,
reducing server rental costs, electricity consumption and greenhouse gas
emissions. These aspects are indeed important and discussed in the recent
research that shows that the servers are often used only at 5% -15% of their
potential, while being powered at 100% [5] [6]. Furthermore, SD-WAN allows
for most of the maintenance and monitoring services to be done remotely, and
thus it has the potential to reduce CO2 emissions by reducing the need for
using transport commuters.
Finally, one of the main aspects related to the ethics in this thesis is the use
of policies to allow/prevent access to certain services and/or databases by
unauthorized users. In this context, the thesis will discuss the increasing need
for firewalls and the other security mechanisms that can provide protection
from external attacks, and thus prevent data and private information thefts.
Finally, the thesis will discuss how the SD-WAN technology can simplify the
implementation of some authentication and access procedures.
6 | Introduction
 Background | 7

Chapter 2

Background

This chapter introduces different types of internet connection technologies.

Subsequently, it presents the SD-WAN technology that derives from SDN.
Finally, it discusses several aspects and features that are fundamental to an
efficient network infrastructure design.

2.1 Types of internet connection

technologies
There are different internet connection technologies such as MPLS, cable
internet or wireless, i.e. LTE or 5G, that can be used by the companies
to interconnect them to other offices, datacenters or to resources stored in
the cloud. These technologies differ in terms of monetary cost, operation,
and QoS, and none of them is without technical or economic disadvantages.
By leveraging different types of the Internet connection technologies, SD-
WAN has the potential to increase the performance of the entire network
infrastructure, and by doing so to improve the company experience in terms of
monetary cost, security, availability and QoS. In the following, the first focus
will be on the description and analysis of the main technologies that can be
used for providing the connectivity in different scenarios.
8 | Background

2.1.1 Multiprotocol Label Switching

MPLS overview
MPLS is a routing technique designed to carry and route traffic over corporate
and Internet Service Provider (ISP) networks. MPLS was developed in the
late 1990s as an effective alternative to multi-layer and IP switching over
Asynchronous Transfer Mode (ATM). One of its main advantages is that it
can increase the flow rate of network traffic, since the routers can forward
the received packets by looking into label tables that are usually significantly
smaller than the traditional routing tables. In particular, a router can identify
the entire path that the packet will have to travel, and thus the next hops
only need to forward the received packets to the right port. MPLS, therefore,
forwards packets at level 2 of the ISO-OSI stack and, for this reason, it is
informally indicated that it operates at level 2.5 of the ISO-OSI stack [7] [4].

Figure 2.1 – MPLS forwarding process

The Essentials of MPLS Architecture

The routers that need to establish an MPLS connection are divided into label
switch router and label edge routers. The first type of routers can be found
within the network and they have the task to enable forwarding to the next hop
through modifying the MPLS header; this operation occurs thanks to the use
of the labels, generally very small compared to traditional routing tables. The
label has a one-to-one correspondence between input and output tag, and the
traffic is routed to the correct link based on the tag. An example that illustrates
the flow of an MPLS packet is shown in Figure 2.1.
 Background | 9

The label edge routers instead have the task of calculating the entire path of
the packet and placing/removing an MPLS header that identifies the traffic as
MPLS traffic. To do this, the packet is tagged, as shown in Figure 2.2 with a
4-byte (32-bit) MPLS header consisting of 4 fields:
• label value (20 bits): the effective tag divides different streams of
packets,
• traffic class (Exp) (3 bits): used to prioritize a particular flow (QoS),
• bottom of stack (S) (1 bit): when set it signifies that the current label is
the last in the stack,
• Time To Live (TTL) (8 bits).

Figure 2.2 – Header MPLS

Owing to the rapidity of the switch routers to apply the rules stored in the labels
[8], MPLS provides benefits in terms of performance, scalability, bandwidth
utilization, network congestion, and improved QoS and QoE. Moreover,
thanks to resource reservation, the providers that support the MPLS very often
provide bandwidth, latency and QoS guarantees. The trade-off is the cost of
an MPLS connection, which is significantly higher than that of an internet
connection with the same bandwidth requirement. Consequently, MPLS is
often chosen to route the traffic more sensitive to latency and jitter (e.g., Voice
over IP (VoIP) traffic).
It is important to mention that the MPLS traffic is only accessible to other
MPLS customers on the same infrastructure, and thus the network is not
exposed to Denial of Service (DoS) attacks because only a limited set of
users can access it [8]. For this reason, MPLS itself does not implement
encryption, but the confidentiality can be created by establishing encryption in
a higher layer, for example, by creating a Virtual Private Network (VPN) over
10 | Background

MPLS. Despite the fact, the MPLS network is considered secure compared
to the Internet as there are fewer possible attacks that can create disservice
or theft of user information. The principal risk is related to a physical
intrusion by an attacker. Therefore, it is of primary importance to secure
the network edges (i.e., the network entry points) and to protect them with
security measures, such as firewalls and/or Intrusion Detection System (IDS)
/ Intrusion Prevention System (IPS) [9].
The main disadvantages of MPLS are related to the high costs. The cost of an
MPLS link mainly depends on the geographical location of the sites, on the
provider that provides the service and on the bargaining between provider and
customer. The cost is usually ten times higher than in the case of a traditional
internet connection [10]. This difference in the cost is likely to remain in the
future, since with the diffusion of optical fiber, the cost per megabit has become
lower and the optical fiber has become more available [10]. Finally, in case
of offices located in different countries, there will be the need to agree on
the MPLS network with several providers, making the procedure long and
expensive [8].

2.1.2 Internet
Wired internet connection
Different from MPLS, the internet network is based on a best-effort service,
and thus it does not provide any guarantees on bandwidth, latency, or other
performance metrics. Consequently, it is easier to encounter congestion or
network degradation, which can compromise the QoS. Some of the solutions
(e.g., services offered by the ISP) provide bandwidth guarantees through the
traffic prioritization. However, these guarantees usually come at the expense
of an additional cost.
The main advantage of the internet connection is its price, which is lower than
that of the MPLS mainly because of the high competition among internet
service providers. Another advantage is that a classic internet connection
can be activated/deactivated much faster than an MPLS connection. In
particular, the activation of a new internet connection does not require
particular interactions between provider and customer and can be made at any
time. This efficient activation of a new internet connection comes, however,
at the price of vulnerability in terms of different types of cyber attacks. For
this reason, a company using a classic internet connection needs to have
firewalls, IDS / IPS and, when possible use encrypted VPN tunnels to secure
 Background | 11

the transmission of information from one site to another.

Wireless internet connection

Another type of internet connection is the mobile internet connection, which
can be used to provide data transmission between offices or internet access
to services. Indeed, with the evolution of mobile technology (i.e., with LTE
and 5G technologies), it is possible to establish an internet connection with
bandwidth, and latency values that are increasingly comparable to those of a
wired internet connection [11] [12]. For this reason, companies are becoming
more and more interested in using mobile internet connections, mainly as a
backup solution. Other advantages of the wireless internet connection are
good internet coverage in many areas, monthly fees that are comparable to
those of a wired internet connection, and simplified cable-free installation.
Besides these advantages, the mobile internet connection is also characterized
by certain disadvantages, for example, compared to MPLS mainly in terms of
reliability, security, and bandwidth guarantees.

2.2 Software Defined Networking

The SDN technology is a new approach to network architectures. The SDN
technology facilitates the administration and configuration of the network and
provides the separation of the control plane and the data plane [13]. Figure
2.3 shows that the controller device is positioned in a different layer (the upper
layer) and can control all the lower layer’s devices. The control plane may
consists of one or more controllers. Controllers implement different types of
policies, and they are responsible for the routing and forwarding decisions of
packets for all the network elements. The data plane, instead, is responsible
only for forwarding the received packets. This action is performed according
to the rules indicated by the packet owner. The most common protocol used
for the communication between the control plane and the data plane is a flow-
oriented protocol OpenFlow.
Most of the devices, such as access points, switches, and routers, have only the
data plane functionalities. The lack of a control plane makes the devices more
efficient in making decisions as they only have to consult their flow table. Each
switch has its flow table, and if it finds a match between the packet and one
of the rules, it performs the indicated action (e.g., forwarding the packet on
a defined port or dropping it). The correspondence between packet and flow
table can be detected based on numerous fields such as source, destination IP,
12 | Background

Figure 2.3 – SDN architecture

port, MAC address and VLAN. In case that there is no correspondence (i.e.,
in case of a new flow), the device must contact the controller. The controller
then identifies the best route for the flow, and subsequently, it informs all of
the involved devices about the new rule that needs to be inserted in the flow
table [14].
The main advantages of SDN compared to a traditional network are the
centralization and simplification of the control of an entire network. For
example, SDN allows simple access to the controller to register particular
policies/rules that can be further sent to all the switches. Therefore, SDN has
the potential to reduce errors and configuration times of the various devices in
the network. The centralization control is especially beneficial in the case of
devices distributed across geographically distant locations [15], which will be
discussed in more detail in Chapter 2.4. Apart from the centralized control, the
SDN has the potential to improve data plane performance and perform traffic
programmability. Finally, it allows the implementation of cloud abstraction,
and thus it can simplify the process of unifying the cloud resources. Indeed,
the SDN controllers can manage all the network’s components that comprise
the massive data center platforms [16].
As discussed above, centralization brings the intelligence of the entire network
to a single place, which besides the mentioned advantages may also be a source
of certain disadvantages, for example, in the context of security. In particular,
in case of a failure or cyber-attack, the entire network including the control
 Background | 13

and data planes, access API and OpenFlow may be affected.

There are numerous types of attacks such as unauthorized access and spoofing,
DoS attacks, intrusions, and for each of them resolutions to mitigate their
effects have been made. For example, in the case of unauthorized access, it
is recommended to use checkpoints and authentication systems (i.e. Remote
Authentication Dial-In User Service (RADIUS)). In the case of DoS attacks,
the use of firewalls and algorithms capable of identifying anomalous flows can
be a good solution. Furthermore, the use of the Secure Sockets Layer (SSL) /
Transport Layer Security (TLS) protocol for OpenFlow [14] communications
is often indicated to reduce tampering actions. The SSL / TLS protocol
mainly involves two phases: the handshake phase and the session phase.
The first phase requires the exchange of cryptographic material, the exchange
of certificates between the controller and the switch (sometimes optional),
and the decision of the cipher suite. Once the handshake is completed,
communication is established in an encrypted manner, using a symmetric key
based on the information shared during the handshake [17] [18].

2.3 Software-Defined Access

A technology derived from SDN is Software-Defined Access (SD-Access). It
is usually recommended in the case of scenarios with mobile devices and in the
case of scenarios where segmentation of the services is user-based and requires
a simplification of the administration [19]. An example is a scenario where
most of the devices connected to the network infrastructure are mobile devices
such as laptops and smartphones. The important aspects that need to be taken
into consideration in the case of SD-Access are: mobility, segmentation, and
management of the network.

2.3.1 Mobility
A user who changes the access point potentially changes the network and
will be assigned a new IP. The process of assigning a new IP is complicated
because if a policy-IP connection is made (a policy applied only to certain
IP addresses), it has to be reassigned to the new IP. One way of solving
this problem is to use Locator/Identifier Separation Protocol (LISP). The
LISP is used with SD-Access technology and it is a protocol that informs the
control node on which switch the user’s device is connected to. Using this
information, the controller can dynamically provide routes and services based
14 | Background

on the policies assigned to it.

The mobility together with the Virtual Extensible LAN (VXLAN) makes it
possible to maintain layer 2 reachability even in geographically distant areas.
VXLAN is a network virtualization technology that permits the encapsulation
of layer 2 frames in layer 4 datagrams with the objective of providing layer two
reachability over an underlying layer three network. VXLAN allows to have
a segmentation between different types of users connected to the same switch
[20].

2.3.2 Segmentation
Segmentation is one of the main aspects of SD-Access because the user can
be discriminated based on macro rules (macro-segmentation) such as role
(learned in the authentication phase) or type of device. This aspect allows for a
user to be assigned to a particular virtual network (created with the VXLAN),
which maintains different group of users with different privileges. In addition
to this, it is possible to apply the micro-segmentation. Micro-segmentation
indicates which users can be contacted by another user within the same Virtual
Network (VN). This approach reduces the risk of illegitimate access to various
parts of the network [19].

2.3.3 Management
In SD-Access the administration part becomes simpler than a traditional
network because it is based no more on the network primitives’ use (e.g., IP
addresses or VLANs) for segmentation and access control. The simplification
is mainly related to the fact that there is no need for translation between
business intent and IP and Access Control List (ACL) since the network
operator can define the connectivity matrix between the groups of users.
Consequently, the administration in SD-Access becomes more efficient in
terms of scalability, error occurrence, and complexity. At the network level,
each router keeps track of each endpoint through its IP address, and it adds
a 16-bit tag that represents its group when a new user logs in. In this way,
the connectivity rules contained in the matrix can be applied to the user.
Therefore, the network administration is radically simplified and common
operations such as IP address assignment planning or ACL configurations are
greatly improved [19].
 Background | 15

2.3.4 Summary
SD-Access allows users to access the resources through the customized
policies that allow for a dynamic and centralized implementation. At the
same time, SD-Access allows keeping the same services intact regardless of
the point (switch or access point) to which the user is connected. Moreover,
systems that use Cisco technology, such as Cisco Digital Network Architecture
(Cisco DNA) with Identity Services Engine (ISE) integration, also allow
access to the parts of the system that are responsible for traffic monitoring
and analysis. The SD-Access allows having an overview of the group and
authorization policies, the status of the entire infrastructure and an efficient
traffic control.

2.4 Software-Defined Wide Area Networking

Another technology derived from SDN is SD-WAN. SD-WAN allows applying
the concepts of SDN in the WAN. WAN networks allow companies to extend
their computer networks over large distances and connect the company’s
data centers to remote branches. Furthermore, the WAN provides access to
online applications and services necessary for the performance of corporate
functions.
The use of WAN occurs, most often, across national borders. Long
distances and different providers can facilitate network congestion, packet
delay variation, packet loss, and even service outages disturbing aspects that
reduce QoS and QoE. Another problem is related to the fact that modern
applications, such as VoIP calling, video conferencing, multimedia streaming,
virtualized applications, and desktops, require low latency, low jitter and ever-
increasing bandwidth.
Traditionally, for a company, WAN connectivity was guaranteed through an
internet connection or MPLS. None of the solutions is, however, completely
optimal. On the one hand, the internet connection offers considerable low-cost
bandwidth and ease of purchase/withdrawal, but at the same time, it does not
provide QoE and QoS guarantees. On the other hand, MPLS, at a much higher
cost, can provide constant and predefined bandwidth and latency guarantees.
Maintaining an MPLS infrastructure may, however, be complex and costly
especially in the case of worldwide distributed offices. Furthermore, since
the packets are forwarded from one office to another based on MPLS header,
it is necessary to indicate to the provider the routing rules of their internal
16 | Background

subnets. This aspect makes the reorganization of an office network slower

and not controllable by the company, which may lead to disruptions during
the migration. The last aspect of MPLS that differs from internet is linked
to privacy because the internet provider needs to be aware of the Local Area
Network (LAN) subnets’ type and the relationship between them, which may
not always be accepted by the customers.
One of the features of SD-WAN is the possibility to use multiple connections
based on various technologies. In particular, the appliance can choose the
most appropriate connection based on the user’s policies, the current network
situation (bandwidth, latency, jitter) and the type of traffic. Moreover, SD-
WAN allows to limit or eliminate the use of MPLS links, but, at the same
time, can make the best use of available resources, guaranteeing a good QoS
and QoE to the users. Indeed, over the connectivity, SD-WAN managed the
monitoring part, in order to perform dynamic routing, traffic segmentation,
security and QoS. Therefore, through the analytics, it is possible to apply
policies on a group of users basis to optimize the internet traffic that needs
the cloud access or to areas protected by firewalls [3].
In the following, the components, the topologies, the functioning of SD-WAN,
and the economic advantages will be exposed.

2.4.1 Network Appliances

The Metro Ethernet Forum (MEF) has defined the general architecture of
SD-WAN [21] that is generally composed of several SD-WAN edges, an SD-
WAN controller, and an SD-WAN orchestrator [22]. Each vendor of SD-WAN
products, such as Cisco, Juniper, VMware and Fortinet, names the components
of their SD-WAN with slightly different names or features, which will be
discussed in Chapter 2.4.5.
At the data plane level, there are the SD-WAN edges, like routers and switches.
They are enabled to receive and perform the functions imparted by the control
plane. At the control plane and management level, can be identified the
controller and the orchestrator that usually are organized as in Figure 2.4.
The control plane has the task of distributing reachability and security
information between the edges. Moreover, it has the task of distributing data
and application-route policies from the orchestrator to the edges and enforcing
control policies. Finally, it has the task of performing the best-path calculation
for Equal-cost multi-path routing (ECMP) routes and identifing the best route
 Background | 17

Figure 2.4 – Basic SD-WAN Architecture

to the Edges (second-best too, if configured).

The orchestrator takes care of the primary authenticator for all SD-WAN
components, facilitates the discovery of the new elements in the network, and
notifies edges of their public IP in case where Network address translation
(NAT) is used [3].

2.4.2 Topologies
The interconnections between different branches can be multiple and very
different from each other. However, thanks to SD-WAN it is possible to have
different logical network topology than the physical network topology. This
allows for the coexistence of an underly topology (the real one) and an overlay
topology (the desired one). The various topologies can be categorized in a
mesh topology, hub and spoke topology, and hybrid topology [23] as illustrated
in the Figure 2.5.
Mesh topology: The full mesh topology requires that all the sites have to
be interconnected directly to each other. It provides a different VPN tunnel
for each site interconnection. Therefore, even in the case of few locations,
the number of tunnels can be very high. A high number of tunnels can
lead to hardware/software limitations and loss of performances. The positive
aspects of this design are the capacity of having a low latency compared to
18 | Background

others topologies and that the amount of traffic has not to pass through the
Headquarter (HQ) because there is a direct connection with the destination
site.

(a) Full Mesh Topology (b) Hub and Spoke Topology

(c) Hybrid Topology

Figure 2.5 – SD-WAN Topologies

Hub and spoke topology: In this topology, the main office is set as a hub, while
the other offices as spokes. The peculiarity is that the spoke can communicate
only with one or more hubs, not with the other spokes, and the hub will
therefore act as a link between the various spokes. The feature due to which
hub and spoke topology is highly used is that it allows only a part of the traffic
to be transferred between two sites without going through the data center.
Based on that, a widely used configuration is the so-called hub and spoke.
The advantages are greater scalability, reduced costs and centralization of the
HQ in the network topology.
Hybrid topology: As illustrated in Figure 2.5c, hybrid WAN architecture
 Background | 19

involves the design of multiple hubs and spoke topologies interconnected

by various hubs. At the expense of a more complex design phase, the
hybrid topology can provide higher flexibility, scalability, and lower latency
compared to the more traditional hub and spoke topology. Moreover the hybrid
topology has a reduced number of tunnels compared to the full mesh topology.
When designing a hybrid topology, one of the first steps is to position the
data centers as primary WAN hubs. Other critical and data-centric locations,
such as a company’s headquarters, could also act as a primary hub. Once
the primary hubs are defined, it is crucial to consider efficient traffic patterns
for the remote sites. If there are important nodes, they can be configured as
secondary hubs, i.e., hubs that act as an intermediary between the primary
hubs and the spokes. In the case of a spoke-spoke connection in the same
area, the latency is reduced, but at the same time, the topology implies the
passage through the HQ for the other flows.

2.4.3 The Essentials of SD-WAN Architecture

As previously described, SD-WAN allows using the software to control the
connectivity, management, and services between the data center and remote
branches or cloud instances. In SD-WAN usually, each data flow between two
locations is routed on the link deemed most appropriate based on the policies
applied to the specific flow. Furthermore, all the flows over the WAN paths,
which are usually made to travel on the public network without particularly
precautions, are entered into a VPN tunnel built ad hoc between the source and
the destination. In this way, the confidentiality and security of the transmitted
data are ensured.
The features like the status of the connectivity and its performance is measured
through continuous monitoring of the switches by the controllers. The
interaction between switches and controllers occurs through the use of various
protocols. In the case of Cisco SD-WAN, the Overlay Management Protocol
(OMP) is used [3]. The OMP is a Transmission Control Protocol (TCP)
based extensible control plane protocol. It runs between edges and controllers
and allows for reduced control plane complexity and raises overall solution
scalability.
To ensure secure traffic flows, between the data plane and the control plane, the
OMP packets are encapsulated in TLS / Datagram Transport Layer Security
(DTLS) connections as shown in Figure 2.6 that illustrates a basic SD-WAN
architecture with the main protocols and characteristics. The edges and the
20 | Background

Figure 2.6 – SD-WAN communications between edges and controller

controller send and receive the information related to updating routing and
reachability, policies and security (i.e, the encryption keys) [3].
In the case of reachability and routing, the data plane relies on a routing
protocol such as Open Shortest Path First (OSPF) or Border Gateway Protocol
(BGP) for checking the available routes. It will also use the Bidirectional
Forwarding Detection (BFD) network protocol to detect a link failure and
to notify the controller immediately. BFD is used because it can be set
independently of the routing protocol used and because the echo time could be
defined differently from the routing protocol timer [24] [25]. Besides, the BFD
can be used to monitor a virtual connection, for example, a tunnel. In the case
of SD-WAN, BFD can be used to monitor liveliness, and quality measurement
(latency and jitter) of each VPN created between two locations [3].

Virtual Private Network

In today’s distributed computing environment, VPN offers an attractive
solution to network managers. A VPN is established between a set of
computers that interconnect over a relatively insecure network. Moreover,
the VPNs use encryption and protocols to provide authentication and
confidentiality [17]. With VPN it is possible to use a classic internet
connection like a private WAN, with the same security benefits but
significantly reduced costs. Essentially, a VPN uses encryption and
 Background | 21

authentication in the lower ISO-OSI protocol layers (layer three or layer two)
to provide security and connection over an otherwise insecure network. The
most common network protocol used for this purpose is at the IP layer and is
known as IP Security (IPsec).

IP Security
The IPsec protocol is a layer three protocol, used for most VPN
implementations, and it is considered secure and reliable. It ensures
authentication and integrity for packet sources, confidentiality and access
control [18]. It can work in two modes: transport mode and tunnel mode.
In the first case, only the payload and part of the IP header is encrypted.
Otherwise, in the second case, the whole packet is encapsulated, and hence
the destination address of the gateway is shown as the real destination.
In addition to this differentiation, two types of encapsulation can be chosen:
the Authentication Header (AH) or Encapsulating Security Payload (ESP).
The first one provides only integrity and authentication through Hash Message
Authentication Code (HMAC). The AH allows the recipient, using a hash
function, to verify if the content is authentic and not modified during transport.
The second type of encapsulation, in addition to authentication, encrypts the
TCP/UDP segment, and thus it also ensures confidentiality. In the case of
tunnel mode, the IP header is encrypted too, and thus the destination IP can
be hidden from the third parties, which allows for better confidentiality [26].
It is necessary to establish a Security Association (SA) to ensure that the host
or destination gateway decrypts and correctly verifies the IPsec packets. The
SA is a set of information (unique for each IPsec flow) that allows to correctly
identify and decrypt the packets. These information are based on the used
encryption protocol, key, method (AH or ESP), transport or tunnel mode,
and other specific parameters [18]. To establish the entire communication via
IPsec it is necessary to use Internet Key Exchange (IKE) protocol. IKE is a
key management protocol that uses the Diffie-Hellman key exchange to share
the symmetric keys for encrypted communication for both users.

Auto-VPN
Many SD-WAN vendors, such as Cisco Meraki or Juniper, use the Auto-VPN
concept to create an IPsec tunnel between various peers on the SD-WAN
network [27]. Auto-VPN is a concept that provides the creation of a VPN
tunnel automatically, without having to manually configure the two peers even
22 | Background

if they are protected by firewalls [28]. In case of Auto-VPN, an automatic

NAT Traversal must be performed. The process can be summarized in three
phases, which are illustrated in the Figure 2.7:
• First, it is expected that both devices will register on the cloud, accessible
at a known address.
• The VPN register will then inform each peer with the IP address and
port of the other peer.
• In the last phase, the two peers will try to communicate with the other
peer through the known address. The communication will be possible
through an approach called Hole punching [29] which allows to create a
session between two devices in the presence of NAT. Once the session is
established, the IPsec tunnel will start to allow communication between
the two devices.

Figure 2.7 – Auto-VPN and Automatic NAT traversal with UDP hole punching

The advantages of this technology are many, but the two most important are:
the simplicity of configuration and creation of the VPN tunnel between the two
devices and the ability of the devices to update their cryptographic material
quickly and automatically.
 Background | 23

2.4.4 Economic advantages

SD-WAN has considerable advantages even from an economic point of view.
In fact, the main reason to migrate from MPLS to SD-WAN is strictly related
to the reduction of the MPLS connection’s license costs. In addition to this
aspect, numerous other aspects allow saving time and money. One of the
aspects is, for example, related to the reduction of the downtime, that is, the
time during which the services are not reachable to customers due to a power or
uplink failure. Indeed, according to an estimate by Cisco, SD-WAN can reduce
unscheduled downtime by up to 94% [1]. The downtime reduction approach
on which SD-WAN relies is based on the possibility to use multiple links to
forward the traffic and the presence of backup solutions, for example, LTE.
This feature is more effective thanks to the centralization and softwarization
of the configurations, and the approach allows reducing by up to 58% the time
provisioning of policies and network configurations [1]. Another aspect is
related to the reduction of the installation and maintenance costs as the SD-
WAN provides the possibility of creating the configurations remotely, directly
from the company headquarters or some other remote location.

Zero Touch Provisioning

Zero-Touch Provisioning (ZTP) is a feature that allows for the automatic
provisioning and configuration of the devices. Consequently, it simplifies
network management, reduces the risk of long outages and the risk of errors
due to manual settings. Thanks to the ZTP, it is sufficient to create the
configuration on the management side and, once completed, send it to all
the devices on the network that will change their behaviour in a rapid and
coordinated way [30]. With this method, it is possible to centralize the
configuration of multiple devices that may be located at different places.
Moreover, it is also possible to remotely change access rules, perform routing,
and carry out in-depth monitoring of the network.
ZTP is extremely easy and fast but has possible risks related to unauthorized
access and actions like interception and spoofing that can compromise the
security and correct functioning of SD-WAN [31]. These threats exist because
the ZTP server must accept, by definition, requests from unidentified and
unauthenticated devices on the internet.
In conclusion, ZTP allows to simplify the network configuration but, at the
same time, it requires greater attention from the point of view of security
(i.e., it should be used together with the certificates or other authentication and
24 | Background

authorization mechanisms for mitigating the corresponding security issues).

2.4.5 Vendors comparison

SD-WAN solutions are widely produced and supported by several vendors
such as VMware, Fortinet, Cisco SD-WAN (Viptela), Citrix Net-Scaler, and
Cisco Meraki [32] [33] [34] [35] [36]. Each of them provides an SD-
WAN solution with sometimes different features and services. However, the
choice of a particular vendor is not always determined only by the economic
aspect but also by the possibility of a particular device to support desired
functionalities. For example, Fortinet, which has always been one of the
leading companies in the firewalls sector, produces firewalls branded as next
generations firewalls, and they include the SD-WAN solution. They include
numerous security solutions, such as web filtering, intrusion detection, and
advanced threat detection. On the contrary, VMware is focused on optimizing
the user experience for cloud applications. Also, Cisco SD-WAN and VMware
focus heavily on direct cloud access. Cisco Meraki instead makes its strengths
the User Interface (UI) simplicity, the integration of LTE/5G backup, and
infrastructure management through a cloud-based controller.
Therefore, the task of indicating a preferable solution or vendor for SD-WAN is
challenging since it requires in-depth analysis of various aspects. For example,
it is very important to analyze the size and complexity of the network to
identify whether a particular vendor or solution is adequate for the architecture.
Furthermore, it is advisable to also analyze customer requirements, both from
an economic and technical point of view. Indeed, each seller or appliance has
specific characteristics or particularities that may provide different degrees of
support for diverse requirements of the customers.
As Table 2.1 shows, there are many primary and secondary characteristics of
an SD-WAN network, but not all vendors support them in a complete and
optimized way compared to their direct competitors. For this reason, the
analysis phase for an SD-WAN migration turns out to be essential to obtain
the design of an SD-WAN network that fully satisfies the functionality and
requirements of the customers.
For example, if LTE / 5G backup solution is the most important characteristic
promising candidates are Cisco SD-WAN or Cisco Meraki. On the contrary,
if Infrastructure as a Service (IaaS) and Software as a Service (SaaS) are more
important characteristics, the recommendation would be to use a VMWare,
Citrix or Cisco SD-WAN solution as they allow a more complete optimization.
 Background | 25

Table 2.1 – Vendors comparison

based on [37] [38] [32] [33] [34] [35] [36]
Characteristics VMware Fortinet Cisco Citrix Cisco Meraki
SD-WAN
SW upgrade Require new Not require Not require Not require Not require
to SD-WAN hardware to new hardware new hardware new hardware new hardware
use SD-WAN to use to use to use to use
SD-WAN SD-WAN SD-WAN SD-WAN
Core, edge, Limited Limited Reliable Limited No
and cloud appliances
SD-WAN built to
service core,
edge, and
cloud
locations
Segmentation Yes Limited Yes Limited Yes
True Need Need Automated Not Automated
zero-touch additional additional appliance completely appliance
provisioning authentication authentication activation and zero-touch activation and
steps steps provisioning provisioning
Active-Active No Yes Yes Yes Yes
SD-WAN
Encrypted No No Yes, can Yes Yes, can
traffic detect detect
analysis malware by malware by
matching matching
encrypted encrypted
SHA patterns SHA patterns
without without
decryption decryption
Threat No Limited Yes No Yes
intelligence
Saas/Iaas Limited Iaas No for SaaS Intellligent Limited Iaas No
Connectivity and Saas Limited for path selection and Saas optimization
optimization Iaas for Saas and optimization
automated
gateway for
Iaas
Advanced No significant No significant Yes Only few Only backup
LTE cellular cellular models solution
Solutions support support support it
Wi-Fi 6 / No No Yes No Yes
5G-ready
Same-day Limited Limited One business Limited One business
product replacement replacement day replacement day
replacement options options replacement options replacement
26 | Background

2.5 Key Features of SD-WAN

Many research papers consider different aspects of SD-WAN. In what follows,
some of the SD-WAN aspects that have received the most attention in the
literature will be discussed. These aspects include features related to security,
segmentation, cloud integration, policy framework, load balancing, traffic
shaping, voice optimization, and HA.

2.5.1 Security
Security is one of the most fundamental aspects of SD-WAN. The traditional
approach without SD-WAN involved access to the network only through a data
center, protected by firewalls or other protection systems for threats coming
from outside. With the advent of the increasing use of SaaS and IaaS, the
need to pass all traffic through the data center is therefore obsolete and not
very functional. A possible solution could be the use of SD-WAN together
with a local internet gateway secured by using a firewall in each branch. In
this way, all the inbound/outbound traffic can be analyzed and verified and
each branch can reach the Internet in a secure way [1].
The solution described above is, however, not very scalable, and an alternative
and more used solution is Secure Internet Gateway (SIG). SIG allows all traffic
coming from one or more branches to converge in the cloud, ensuring adequate
security. It keeps unauthorized traffic from entering an organization’s network
by analysis of the incoming/outcoming traffic to prevent malicious website
traffic, viruses, and malware. In this way, the IaaS and SaaS traffic is not
conveyed to the HQ (reducing congestion), but at the same time, adequate
traffic safety and monitoring are guaranteed. SIGs often provide databases on
threats and other useful tools to make security shared with other SIGs around
the world.

2.5.2 Segmentation
An important aspect of the network infrastructures, and in particular of SD-
WAN, is network segmentation. Network segmentation is an architectural
approach that divides a network into multiple segments or subnets. It allows
improving monitoring and control of the network through policies. It also has
the potential to improve security aspects by preventing simultaneous access
to all of the resources. Consequently, in case of intrusion, only the part of
the resources exposed to the attack will be compromised, while the rest of the
 Background | 27

resources will remain secure [39].

The concept of segmentation applied to internal resources can also be applied
to internet browsing. Indeed, users belonging to different groups usually have
access to various functions that may limit their browsing on the internet. For
example, a company with guests will only use firewall and URL filtering to
prevent unauthorized access to a particular website. So, in this case, choosing
to use a IPS may not be necessary [1].
There are various methodologies to implement segmentation and they need to
be used in a coordinated way in order to achieve a particular functionality. The
first way is to implement segmentation physically and to divide the network
into various subnets. This is a feasible and relatively secure strategy, but it
allows limited flexibility and requires significant time expenditure when it is
necessary reorganized the IP addresses of the entire network.
The second way is to implement segmentation by using VLANs. It means
to split the network logically, and thus to allow a rapid reconfiguration at
the software level. The VLAN, therefore, allows dividing the traffic at
layer two that passes on the same switch or physical link, making flows
belonging to different VLANs invisible to each other. With VLANs, it
is possible to virtually create two or more invisible networks at layer two,
which may increase the security and the efficiency of the switch, reduce the
broadcast domain, and the work of the Spanning Tree Protocol (STP) if the
network topology changes [40]. Although VLANs can increase the security
in terms of visibility and simplify the management of security policies, they
are vulnerable in terms of cyber attacks that can compromise the network’s
security. Some of these are MAC flooding, ARP spoofing and LAN hopping
[41].
The third way is segmentation through ACL and/or authentication. This type
of segmentation takes place through the identification of the user connected
to the network. Based on his membership group, the user is allowed to have
access only to parts of the network infrastructure, i.e., to particular resources.
Finally, the last type of segmentation, also called micro-segmentation [42],
allows traffic to be segmented through policy and analysis of the application
layer (level seven) [43]. This type of segmentation is more granular and
flexible. As illustrated in the Figure 2.8, the portion of the reserved bandwidth
depends on the service type.
Segmentation in SD-WAN is used in various areas of the network
28 | Background

Figure 2.8 – Micro-segmentation

infrastructure. For example, on the LAN side and also on the WAN side for
the VPN traffic. On the LAN side of the offices, the traffic is often split into
various VLANs in order to expose only the necessary ones via SD-WAN to
the other offices. In this way, only part of the network is shared with the other
offices, allowing some areas to be kept private. Otherwise, from the point
of view of VPN traffic, segmentation and micro-segmentation are applied
to make traffic flow on a specific connection based on the user’s type. The
user is first differentiated based on the ACL and the destination of the traffic
(macro segmentation) and, after that, based on the policies and analysis of the
application layer (micro-segmentation).

2.5.3 Cloud integration

The integration of cloud computing service models such as SaaS, Platform as a
Service (PaaS) and IaaS [44], is becoming more and more used and requested.
Cloud integration happens mainly because of the rapid increase in the number
of services and applications hosted in the cloud. Users can easily access the
services regardless of the types of their devices, and they can also easily share
the services within the same company. Unfortunately, the data centers are
usually not located in a geographically area close to the user. Moreover, the
users do not have the possibility to verify and decide where their personal
data will be stored. These two elements make access to cloud resources more
subject to higher latencies and, therefore, to lower QoE.
Traditionally, the access to cloud services occurs through the passage in the
 Background | 29

data center or the centralized internet gateway. In particular, traffic to the cloud
is limited due to the need to concentrate traffic to the outside in order to monitor
it. Monitoring can take place through the use of firewalls and other systems
that ensure the security of the internal network. However, this strategy leads
to higher latencies and, therefore, to low QoE [45].

Figure 2.9 – Direct access to internet for access to cloud services

On the contrary, implementations of SD-WAN from various providers, such

as Cisco SD-WAN Cloud OnRamp [45], offers cloud connection solutions
bypassing the HQ. These solutions permit accessing the internet directly from
the branch or a regional hub. Moreover, these types of cloud integration
solutions dynamically identify the best path based on the types of provided
services, and thus they allow for the optimization of the QoE and the decrease
of the traffic inside the company network. The service optimization is possible
thanks to dedicated policies and the ability to identify specific cloud URLs,
such as Microsoft 365.
A similar concept is applied in the case of traditional internet traffic; an
example is the Cisco Meraki solution that allows traffic not destined to other
locations of the SD-WAN infrastructure to be routed locally to the internet,
reducing the traffic passing through the hubs.

2.5.4 Policy Framework

Another aspect of SD-WAN is the policies to route traffic over a specific link,
which are fundamental for the proper functioning of the segmentation and
routing of traffic on the various links. The policies mainly belong to two
macro-categories: localized data policy and centralized policies [46].
Localized data policies are local policies for a single device. They have the
task of classifying incoming packages based on the level of their importance.
In other words, they allow the assignment of each packet to a particular queue,
30 | Background

based on its priority level. Hence, the local data policies can be recapped as
access lists that allow the provision of QoS [3].
Centralized policies, on the contrary, can concern the control plane or the data
plane. The control plane policy affects the communication routes between the
controller and switch. The data plane policy instead affects only the rules that
select the routes for the forwarding of packets. Besides, they are also used to
decide which packets to analyze at the payload level [46].

2.5.5 Load balancing

One of the most interesting aspects of SD-WAN is the load balancing feature,
which gives the possibility to use multiple links to send the traffic. The load
balancing function is based on policies or the network’s status.
SD-WAN offers the possibility to establish a VPN tunnel for each physical
WAN link. Based on the link state in terms of congestion, latency and failures,
SD-WAN edges can decide to migrate part of the traffic to the secondary
link. This action allows the mitigation of the bottleneck effect, selection of
the best links for each application and prioritization of the applications [47].
Furthermore, the network administrator can limit the bandwidth of a specific
link to avoid its overloading and it can also limit the bandwidth used by a single
user to avoid the saturation of the network due to the excessive traffic of some
users connected to it.
More recent approaches show that it is possible to change the flow paths
not only in a reactive way but also in a pro-active way. Reactive load
balancing is done upon the bandwidth or latency limit is reached. On the
contrary, pro-active load balancing relies on the use of the deep learning
algorithms that forecast the network congestion [48]. In particular, the
deep learning algorithms allow to analyze the trend of the network’s vital
parameters and, based on the records of the network, decide if the routing
rules need to be modified [49]. This approach is not completely ready for
current implementations yet, but it represents an interesting solution from a
performance point of view. Indeed, by anticipating the network congestion,
the pro-active load balancing has the potential to further increase the QoE of
the users.
 Background | 31

2.5.6 Traffic shaping

Traffic shaping allows to analyze, tag, and prioritize the traffic at the
application level. Traffic shaping could be achieved by using a Differentiated
Services Code Point (DSCP) tag or by analyzing traffic at the application level
[43]. In such a way, the traffic can be segmented and prioritized based on
certain rules.
For example, consider that there are two streams of traffic, VoIP traffic that
is latency-sensitive and file transfer that is bandwidth-intensive. In case these
two rules are used together, the traffic shaping will increase the throughput
of the second stream only if the quality of the first stream is considered
satisfactory [47]. Otherwise, in case of dropping and consequent low QoS
and QoE for the first flow, the throughput of the second flow is reduced to
improve the latency of the VoIP traffic [47].
Traffic shaping also allows monitoring and managing the use of network
resources. In particular, traffic shaping allows for resource management that
can be done not only from the point of view of a single user but also based on
the type of traffic. This allows for the solutions that can be used to optimize
the entire network. For example, a possible solution could be to create policies
and rules for particular types of traffic that may be specific for the periods of
the day during which the network is congested and the QoS reduced.

2.5.7 Voice optimization

Two of the flows most subject to loss of performance, and so low QoE for users,
are the VoIP and streaming flows. VoIP flows require not only a large amount
of dedicated bandwidth but also a low jitter and a relatively low latency.
Meeting these requirements together is challenging, and thus VoIP flows are
very often subject to loss of performance, and to a low QoE for users.
There are numerous approaches developed for mitigating the issues that
occur in the case of VoIP traffic. Some of them, such as traffic shaping
allow prioritizing the VoIP flows by using particular tags. This method
occurs through traffic shaping, and is applied in both traditional and SD-
WAN approaches. Other approaches are based on detection and correction
techniques that are used to compensate for packet loss or packet errors. One
of the most known techniques of this type is the Forward Correction Code
(FEC) described in Figure 2.10a. The FEC technique allows adding series
of bits at the end of a packet. FEC bits permit verifying the correctness of
32 | Background

(a) Forward correction code (b) Packet Duplication

Figure 2.10 – Voice Optimization Strategies

the transmitted data and the recovery of the corrupted packets in the case of
few errors. However, the FEC technique may not be effective in the case of
burst errors that can result in damaging both the packet and the FEC data. For
this reason, some other methodologies need to be used in order to separate
the repeated information in different packets. One strategy is to perform the
FEC check at each hop to immediately identify the presence of errors and
carry out the correction. However, this requires that the TCP/IP flow control
is not end-to-end. Another strategy that can work together with the one just
described is to make the FEC relative to n separate packets. For example, the
first packet of FEC will not refer to sequential packets (i.e. 1, 2, 3, 4, 5) but
to interspersed packets (i.e. 1, 11, 21, 31, 41). In this case, the risk of burst
error is reduced since the packets are transmitted at different times. However,
one of the possible disadvantages of the described solution is that it requires a
greater buffer memory for saving all the packets received before carrying out
the FEC check [50].
The issues related to VoIP are usually related to the transmission of VoIP traffic
through a single link. With the introduction of SD-WAN, it is possible to
choose between more than one link and decide in real-time which one to use
based on the current network conditions. The choice of the link allows to
obtain better QoE of the call than the traditional approaches for the same values
of bandwidth, latency and jitter [51]. Beyond that, suppliers are increasingly
adopting the use of a particular technique called packaged duplication [1].
This technique involves sending the same information on two different physical
links, as illustrated in Figure 2.10b. In this way, if the packet loss occurs, it
will be sufficient to use the backup stream to obtain the lost packets. The
disadvantage of this technique is a major use of bandwidth, but, thanks to
the ever-increasing bandwidth offered by the providers, the method may be
 Background | 33

acceptable in particular scenarios. Furthermore, in the case when the extra

bandwidth is required by other applications, there is a possibility to terminate
the second link and by doing so to reduce the bandwidth used for a VoIP call.

2.5.8 High Availability

The HA is a characteristic of a system that aims to guarantee a level of
operation above the average through a set of procedures and measures.
Usually, the HA is quantified with a percentage value between 0 and 100,
where one hundred corresponds to a network infrastructure that is always
operational. It is very difficult to achieve 100% network availability, but, in
recent years, huge efforts have been made to reach higher availability values.
Systems with an availability equal to 99.999% are commonly defined as three
nines, systems up 99.9999% of the time four nines and so on. To give an
example, a four nines system will have downtime in a year of less than one
hour. Low downtime is especially important for fundamental services, such
as hospitals or electricity grid management networks. These services must
remain available as much as possible and, for this reason, they increasingly
rely on network and cloud infrastructure.

Failover
One of the main aspects of HA is related to failover, which is the ability to
create redundancy at the level of equipment and/or links to keep the entire
infrastructure up and running. It must take place at all levels of SD-WAN:
data, control and application planes, and the connections between the various
levels. All these particular levels are equally important, and for each of them,
several solutions have been designed to overcome the issues. The following
sections provide a brief overview of the mentioned levels.
API and management plane
The management level of SD-WAN is ensured through a cluster of
applications. The clusters are located in the same Data Center (DC) and are
all kept active at the same time. Moreover, to maintain geo-redundancy, they
are also duplicated in various DCs, in which case the applications have to
remain active/passive [3]. In addition to this, the applications are designed
in a modular way to avoid cascading faults between different services, and
therefore to limit inefficiencies between different functions of the management
level [52]. Finally, in the event of a general fault (e.g., caused by non-
reachability of all management servers), SD-WAN is designed to maintain the
34 | Background

current state and therefore guarantee operation while maintaining the current
state policies [52].
Control plane
There are numerous approaches used at the control level to guarantee the
HA in case of fault of one or multiple controllers. Some of the approaches
provide SMaRtLight [53], i.e., the use of a data store that allows obtaining an
immediate backup in case of a controller failure. This approach allows copying
the configuration to another controller. Other approaches, implemented
with various protocols, provide the establishment of continuous update
messages and verification of the appliance’s status between active and dormant
controllers [52]. An example of implementation is one proposed by Fonseca
and is called CPRecovery [54].
In addition, it is also possible to use controllers not belonging to the
zone concerned, and therefore re-associate the switches to the remaining
controllers. This approach could be very interesting in the case of several
controllers associated with various devices in the network, and it will be later
discussed in more detail.
Data plane
At the data plane, the HA can be achieved connecting two routers and
configuring them through the Virtual Router Redundancy Protocol (VRRP).
The VRRP allows to virtualize multiple physical routers and merge them into
a single virtual router (i.e., a router with a virtual IP), as shown in Figure 2.11.

Figure 2.11 – Virtual Router Redundancy Protocol

In this way, the traffic is always directed towards the virtual IP address, and it
 Background | 35

is the task of the VRRP protocol to redirect the traffic towards the new master
router. In the case of master failure, the secondary router (slave) takes its place
and continues routing the traffic [3]. This approach is particularly effective
because the devices do not change the gateway as it always remains the one
defined as a virtual router.
It is also necessary to maintain the redundancy on a physical level, and
a technique called switch stack has been adopted for this purpose. This
technique relies on using two or more physically separate switches. They are
interconnected through special ports called stack ports [55]. The switches
have to be configured such that they are aware of the stack. This operation can
be done via the primary switch that is in charge of managing and discovering
the other switches in the stack. The other switches remain as slaves and can
replace a master if a master failover is identified [55].
Link Aggregation Control Protocol
Redundancy has been also considered for links, and in this context, it has been
standardized by IEEE 802.3ad, which provides the standard for the aggregation
of multiple physical interfaces in a single logical link [56]. According to
this standard, the traffic passes on the various links by exploiting the load-
balancing of the Link Aggregation Control Protocol (LACP), which allows
the use of the bandwidth of all the links added together. For example, in the
case of 4 links of 10 Gbps, the connection looks like a single logical link of 40
Gbps. Besides, if the links are configured in a HA configuration in case of a
single link failure, the link will remain unaltered, with only a loss of bandwidth
(e.g., in the described example, the bandwidth loss will be 25%).

Failure detection
The rapid identification of a failure is one of the most essential steps for
the proper functioning of HA. The failure detection usually occurs through
protocols and algorithms that have the task to identify the malfunction of a
link. The monitoring of the failures can happen between the control and data
plane or between two switches in the data plane.
In the case of failure at the data plane level, a possible approach is Sentinel
[57], which supports for the creation and installation of backup tunnels for
each link that could fail. In this way, when a failure occurs on a specific
connection, it will be sufficient to use the previously created backup. Other
implementations, such as SafeGuard [58] provide the proactive creation of
tunnels and backups. SafeGuard also provides an analysis of the traffic present
36 | Background

on each link, which allows for use of traffic weighting methods. In this way,
SafeGuard can prevent situations in which the failure of a link leads to an
overload of a backup link. There are also other approaches that use the BFD
protocol to verify that, after a link failure, a quick recovery is carried out on
the backup link [25] and that the remaining links are not overloaded. These
two objectives can be achieved through a recalculation of the optimal path
with BFD, once when the traffic migration on the secondary link is finished.
Variations to the above approaches are called hybrid [59], and they allow
for the use of a backup tunnel after having detected the malfunction of the
link. When the malfunction of the link is detected, BFD is used to check the
status of the remaining links. Subsequently, the routes are recalculated based
on the new parameters, which allows for a complete change of the topology.
Finally, there are approaches that exploit relations derived from the monitoring
system and the virtual connections. The main purpose of these approaches is
to deduce the topology of the underlying route and to calculate the best path
at the presentation of the failure [60].
In the case of a failure between the control and the data plane, the two widely
used algorithms are Greedy failover (GF) and Pre-partitioning failover (PPF)
[61]. The GF algorithm is based on echo messages between the controller and
switch that are used to verify the actual presence and operation of the devices.
When a switch loses connectivity with its controller, it sends a frame using
Link Layer Discovery Protocol (LLPD), which indicates that the switch is no
longer controlled by the controller. The other switches forward this message to
their own controllers, and subsequently one of the notified controllers sends a
request to the orphaned switch. At the end of the operations, the controller
that becomes associated with the orphaned switch updates the information
concerning the switches it controls. The PPF instead is pro-active, and it uses
Greedy’s algorithm. The main difference is that the PPF algorithm already
selects, for each switch, the controller backup. Therefore, when a controller
detects a failure of another controller, it tries to establish a connection with all
the switches affected by the failure. If the switches recognize a controller as a
backup, they accept the new association. Once the connection is established
with each of the orphaned switches, the backup controller updates its database.
Finally, approaches that take inspiration from Sentinel and SafeGuard were
applied to make the control-data connection fault-tolerant [62]. The main
idea on which these approaches are based is to proactively create backup links
between all the controllers and the switches. Consequently, in the event of a
controller fault, it not necessary to redistribute switches at the moment as the
 Background | 37

secondary paths have already been defined. A full-mesh approach, proposed

in [62], demonstrates the rapid migration of the switches from one controller to
another. Although interesting, the proposed method is likely to not be feasible
in reality due to the high overhead.
38 | Background
 Methodology | 39

Chapter 3

Methodology

This chapter describes the research process and the phases of the performed
experimental design. It also discusses the data collection required for the
experiments, its analysis and evaluation.

3.1 Research Process

In what follows the phases taken to conduct an accurate investigation of the
subjects are discussed. As illustrated in Figure 3.1, there are four main phases:
background study, analysis of the current implementation and needs, design
and implementation, and implementation verification.

Background study
The background study involved a systematic analysis of different topics
related to the SD-WAN. First, the literature on the characteristics of SD-
WAN networks was studied. Second, the literature on planning, design, and
implementation of SD-WAN networks was reviewed. Finally, the SD-WAN
solutions provided by several vendors were compared.

Existing infrastructure analysis

The second phase involved the analysis of the infrastructure already present on-
site. The primary objectives were to understand how the current infrastructure
works and what are the clients needs, and then to identify the SD-WAN
solutions that are feasible within the current organization. These objectives
40 | Methodology

were achieved by acquiring the required knowledge from the company

colleagues, reviewing vendors’ documentation, and using the knowledge
obtained during the background study.

SD-WAN infrastructure design

The focus in this phase was on the design of a new network infrastructure
that uses SD-WAN. The proposed SD-WAN infrastructure was designed
based on numerous aspects that were studied during the previous two
phases. In particular, the SD-WAN infrastructure design involved selecting
a set of methodologies and specifications that are in compliance with the
characteristics of the current infrastructure and the best practice proposed by
the vendors.

Evaluation
In the last phase, the proposed solution was evaluated through simulations.
The first set of simulations was performed in order to evaluate the aspects
related to traffic shaping. Then, the ability of the new network infrastructure
to withstand failover events was tested. Finally, the previous and the proposed
infrastructures were compared in terms of the fault detection performance.

Figure 3.1 – Representation of the procedural phases

 Methodology | 41

3.2 Experimental design

The project presented in the thesis introduces numerous aspects that can
be analyzed. In particular, it will be necessary to examine the current
structure (AS-IS) based on the physical implementation and documentation
given by the customer. Subsequently, the best product will be identified,
based on Chapter 2.4.5 and the customer requests, and then a design of a
new network infrastructure will be proposed. Downstream of these phases the
infrastructure will be implemented through a careful design analysis to obtain a
final network infrastructure well integrated with the previous one. This stage
includes configuring the segmentation, policies, and custom rules for traffic
shaping based on general practice and requirements of the customer. Finally,
the correct functioning of the infrastructure and its features will be verified
through ad hoc tests. These tests will be performed by simulating some test
cases, which will confirm the expected behaviour of the infrastructure.

3.2.1 Reliability and validity of the network

infrastructure
Since the analyzed network infrastructure is an existing and functioning
implementation, the data collected and the behaviour of the entire
infrastructure are consistent and represent the real behaviour of an SD-WAN
network infrastructure. Therefore, it is possible to recreate the entire or only
the parts of the described infrastructure and to analyze its behaviour in multiple
ways. The simplest way is to use virtualizers that allow for the use of software
for simulating the case under analysis. Another possible solution may be to
recreate the network infrastructure with the same hardware and implement
the same rules. Although in both cases, the latency values may be different
because they are unpredictable values, the results related to load balancing,
traffic shaping and HA are expected to be the same because the operation will
be based on rules implemented in the same way.
42 | Methodology
 From MPLS based network to SD-WAN infrastructure | 43

Chapter 4

From MPLS based network to

SD-WAN infrastructure

This chapter describes the network infrastructure present on the study site,
the customer change requests, the functionalities that a new infrastructure
should provide, and the various steps that need to be performed during the
design of the new network infrastructure. Moreover, the chapter describes
the implementation of the SD-WAN solution, with a particular focus on the
various features, like SD-WAN and traffic shaping. Therefore, this chapter
collects all the operational steps carried out during the degree project.

4.1 Overall Network Structure

The flexibility and scalability of SD-WAN allow it to be applied in
very different network infrastructures. In the analyzed case, the network
infrastructure is an existing implementation of an Italian company that
performs billing, personnel management and accounting services for large and
medium-sized enterprises and industrial groups. The corporate infrastructure
is structured in numerous offices located in different cities. The entire
infrastructure consists of an HQ, identifiable for the structural complexity
and the number of operators operating in it, and eight secondary sites (or
branches). The HQ and the various offices are interconnected due to the need
to communicate with each other. Moreover, the secondary branches need
access to the data center located in the company’s headquarters.
44 | From MPLS based network to SD-WAN infrastructure

4.1.1 Headquarter Network Infrastructure

Different from the branch offices, the HQ has a data center that stores the
user data and relevant information. Furthermore, the data center provides
the applications that allow the end-user to use various services and carry out
their activities. To secure the data center, user access, and the entire network
infrastructure a firewall has been placed between the internet and the internal
network. The firewall, among its many features, allows to identify possible
intrusions from outside by unauthorized users (IDS) and to prevent them (IPS).
Moreover, the firewall permits monitoring the traffic from the internal network
to the outside. Usually, this traffic is produced by employees of the company.
In the considered case, the firewall consists of a pair of Checkpoint 4400 1 .
Checkpoint is a leader in the firewall industry and offers multiple features
such as threat prevention, policy creation and traffic analysis. The firewalls are
organized with a cluster infrastructure in active-passive mode. It means that
they are grouped to look like a single logical entity that exposes a single Virtual
IP (VIP). One of the important features of the cluster infrastructure is that it
offers HA by supporting redundancy in the case of a failover. However, the
active-passive mode does not allow sharing the traffic processing load because
the secondary firewall is in standby mode. The VIP feature is available only if
the two physical firewalls are the same type and connected to each other with
one or more physical links that allow communication between them.
Finally, in the HQ, there are also virtually implemented load balancers. In
particular, a pair of Citrix Netscaler, model 10.1.x is used to perform traffic
optimization and load balancing at Layer 4 and Layer 7 of the ISO-OSI stack,
and to accelerate web applications through data compression and caching.
Besides these functions, a pair of Citrix Netscaler is used for monitoring
servers’ health and assigning network and application traffic to additional
servers for efficient use of resources.

Organization of the connectivity

Due to its importance, the headquarters must be provided with connectivity to
the internet and the other branches. As shown in Figure 4.1, connectivity to
the internet is achieved through two WANs. As illustrated in the figure, the
traffic from/to the core of the headquarter is routed through the firewall. In
this way, the traffic is analyzed and verified before being sent/received by the
1
https://www.checkfirewalls.com/4400.asp
 From MPLS based network to SD-WAN infrastructure | 45

HQ. Furthermore, downstream of the firewall, there is a router to select the

WAN. The main purpose of this router is to bring together more connectivity
and convey the flows to the firewall.

Figure 4.1 – AS-IS Network Infrastructure

As illustrated in Figure 4.1, the current implementation provides connectivity

with other offices through an MPLS connection and VPN tunnels. The main
purpose of this infrastructure is to provide the backup link in the case of a
failure of the primary link, and thus to provide a resilient connection between
the HQ and the secondary offices. For both types of connection, a router has
been placed in the internal area of the HQ. In such a way, the traffic to the
branch offices occurs either via a VPN tunnel or via MPLS in a secure manner
and, therefore, without the need to analyze the traffic through the firewall.
In the MPLS case, the traffic is sent through a network completely separate
from the public internet. This makes it difficult for an attacker to carry out an
attack such as sniffing since it would need to physically connect with a device to
the edge router of the MPLS network (c.f., Chapter 2.1.1). Similarly, the router
for connection via VPN tunnel does not require a firewall. This is because all
the traffic (incoming and outgoing) is properly encrypted and authenticated. If
the traffic is sent on the MPLS link, the data flow is sent to the corresponding
router (router R3 in Figure 4.1). As shown in Figure 4.2, there is a point to
point connection with the edge router that belongs to MPLS infrastructure.
Otherwise, if the traffic is sent via a VPN tunnel, the traffic is forwarded to the
corresponding gateway (router R2 in Figure 4.1) and encapsulated correctly
46 | From MPLS based network to SD-WAN infrastructure

based on the IPsec standard and the necessary information (IP addresses,
authentication certificates and cryptographic material) required to establish
the VPN tunnel.

Figure 4.2 – Organization of the headquarters’ subnets

4.1.2 Branch Network Infrastructure

The branch offices appear all with a similar topology as the result of smart
previous network planning (c.f. Figure 4.1). The various locations segment
the internal traffic through the use of VLANs and thus if the traffic needs to
be forwarded to the outside, it is sent to the default gateway. For each branch,
the default gateway consists of two routers virtualized using the Hot Standby
Router Protocol (HSRP) which is a Cisco’s proprietary protocol corresponding
to the VRRP protocol. Therefore, there is an active router and a backup router
that is on standby and can be used to increase availability in the case of router
failover or loss of connectivity.
The primary router is the router dedicated to the MPLS connection, and it is
directly connected to the MPLS edge router provided by the ISP. Therefore, the
primary router allows the traffic to be forwarded to the HQ or other offices via
the MPLS infrastructure. The router in stand-by is connected to the internet
link, and through the use of VPN tunnels it allows for creating a backup link
 From MPLS based network to SD-WAN infrastructure | 47

between the HQ and the peripheral site. The local internet traffic is routed
towards the default gateway. If the primary router is active and the secondary
is passive, the traffic is forwarded from the primary to the secondary gateway
that permits local internet navigation. Therefore, it is not required to transfer
all the traffic via MPLS to the HQ in order to provide internet browsing.

4.2 Customer requirements and motivations

This section describes the customer’s needs and motivations regarding the new
network infrastructure based on SD-WAN. Specifically, the disadvantages of
the previous infrastructure, based on MPLS, and the requirements for the new
infrastructure based on SD-WAN will be investigated.

4.2.1 Motivation for changing the current network

infrastructure
Despite a good architectural solution, the previous system (AS-IS) has
numerous disadvantages. First, it is not capable of supporting an in-
depth analysis of traffic passing through VPN tunnels or in the MPLS
network. Second, the AS-IS infrastructure cannot support simultaneous
uploads through MPLS and the internet, and thus it does not allow for
bandwidth optimization. Third, the infrastructure on which the AS-IS is based
cannot be updated efficiently, since the quick access to the network topology
and its status is not supported. Finally, the AS-IS network infrastructure is
neither flexible nor scalable since a simple operation such as an addition or
removal of VLANs is heavily dependent on the ISP. For these reasons, the
customer has chosen to translate the network infrastructure from the AS-IS,
based on MPLS to one based on SD-WAN.

4.2.2 Customer requirements

One of the main requirements specified by the customer was the ability to
remotely control the entire infrastructure, which would allow for efficient
access to the status and the performance of the network. Furthermore, as
specified by the customer (c.f. Table 4.1), the new infrastructure was expected
to use SD-WAN technology, and support the features such as load balancing
and traffic shaping. Concerning the load balancing, there were no needs to
have more than two uplinks per site. However, the requirement was to support a
backup link through the use of mobile telephony (LTE / 5G), whose activation
48 | From MPLS based network to SD-WAN infrastructure

had to take place only in the case of failure of both primary links. Finally,
concerning the HA, it was required to implement architectural and technical
features capable of guaranteeing adequate network availability.

Table 4.1 – Customer requirements

Client needs Why those requirements are not

met by the existing solution
Remote control of the entire The current infrastructure does not
infrastructure have the ability to give a quick view
of the entire infrastructure to
monitor it
SD-WAN with load balancing and The current infrastructure has a
traffic shaping double uplink but is not used
simultaneously
LTE Backup The current infrastructure has not a
backup wireless uplink
Increasing HA The current infrastructure has not
redundant devices to mitigate
failures

Network infrastructure implementation with low service disruption

In addition to the above purely technical requests, the customer expressed the
need for a disruption-free transition from the current MPLS based to a new SD-
WAN based infrastructure. Fulfilling this requirement is particularly complex
but fundamental for companies that have multiple offices in different places.
The process of completing the migration of the entire infrastructure can take
days or weeks, and during this period it is necessary to ensure the functioning
of all the offices. In order to achieve this, one needs to consider both the
problem of static routes and MPLS routes.
The first problem refers to the fact that, both in HQ and in the peripheral
offices, the default gateways have to be changed. For example, as a part
of the network topology simplification, some static routes implemented in
the HQ need to be erased. The second problem concerns the MPLS routes
in the transitional period. More specifically, it is necessary to remove the
announcement of the internal networks of the migrated office, and to insert
a default gateway rule towards the HQ. In order to achieve this, the HQ needs
to take care of forwarding the traffic through a new VPN tunnel by using
the SD-WAN technology. The new rule allows a provisional communication
 From MPLS based network to SD-WAN infrastructure | 49

between the non-migrated office and the migrated office. Indeed, in the case of
communication between two branches (one already migrated and the other one
waiting for the migration), the flow needs to pass through the HQ to guarantee
the reachability. Therefore, the entire network infrastructure has a hub and
spoke topology with the HQ being a unique hub. Initially, the traffic passes
through the MPLS path from a non-migrated site to the HQ using the existing
interconnection. Subsequently, the traffic is sent to the SD-WAN concentrator
and then via the VPN tunnel from the HQ to the destination site that is already
being migrated. The exact operation will be explained in detail in Chapter
4.3.2.

4.3 Design of a new SD-WAN infrastructure

After the initial phase of analyzing the AS-IS and customer’s requests for the
new infrastructure, a phase of cost/benefit analysis of the various vendors
is performed. This phase is fundamental in identifying a solution that
satisfies the customer’s needs and, at the same time, allows savings in terms
of monetary cost and complexity of managing the network infrastructure.
After a careful analysis of the similar solutions offered by different vendors
(c.f., Chapter 2.4.5), the decision was to rely on Cisco Meraki for the new
network infrastructure. Indeed, Cisco Meraki offers numerous products for
the implementation of SD-WAN. Its strengths are the compactness of its
products, the simplicity of configuration and visualization of the network
status thanks to a complete and intuitive dashboard, and relatively low costs
compared to direct competitors. Meraki Dashboard is one of the strengths of
the Meraki solution. Beyond allowing a quick overview of the entire network
infrastructure, the dashboard permits the configuration of the appliances, the
firmware update, and the creation of stacks between two or more devices.
To summarize, Cisco Meraki makes traditionally complicated operations
intuitive and remotely configurable. The negative feature of Cisco Meraki
compared to other vendors is the constraint of using its simple and intuitive
dashboard. Indeed, to configure a device, an internet connection is needed to
access the dashboard to download the configuration. Moreover, the dashboard
does not permit configuring devices with particular configurations to solve
uncommon topologies such as hybrid topologies with multiple hubs connected
in cascade. Cisco Meraki does not support special functions like intelligent
cloud integration (Cisco Cloud OnRamp).
Cisco Meraki products are divided into different categories based on the
50 | From MPLS based network to SD-WAN infrastructure

functionality that the appliance must perform. There are switch products
(MS), network security & SD-WAN (MX) products, access points (MR),
wireless WAN (MG), and Internet of Things (IoT) products divided into
cameras (MV) and environmental sensors (MT). Each of the categories has
sub-products that differ in minor characteristics and computational power.

4.3.1 Network appliances

Based on the customer’s requests, the choice fell, in particular, on the MX84
model 1 for the HQ and MX67C 2 for the peripheral offices.

Table 4.2 – Comparison between client requirements and proposed solution

Client needs Meraki MX84 Meraki MX67C

SD-WAN and Yes Yes
Auto-VPN
Double WAN Yes Yes
LTE Backup Only with USB device Yes
Easy management Yes Yes
through Dashboard
Load Balancing Yes Yes
Segmentation and Yes Yes
Traffic shaping

Meraki MX67C
The MX67C model is classified by Cisco Meraki as specific for a small
branch. Indeed, it supports a limited number of users (up to 50) and four LAN
Ethernet ports. Despite this, the MX67C appliance fully satisfies the required
characteristics as it allows to establish, via auto-VPN, the VPN tunnels for SD-
WAN and to monitor the entire network and the devices connected to them via
the Meraki Dashboard.
Hardware and interfaces
The MX67C is characterized by a WAN port and four LAN Ethernet ports with
1 Gigabit Ethernet (GbE) capacity. However, when SD-WAN is performed, it
is possible to configure via software one of the LAN ports as an additional
1
https://meraki.cisco.com/product/security-sd-wan/
medium-branch/mx84/
2
https://meraki.cisco.com/product/security-sd-wan/
small-branch/mx67c/
 From MPLS based network to SD-WAN infrastructure | 51

uplink port. Moreover, there is a slot for the LTE cellular model and an
Universal Serial Bus (USB) 2.0 port for connecting an additional device in
the case of LTE failover.
Network
Besides SD-WAN typical features, such as load balancing, the MX67C
supports a few additional network features. Some of them are VLANs,
creating static routes, and also performing the Dynamic Host Configuration
Protocol (DHCP) server service for internal networks.
Security
From a security point of view, the MX67C performs firewall functions. In
particular, the MX67C allows traffic segmentation based on identity-based
policies, traffic analysis at the application level (layer 7), content filtering and
web searches. Moreover, an intrusion prevention system (IPS) and advanced
protection against malware are guaranteed based on Cisco Advanced Malware
Protection (AMP) technology.
Throughput
The MX67C provides a stateful firewall throughput of 450 Mbps and the
throughput of each VPN is equal to 200 Mbps. For LTE connectivity, the
link speed is around 300 Mbps.
Power supply
The MX appliance is a compact device designed for small branches. Its
electrical consumption is around 18 Watt, which is a considerably low value
when compared to other products that usually consume hundreds of watts
during their normal use.

Meraki MX84
The MX84 model is classified by the same manufacturer as specific for a
medium branch. It allows for a larger number of users (up to 200) and more
interfaces than the previous model (Meraki MX67C). The MX84 is positioned
in the HQ because it is the concentrator of all VPN tunnels (it is used the
hub and spoke topology) and because it acts as a gateway for internet access.
Therefore, the MX84 needs to have higher performance than the peripheral
sites. As for Meraki MX67C the VPN tunnels for SD-WAN are created though
auto-VPN and thus it is possible to monitor the entire network and the devices
connected to the network infrastructure through the Meraki Dashboard.
Hardware and interfaces
52 | From MPLS based network to SD-WAN infrastructure

The MX84 is characterized by two WAN ports and eight 1 GbE LAN ports.
The MX appliance is also equipped with two slots for small form-factor
pluggable transceiver (SFP) at 1 Gbps for fiber optic connections. There is no
slot for the LTE backup, as it is assumed that it is difficult to provide services
only with the LTE backup. Furthermore, the availability of WAN connectivity
is significantly greater in HQ than in the peripheral offices.
Network
Similar to the MX67C, the MX84 supports the features such as load balancing,
VLANs, static routing, and DHCP service for internal networks. In addition
to these features, the MX84 appliance supports application prioritization and
web caching, which can be used to speed up services for the end-users.
Security
The MX84 appliance supports the same set of security functions as MX67C,
therefore it is sufficient to refer to the characteristics already described above.
Throughput
In terms of throughput, the MX84 has better performance than the MX67C.
The MX84 supports a larger number of users, the throughput of the stateful
firewall of 500 Mbps, and the throughput of each VPN is equal to 250 Mbps.
Beyond this, the number of supported tunnels is equal to 100, instead of only
50 for the MX67C model, used in the peripheral offices.
Power supply
Although the whole MX series is particularly undemanding in terms of
electricity consumption, the MX84 has an ordinary consumption of 100W, an
index of greater computational power that allows it to manage a larger number
of clients, VPN tunnels and features.

Additional features and considerations

All Meraki devices are registered through the Meraki Dashboard. The
Dashboard allows the management and configuration of appliances, the
supervision of throughput and latency connectivity, and the centralized policy
management. It is effective for remote scheduling of firmware upgrades and
security patches, and performing real-time diagnostics in the case of any type
of critical issues. Finally, the use of the Meraki Dashboard allows to change,
even substantially, the behaviour of the devices and to activate/deactivate
features efficiently.
Cisco Meraki does not provide a default support for the additional features
 From MPLS based network to SD-WAN infrastructure | 53

such as the integration of IaaS and SaaS connectivity and the "split-tunnel"
functionality. The "split-tunnel" function allows direct forwarding of the traffic
to the local internet in the case of certain applications that are considered
safe and reliable (e.g., cloud or similar applications defined by the network
manager). These features can be obtained through additional licenses or, in
case of IaaS and SaaS connectivity, for some cloud service providers only.
However, the customer in the case of this project did not consider additional
features necessary, and thus Cisco Meraki was chosen as the most adequate
solution in terms of the supported features and monetary cost.
Another important aspect that needs to be considered when designing a SD-
WAN based network infrastructure is the HA. In this case, neither the MX67C
nor the MX84 allows power redundancy. However, for the HQ is required
more HA than a branch, so it was created a cluster with two MX84 to mitigate
a device failure. This is necessary as the HQ is the core in the hub and
spoke network infrastructure and, therefore, acts as an intermediary between
all the branches. Consequently, in the case of an MX84 failure, the branches
remain in communication with each other and with the HQ, thanks to the
secondary MX84 in active-active mode. Otherwise, for the secondary offices,
the creation of an MX cluster is not envisaged as the failure of the device
would only lead to the non-reachability of the secondary office, with minor
effects compared to the failure in the case of the HQ. In the event of a device
malfunction, the replacement of the device is guaranteed by Cisco Meraki in
a single working day. From the Meraki Dashboard it is possible to clone and
import the entire configuration file on the new device, and thus reconfiguring
a new MX for replacement is quick and easy.

4.3.2 SD-WAN Network Infrastructure

With the new devices, the entire network infrastructure changes, some
elements of the old infrastructure are being removed, and others added.
In particular, the network infrastructure was split into two different
infrastructures. The first one, referred to as intermediate infrastructure, was
used during the office migration phase and its main purpose was to support
the transition from the old to the new infrastructure. The second one, referred
to as final infrastructure, took the effect after the migration of all branches to
the Meraki infrastructure was completed.
54 | From MPLS based network to SD-WAN infrastructure

Intermediate infrastructure
Headquarter
The MX84s are installed in the HQ and a Demilitarized zone (DMZ) is created
on the firewall to allow the MX84s to communicate with the HQ’s internal
network and the internet. The first reason for allowing the reachability from
the HQ to the MX84 is the necessity to be able to forward the traffic coming
from the VPN tunnels towards the MPLS edge router for the reachability of the
offices not yet migrated. Moreover, the connectivity is necessary to provide
connectivity to the new offices and allow the registration and configuration of
MX devices to the Meraki Dashboard. Therefore, the MX84s have, as can be
seen in Figure 4.3 and Figure 4.4, one uplink connected to the network called
WAN II and one uplink connected to the edge router of the MPLS. Moreover,
each MX84 has a LAN port connected to the firewall’s DMZ. Furthermore,
being the two MX84 devices in the stack, they are provided with a virtual IP
and two physical IPs for each WAN port. On the LAN side, however, this
distinction is not necessary. In this phase, the routers named R2 and R3 (c.f.
Figure 4.3) are left in their position to guarantee backup reachability for the
other sites not yet migrated.

Figure 4.3 – Intermediate network infrastructure

 From MPLS based network to SD-WAN infrastructure | 55

Figure 4.4 – Intermediate network infrastructure with VPN

Branch
For the migration of each branch-side location, the router cluster that performs
the HSRP is replaced with an MX67C. The MX67C has the LAN port
connected towards the internal network and has the two WAN ports connected
to the MPLS edge router and the internet. The connectivity to the HQ and
other locations is guaranteed by the VPN tunnels established through the
auto-VPN. Therefore, the device monitoring through the Meraki Dashboard
is allowed through the HQ internet connection. However, for non-migrated
offices, there is no reachability with the migrated office because the LAN is no
longer advertised on the MPLS network. Therefore, it is necessary to forward
the traffic for the new offices through the HQ via R3 router. Subsequently, the
traffic is sent through static routes towards the firewall that further forwards it
on the DMZ relative to the MX84s. Finally, the MX84s encapsulate the traffic
in the VPN tunnel that reaches the newly migrated site. The setup described
permits reachability between heterogeneous sites from the migration point
of view to guarantee connectivity and productivity that would otherwise be
significantly affected.

Final infrastructure
At the end of the migration of all secondary offices, it is possible to eliminate
the part of the topology used in the intermediate phase, as illustrated in Figure
56 | From MPLS based network to SD-WAN infrastructure

4.5. Precisely, the static routes on the core that allowed traffic forwarding from
the MPLS circuit to the MX84s need to be eliminated. These static routes are
no longer necessary because all traffic to and from the offices is sent/received
by the MX84s, which act as a VPN concentrator for traffic between the offices.
The remaining traffic is forwarded by default in the direction of the firewall.
The firewall forwards the packets based on the IP to the internet port or to the
internal network. Furthermore, the routers R2 and R3 are removed because
they are not longer used.
Branch to headquarter
The traffic from the branch office to the HQ is routed from the MX67C, which
acts as a gateway. Based on the exposed VLAN, the MX67C decides whether
to forward the traffic through the VPN tunnel to the HQ. The traffic is decrypted
once it reaches the MX84. After the MX84 the traffic reaches the firewall via
the DMZ, and from there, it is directly accessible to the HQ.

Figure 4.5 – Final network infrastructure

Branch to internet
The internet traffic from each branch is conveyed to the HQ and routed from
there under the firewall supervision. The concentration of all internet flows in a
single point brings an advantage because there is only one "insecure" gateway
to the internet. The gateway is protected through a firewall that analyzes
all incoming and outgoing traffic for the entire organization. Indeed, in the
previous configuration, each site could go out on the internet locally, making
the surface exposed to attacks from the outside much greater.
 From MPLS based network to SD-WAN infrastructure | 57

Branch to branch
The traffic between two branches takes place in a similar way to that between a
branch and the HQ because the SD-WAN topology is hub-spoke, with the HQ
being the only hub. Therefore, the traffic passes from branch to branch through
MX84 in HQ. This approach allows increasing the choice of the best path
because there are two physical connections for each hub-spoke. Therefore,
the communication between two branches could travel on the internet from
the starting branch to the HQ and on the MPLS from the HQ to the arrival
branch. The path decision depends on the network congestion and the health
parameters of the connection. On the contrary, in the full-mesh approach,
it is not possible to change the physical connection because there is not an
intermediate hub. The result is that a single section affected by network
congestion involves the use of the MPLS connection, considered more reliable
in terms of QoS, with possible overload repercussions.

4.4 Implementation of key SD-WAN aspects

The implementation of the network infrastructure described above required a
connection to the internet of the appliances to enable the device registration
and then the configuration via the Meraki Dashboard. As the next step,
certain features were configured in order to support the appliance’s behaviour
specified during the design phase.
First, the MX84s in HQ were placed in HA mode by selecting the warm spare
command present on the dashboard. In this way, in addition to being able to
select the primary device via software, it is possible to manage two appliances
as a single logical unit to activate particular settings and configurations.
Moreover, for the WAN interfaces the VIP functionality was used. The
VIP functionality allows to expose a single virtual public IP and assign an
additional IP to each appliance. In this way, the internet traffic is always
sent to the virtual IP and then forwarded to the appliance with the active state
status. Hence, in the event of an appliance failure, the destination IP will not
change and, therefore, even the VPN tunnels will not have to be recreated.
To summarize, the VIP functionality eliminates the disruption due to the re-
creation of the VPN tunnels.
The HQ physical LAN connections require a particular topology to satisfy
the Meraki best practices in terms of HA [63]. Indeed the two MX84s LAN
connections are terminated to the switch core in trunk mode with the objective
of forwarding the traffic to the firewalls. The resulting configuration is
58 | From MPLS based network to SD-WAN infrastructure

Figure 4.6 – Meraki recommended design - switch stack

illustrated in Figure 4.6. The cross-connection provides a better HA guarantee

than a single connection between MX and switch core. For example, different
from a single connection, the cross-connection allows having HA in situations
such as failure of an MX appliance, a switch belonging to the core stack, or a
LAN connection between the MX and the switch core.

Figure 4.7 – SD-WAN topology settings

 From MPLS based network to SD-WAN infrastructure | 59

The last step of this particular configuration is to set a static route that forwards
all the traffic to the firewall. The firewall selects on which interface (internal
or external) to forward the traffic. The static route is necessary as the MX84s
are directly connected to the internet to establish the VPN tunnels with the
other offices. Therefore, in the case of internet traffic, the MX would attempt
to send the traffic to the WAN port without going through the firewall.
For each appliance the VLANs and the public IP are set based on the
configuration on the previous router. Figure 4.7 illustrates the configuration
needed for establishing the VPN tunnels for the SD-WAN. As shown in the
figure, one needs to set: the role of the appliance (hub or spoke), the hubs
to refer to, and the subnets exposed to the other branches (in this particular
case, there is only one subnet). At the end of these operations, the VPN
tunnels are established, and the overlay topology is created. Subsequently,
the configurations relating to the SD-WAN features are set, and these features
are general for all the sites of the analyzed topology.

Load Balancing
The load balancing involves the use of two internet links with the addition of
the cellular connection in case of failure of both WAN links. As illustrated
in Figure 4.8a, each link has a bandwidth limit that depends on the link
characteristics, and the WAN I link is enabled as primary.
SD-WAN allows for creating policies for splitting the traffic between the two
links. In this case, as illustrated in Figure 4.8b, the VoIP traffic migrates on
WAN II if the performance of the network is too degraded. In particular, the
link change is performed if any of the requirements for the latency, jitter, or
loss parameters are not satisfied. Otherwise, if the QoE is satisfactory, the
primary link should be used.
60 | From MPLS based network to SD-WAN infrastructure

(a) Uplinks settings

(b) SD-WAN policies

Figure 4.8 – Load balancing settings

Traffic shaping
The traffic shaping rules are used to allow for better management of the
available bandwidth resources and to improve the QoE for essential services.
In particular, two rules are created, to improve the QoE of users (c.f., Figure
4.9).
 From MPLS based network to SD-WAN infrastructure | 61

Figure 4.9 – Traffic shaping settings

The first rule allows guaranteeing unrestricted bandwidth for VoIP traffic.
Owning to this rule, it is possible to have a separate queue for the VoIP traffic,
and thus to provide a better VoIP service in terms of latency. The second rule
concerns online backups and allows for a maximum bandwidth of 2 Mbps for
each user. Indeed, a backup is not an essential service, and therefore the file
fetching can last more time but with less instant impact on bandwidth.
62 | From MPLS based network to SD-WAN infrastructure
 Results and Analysis | 63

Chapter 5

Results and Analysis

The following chapter presents the results obtained from tests carried out on
the SD-WAN infrastructure in a hub and spoke topology. The tests mainly
focus on the features of load balancing, traffic shaping, and certain failure
situations that can be mitigated thanks to the HA strategy.

5.1 High Availability tests

In the network infrastructure, failures are intentionally created to perform the
HA tests and verify if the expected behaviour is the same as the one obtained
through the tests.

5.1.1 Link failure

The first test that was performed with the objective of verifying the resilience
of the created network infrastructure is the link failure test. The test was
based on evaluating the switching time between the primary and the secondary
link for three different setups referred to as routers in HSRP, MX without
Active-Active VPN and MX with Active-Active VPN, respectively. The routers
in HSRP setup corresponds to the previous infrastructure with two routers
configured with the HSRP. On the contrary, the MX without Active-Active VPN
and MX with Active-Active VPN setups can be used with the MX67C appliance
with the difference that the MX with Active-Active VPN setup supports the
active-active functionality. This functionality allows creating VPN tunnels in
advance, even on the link that is not being used, and thus it allows for reducing
the link switching times as we will see in the following.
64 | Results and Analysis

20

Time (seconds) 15

10

0
Failure Recover

Routers in HSRP MX without Active-Active AutoVPN

MX Active-Active AutoVPN

Figure 5.1 – Link failure times

The above setups were evaluated in terms of the switching time between the
primary and the secondary link, and the time needed to recover the primary
connectivity, respectively. The corresponding results are shown in Figure 5.1.
The results suggest that the routers in HSRP setup has the worst performance
with the switching time and the time needed to recover the primary link
equal to approximately 20s and 18s, respectively. This is mainly because
multiple actions need to be performed upon the failure. For example, during
the switching time the fault is identified based on the HSRP timer, the traffic
control is migrated from the primary to the secondary router, and the new VPN
tunnel on the secondary link is established.
The use of a single MX for managing both links halves the detection times and
the time needed for the new VPN tunnel creation, which results in a switching
time that is approximately equal to 9s in the case of the MX without Active-
Active VPN setup. Furthermore, the time to recover the primary link is null
because the appliance first recreates the tunnel and then migrates the data flow.
Finally, in the case of the MX with Active-Active VPN setup, the results show
that both the switching time and the time needed to recover the primary link
are equal to zero. This is because a secondary VPN tunnel on the secondary
uplink already exists at the time of the failure. Therefore, one can conclude
that the MX with Active-Active VPN setup is resilient to the link failures, and
 Results and Analysis | 65

thus that it has the potential to provide a high QoE.

2.2 WAN I
2 WAN II
Cellular
1.8
1.6
Bandwidth (Mbps)

1.4
1.2
1
0.8
0.6
0.4
0.2

2 6 10 14 18 22 26 30 34 38 42 46 50 54 58

Time (seconds)

Figure 5.2 – Internet traffic analysis in the link failure scenario

Furthermore, the link failure was evaluated through a bandwidth analysis of

the MX67C appliance. The MX67C device allows the use of two uplinks and
an LTE backup. Figure 5.2 shows the bandwidth usage on the primary WAN
I, secondary WAN II and LTE uplinks, respectively. As illustrated in Figure
5.2, both primary and secondary uplinks were initially used in load balance
mode. When the failure occurred and the primary uplink WAN I became
disconnected, the traffic was instantaneously migrated to the secondary uplink.
This rapidity of migration was possible because both VPN tunnels were
already established, thanks to the setting MX with Active-Active VPN, and
hence it was not needed to wait for the creation of the VPN tunnel. However,
the same behaviour could be observed with the setting MX without Active-
Active VPN because the load balancing already implies the creation of VPN
tunnels on both uplinks. In the case of a failure of the secondary uplink, the
traffic stops for about 10s. This happens because the cellular uplink is a backup
that is activated only in case of failure of both primary and secondary uplinks.
Hence, in this case it is needed to wait for approximately 10s while the VPN
66 | Results and Analysis

tunnels are being created (c.f., Figure 5.1).

5.1.2 Appliance failure

Another test related to HA is the use of the appliances in warm spare mode.
In this scenario, the provision of service is still guaranteed in case of a power
failure or hardware failure event. The infrastructure with the MX84 appliances
was used only in the case of the HQ since the HQ is considered as a more
important location than the peripheral offices. This test was performed with
two primary objectives. The first objective was to evaluate how the system
works in warm spare mode and the second objective was to evaluate how much
the VIP functionality affects the switch times of the appliances.

80
Time (seconds)

60

40

20

0
primary-secondarysecondary-primary

VIP disable VIP enable

Figure 5.3 – Appliance failure times

Figure 5.3 shows the switching times between the primary and secondary
appliances together with the switching times between secondary and primary
appliances in the case when the VIP functionality is both disabled and enabled.
The results show that, when the VIP functionality is disabled and failure of the
primary appliance occurs, it is needed to wait for approximately 90s before
the secondary appliance becomes operational. On the contrary, when the VIP
functionality is enabled the switching time is reduced by approximately 91%.
This is because in the case with disabled VIP functionality, the spare appliance
uses an IP other than the primary. This means that the IP pointed by the branch
appliances is a virtual one, and hence changing the routes and recreating VPN
 Results and Analysis | 67

tunnels is time-consuming. On the contrary, in the case with enabled VIP, the
traffic is routed on the same IP, which reduces the failover time.
The results also show that the switching times from secondary to primary
appliance are smaller than the switching times from primary to secondary
appliance in both cases because the primary appliance is preemptive, therefore,
when the primary appliance returns active, it informs the secondary appliance
and takes its role back. Finally, based on the above observations, one can
conclude that the VIP functionality has the potential to improve the availability
of the network infrastructure, and thus to reduce the disruption of the services.

5.2 Load balancing tests

The effect of load balancing can be evaluated by considering the bandwidth
usage on the primary and the secondary links. Figure 5.4 shows the bandwidth
usage for two uplinks and the setup illustrated in Figure 4.8a.

1.4 WAN I
WAN II
1.2

1
Bandwidth (Mbps)

0.8

0.6

0.4

0.2

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

Time (seconds)

Figure 5.4 – Uplink bandwidth consumption analysis with load balancing

According to the Meraki documentation, the flows are split between the WAN
68 | Results and Analysis

uplinks according to the round-robin strategy [64]. Therefore, the algorithm

starts with the uplink marked as primary. Since in the case of the downlink,
the available bandwidth on the WAN I link is five times greater than on the
WAN II link (c.f., Figure 4.8a), the Meraki load balancing algorithm initiates
the five flows on the WAN I link and one flow on the WAN II link. The
corresponding bandwidth consumption is shown in Figure 5.4 for WAN I
and WAN II links, respectively and can be observed that the proportions of
bandwidth consumption of the two connectivity are respected.

200
70 180

60 160

140
Bandwidth (Mbps)

Bandwidth (Kbps)
50
120
40
100

30 80

60
20
40
10 WAN I
20
WAN II
0
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

Time (seconds)

Figure 5.5 – Internet traffic analysis with SD-WAN policies - bandwidth usage

Next, the traffic shaping rule was evaluated for the settings shown in Figure 4.9.
The traffic shaping rule enables the routing of VoIP traffic on the primary link
as long as the latency, jitter and loss parameters remain in the predetermined
range. In order to validate the functioning of the traffic shaping rule, the
test was performed for a high amount of background traffic on the WAN I
uplink. The background traffic was started prior to the VoIP traffic and the
corresponding bandwidth usage and the latency are shown in Figure 5.5 and
Figure 5.6, respectively. Figure 5.5 shows that the bandwidth consumption on
 Results and Analysis | 69

the WAN II link increases from 0 to approximately 180 Kbps after introducing
the VoIP traffic. As expected, this happens because the VoIP traffic was routed
to the WAN II link due to high latency caused by the congestion on the WAN
I link. The increase of the latency on the WAN I link is shown in Figure 5.6
as result of the background traffic on WAN I link.

20
18
16
Latency (milliseconds)

14
12
10
8
6
4
WAN I
2
WAN II
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

Time (seconds)

Figure 5.6 – Internet traffic analysis with SD-WAN policies - latency values

5.3 Traffic shaping test

Finally, the last feature that was analysed is traffic shaping. The Meraki
appliance segments the internet traffic at the application level. This is
illustrated in Figure 5.7, which shows the main applications listed in the
decreasing order of the bandwidth usage. Owning to the functionality that
allows the traffic segmentation at the application level, the Meraki allows for
creating rules based on the traffic type (c.f. Figure 4.9). These rules can, for
example, limit the use of the bandwidth of a particular service or ensure a
certain QoS. In the tested case, the traffic was generated by a backup software,
which usually utilizes a significant amount of bandwidth that can preclude
70 | Results and Analysis

the QoE of the other services. In order to keep the amount of traffic at the
reasonable level, the limit of 2Mbps was imposed on the traffic generated
by a backup software, This limit is shown in Figure 5.8 together with the
bandwidth consumption in the case of backup traffic, bandwidth consumption
in the case of the other type of traffic referred to as mixed internet traffic,
and the total bandwidth consumption, respectively. The figure shows that the
total bandwidth consumption increases due to the start of the backup software.
However, the limit imposed for the backup traffic is respected on average.
Consequently, the internet traffic does not increase uncontrollably, and thus
one can expect that the traffic shaping feature can be used for reducing the
occurrence of the slowdowns and the service interruptions.

Figure 5.7 – Internet traffic segmentation at the application level

 Results and Analysis | 71

3.5
Total internet traffic
Mixed internet traffic
3 Backup
Limit imposed
2.5
Bandwidth (Mbps)

1.5

0.5

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40

Time (seconds)

Figure 5.8 – Internet traffic analysis with traffic shaping

72 | Results and Analysis
 Discussion | 73

Chapter 6

Discussion

The transition from MPLS based to SD-WAN infrastructure involves multiple

challenges. Initially, it was necessary to identify a new design for the network
infrastructure based on the existing solution and, based on the customer’s
requirements, the characteristics and the monetary cost, identify the most
suitable appliances. Moreover, this thesis addresses challenges related to
the correct implementation of routes and IP addresses for the new SD-WAN
based network and the coexistence between the two infrastructures during the
migration. The coexistence of the two infrastructures was possible thanks to
the correct design and performed as described in the thesis. Additionally, one
of the main requirements was to perform the migration with the minimum
disturbance of the company’s operations. The migration of each site was
performed efficiently with a downtime of less than thirty minutes. In order
to achieve this, most of the functionalities were implemented after verifying
the connectivity between the newly migrated office and the other offices,
and subsequent tests were performed in order to verify the efficiency and
correctness of the implementation.
The HA tests made it possible to verify the operation of the backup
links and devices set in warm spare mode. The uplinks and appliances
redundancy generates higher reliability of the infrastructure, and in the event
of a malfunction, the network administrator is immediately alerted by the
dashboard. Furthermore, the conducted tests showed that the transition from
one link to another is significantly faster than in the case of the methods used
for MPLS based infrastructure.
The load balance tests show, based on the uplink bandwidth parameter and the
74 | Discussion

downlink bandwidth parameter, how the traffic is balanced according to the

implemented algorithms. The load balance functionality allows having the
same percentage of use of both links. Moreover, SD-WAN allows for creating
policies to transmit certain types of traffic on a specific uplink, and in the case
of low performance it allows for migrating the internet traffic to another uplink.
Finally, the segmentation at the application level is carried out by Meraki
appliances. The appliance allows applying traffic shaping rules, as shown
in the example in Chapter 4.4. This example shows the implementation of
the traffic shaping rule for the internet traffic produced by online backups.
Additionally, it is possible to create rules for multiple applications, even
specifying a single domain to discriminate. The network manager can choose
to apply a bandwidth limit for every single user and not only globally. This
feature is useful in the case of different types of users connected to the
same network, and thus this functionality is very often used in SD-Access
environments.
 Conclusions and Future work | 75

Chapter 7

Conclusions and Future work

The following chapter aims at drawing conclusions related to the entire

thesis. In addition to discussing the outcomes, possible future works will
be formulated. Finally, a reflection will be held on the issues of ethics and
sustainability related to SD-WAN.

7.1 Conclusions
This thesis summarizes the main characteristics of the SD-WAN technology
and provides a detailed description of the process carried out during the
migration from an MPLS based infrastructure to the SD-WAN infrastructure.
In particular, the thesis describes the steps that need to be followed in order to
perform the migration in a way that allows for the proper network functioning
during the transition period. Finally, the thesis presents the results concerning
the impact of some of the main SD-WAN features on the overall network
performance.
In the first part of the thesis, the focus was on the analysis of the SD-WAN
technology and its main features such as load balancing, traffic shaping,
cloud integration and HA. Furthermore, the first part of the thesis provides
a summary of the various solutions proposed by the main vendors in the SD-
WAN sector.
In the second part of the thesis, the focus was on the process of migrating an
existing MPLS based infrastructure to an SD-WAN infrastructure using Cisco
Meraki. The implementation of the new infrastructure required numerous
phases for its realization. Each of them required in-depth knowledge of the
76 | Conclusions and Future work

SD-WAN features implemented in the chosen product, the previous state of

the network infrastructure and the customer’s requirements.
The correct functioning of the new network infrastructure was verified
by checking the reachability between the various branches located in the
various company offices. After validating that the new network infrastructure
functions in an expected way, some of the main SD-WAN features were
evaluated. The examples of these features are the use of appliances in
HA mode or multiple links in the case of an uplink failure, traffic shaping
and load balancing. The results showed that thanks to these features, the
new infrastructure outperforms the old one in terms of the HA, bandwidth
utilization and the overall network efficiency.
From the point of view of monetary costs, the immediate cost reduction was
not observed and this is because the pre-existing internet connections were
maintained. Despite this aspect, it was shown that the SD-WAN infrastructure
improves the performances and, therefore, it is easy to hypothesize the
possibility of divesting the MPLS network in favour of a less reliable but
less expensive internet connection. Moreover, the greater HA reduces the
downtime, which allows having fewer disruptions for the end-user. Finally,
owing to the possibility of remote maintenance and reconfigurations, the SD-
WAN infrastructure has the potential to provide significant savings and faster
interventions in the case of failure.
Finally, the conclusion is that the implemented SD-WAN infrastructure
functions correctly and supports network management features that can
provide considerable advantages both for the user and the network manager.

7.2 Future work

There are several aspects that can be interesting topics for future work.
The first aspect is related to the implementation of the SD-WAN infrastructure
in a full-mesh configuration. This would allow investigating if there are real
advantages in terms of appliance performance or traffic produced by a single
appliance and by the entire network. The full-mesh analysis is not complicated
because the SD-WAN network is an overlay network, and thus it is enough to
change the appliance settings without making any physical changes.
The second aspect concerns the comparison of Cisco Meraki with other
vendors (e.g., Cisco Viptela or Fortinet) in terms of implementation
 Conclusions and Future work | 77

complexity, monetary cost, network security, and other relevant network

performance metrics. Moreover, the procedures for creating the SD-WAN
network are also different, and, therefore, it would be interesting to investigate
which vendor has the fastest and most intuitive procedure to create the the
SD-WAN network equivalent to Cisco Meraki.
The third aspect that would be interesting to investigate is the use and
the implementation of cloud integration. A possible IaaS and SaaS
implementation would permit to test the cloud integration functionality
combined with SD-WAN. In addition to testing how the cloud integration
works, the analysis of savings in terms of resources and infrastructure
optimization could be especially interesting considering the increasing
demand for cloud services.

7.3 Reflections
This thesis shows that the SD-WAN technology allows a substantial
improvement of the network infrastructures. Improvements include security,
HA, and easier integration of the new offices into the existing network.
SD-WAN solutions considered in this thesis also have a great value in terms
of environmental sustainability. For example, the SD-WAN technology allows
for the remote establishment and maintenance of the overlay network, and
thus it reduces the physical interventions by network infrastructure managers.
Since the physical interventions are usually associated with diverse transport
commuters, the SD-WAN technology has the potential to reduce pollutant
emissions.
Regarding the ethical aspects, SD-WAN technology supports reliable
transmission of sensitive information over insecure internet connections.
The reliability is due to the fact that VPN tunnels are used to make the
connection encrypted. Furthermore, the advantage of SD-WAN is the
possibility of creating these VPN tunnels automatically, eliminating the
problems of exchanging cryptographic material. Finally, most of the SD-WAN
infrastructure makes fundamental services for users less prone to malfunctions
and inefficiencies created by ISPs and preserves the rights to make the network
services accessible to all users.
78 | Conclusions and Future work
 REFERENCES | 79

References

[1] R. R. Kasturi, “Trends in sd-wan and sdn,” [Online] https://link.springer.

com/article/10.1007%2Fs40012-020-00277-5, [Accessed: 2021-02-
10].
[2] Cisco, “Cisco visual networking index (vni) complete forecast
update, 2017–2022,” [Online] https://www.cisco.com/c/dam/m/
en_us/network-intelligence/service-provider/digital-transformation/
knowledge-network-webinars/pdfs/1213-business-services-ckn.pdf,
[Accessed: 2021-03-18].
[3] Cisco, “Cisco sd-wan,” [Online] https://www.cisco.com/c/dam/global/
da_dk/assets/pdfs/cisco_virtual_update_cisco_sdwan_viptela.pdf,
[Accessed: 2021-08-25].
[4] E. Rosen, A. Viswanathan, and R. Callon, “Rfc3031: Multiprotocol label
switching architecture,” USA, 2001.
[5] Vaibhav Gupta, “Prepare for a hyper-connected world with
5g and sd-wan,” [Online] https://www.hcltech.com/blogs/
prepare-for-a-hyper-connected-world-with-5G-and-sd-wan, [Accessed:
2021-08-25].
[6] VMWare, “Green sd-wan,” [Online] https://sdwan.vmware.com/
sd-wan/understanding-sd-wan/green-sd-wan, [Accessed: 2021-03-08].
[7] V. Untz, M. Heusse, F. Rousseau, and A. Duda, “Lilith : an
interconnection architecture based on label switching for spontaneous
edge networks,” 09 2004. doi: 10.1109/MOBIQ.2004.1331721. ISBN
0-7695-2208-4 pp. 146 – 151.
80 | REFERENCES

[8] G. Swallow, “Mpls advantages for traffic engineering,” IEEE

Communications Magazine, vol. 37, no. 12, pp. 54–57, 1999. doi:
10.1109/35.809385
[9] C. Luciani, “From mpls to sd-wan: Opportunities, limitations and best
practices,” June 2019. [Online]. Available: https://www.diva-portal.org/
smash/get/diva2:1417117/FULLTEXT01.pdf
[10] Networkworld, “Why does mpls cost so much more than internet
connectivity?” [Online] http://chipps.com/4/Gottlieb.pdf, [Accessed:
2021-02-04].
[11] O. Al-Saadeh, G. Wikstrom, J. Sachs, I. Thibault, and D. Lister, “End-
to-end latency and reliability performance of 5g in london,” in 2018
IEEE Global Communications Conference (GLOBECOM), 2018. doi:
10.1109/GLOCOM.2018.8647379 pp. 1–6.
[12] A. Narayanan, E. Ramadan, J. Carpenter, Q. Liu, Y. Liu, F. Qian,
and Z.-L. Zhang, “A first look at commercial 5g performance on
smartphones,” in Proceedings of The Web Conference 2020, ser. WWW
’20. New York, NY, USA: Association for Computing Machinery, 2020.
doi: 10.1145/3366423.3380169. ISBN 9781450370233 p. 894–905.
[Online]. Available: https://doi.org/10.1145/3366423.3380169
[13] K. Benzekki, A. El Fergougui, and A. Elbelrhiti Elalaoui,
“Software-defined networking (sdn): a survey,” Security and
Communication Networks, vol. 9, no. 18, pp. 5803–5833,
2016. doi: https://doi.org/10.1002/sec.1737. [Online]. Available:
https://onlinelibrary.wiley.com/doi/abs/10.1002/sec.1737
[14] F. Hu, Q. Hao, and K. Bao, “A survey on software-defined network and
openflow: From concept to implementation,” IEEE Communications
Surveys Tutorials, vol. 16, no. 4, pp. 2181–2206, 2014. doi:
10.1109/COMST.2014.2326417
[15] Z. Yang, Y. Cui, B. Li, Y. Liu, and Y. Xu, “Software-defined wide
area network (sd-wan): Architecture, advances and opportunities,” in
2019 28th International Conference on Computer Communication and
Networks (ICCCN), 2019. doi: 10.1109/ICCCN.2019.8847124 pp. 1–9.
[16] M. Banikazemi, D. Olshefski, A. Shaikh, J. Tracey, and G. Wang,
“Meridian: an sdn platform for cloud network services,” IEEE
 REFERENCES | 81

Communications Magazine, vol. 51, no. 2, pp. 120–127, 2013. doi:

10.1109/MCOM.2013.6461196
[17] B. Agborubere and E. Sanchez, in OpenFlow Communications and TLS
Security in Software-Defined Networks, 06 2017. doi: 10.1109/iThings-
GreenCom-CPSCom-SmartData.2017.88 pp. 560–566.
[18] W. Stallings, Network Security Essentials: Applications and Standards,
Fifth Edition. Pearson, 2014. ISBN 9780133370430
[19] J. Paillisse, M. Portoles, A. Lopez, A. Rodriguez-Natal, D. Iacobacci,
J. Leong, V. Moreno, A. Cabellos, F. Maino, and S. Hooda, SD-
Access: Practical Experiences in Designing and Deploying Software
Defined Enterprise Networks. New York, NY, USA: Association
for Computing Machinery, 2020, p. 496–508. ISBN 9781450379489.
[Online]. Available: https://doi.org/10.1145/3386367.3431288
[20] Cisco, “Sd-access segmentation design guide,” [Online]
https://www.cisco.com/c/dam/en/us/td/docs/solutions/CVD/Campus/
CVD-Software-Defined-Access-Segmentation-Design-Guide-2018MAY.
pdf, [Accessed: 2021-02-05].
[21] MEF, “Understanding sd-wan managed services,” [Online] https://www.
google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=
2ahUKEwjQ6YqcgNXuAhUH-6QKHUPiCUsQFjABegQIChAC&
url=https%3A%2F%2Firp-cdn.multiscreensite.com%2F55714467%
2Ffiles%2Fuploaded%2FUnderstanding_SD-WAN_Managed_
Services_MEF.pdf&usg=AOvVaw3Aj5ZikV6aGDMb45BQ7Wmi,
[Accessed: 2021-02-06].
[22] G. Mine, J. Hai, L. Jin, and Z. Huiying, “A design of sd-wan-
oriented wide area network access,” in 2020 International Conference
on Computer Communication and Network Security (CCNS), 2020. doi:
10.1109/CCNS50731.2020.00046 pp. 174–177.
[23] M. S. Dayananda and A. Kumar, “Architecture for inter-cloud
services using ipsec vpn,” in 2012 Second International Conference
on Advanced Computing Communication Technologies, 2012. doi:
10.1109/ACCT.2012.32 pp. 463–467.
[24] D. Katz and D. Ward, “Rfc5880: Bidirectional forwarding detection
(bfd),” USA, 2010.
82 | REFERENCES

[25] N. L. M. Van Adrichem, B. J. Van Asten, and F. A. Kuipers,

“Fast recovery in software-defined networks,” in 2014 Third
European Workshop on Software Defined Networks, 2014. doi:
10.1109/EWSDN.2014.13 pp. 61–66.
[26] A. Yassin and F. Yalcin, “Enterprise transition to software-defined
networking in a wide area network,” 2019. [Online]. Available: http:
//www.diva-portal.org/smash/get/diva2:1322911/FULLTEXT01.pdf
[27] Juniper Networks, “Introduction to autovpn implementing autovpn
network design using the srx series with ibgp as the dynamic routing
protocol,” [Online] https://www.juniper.net/us/en/local/pdf/app-notes/
3500214-en.pdf, [Accessed: 2021-02-12].
[28] Cisco Meraki, “Cisco meraki auto vpn,” [Online] https://meraki.cisco.
com/lib/pdf/meraki_whitepaper_autovpn.pdf, [Accessed: 2021-02-16].
[29] B. Ford, P. Srisuresh, and D. Kegel, “Peer-to-peer communication across
network address translators,” 04 2006.
[30] Z. Yang, Y. Cui, B. Li, Y. Liu, and Y. Xu, “Software-defined wide
area network (sd-wan): Architecture, advances and opportunities,” in
2019 28th International Conference on Computer Communication and
Networks (ICCCN), 2019. doi: 10.1109/ICCCN.2019.8847124 pp. 1–9.
[31] S. Gordeychik and D. Kolegov, “Sd-wan threat landscape,” ArXiv, vol.
abs/1811.04583, 11 2018.
[32] Cisco, “Cisco sd-wan,” [Online] https://www.cisco.com/c/en_uk/
solutions/enterprise-networks/sd-wan/index.html, [Accessed: 2021-03-
07].
[33] VMware, “Vmware sd-wan,” [Online] https://www.vmware.com/
products/sd-wan.html, [Accessed: 2021-03-07].
[34] Citrix, “Citrix sd-wan,” [Online] https://support.citrix.com/plp/
products/citrix_sd_wan_netscalar_sd_wan/tabs/popular-solutions#,
[Accessed: 2021-03-07].
[35] Cisco Meraki, “Cloud managed network security i& sd-wan,” [Online]
https://meraki.cisco.com/products/security-sd-wan/, [Accessed: 2021-
03-07].
[36] Fortinet, “Sd-wan definition and solutions,” [Online] https://www.
fortinet.com/products/sd-wan, [Accessed: 2021-03-07].
 REFERENCES | 83

[37] Cisco, “Sd-wan vendors comparison chart,” [Online] https:

//www.cisco.com/c/dam/en/us/products/collateral/routers/
competitive-comparison-chart.pdf, [Accessed: 2021-02-08].
[38] NetManias, “Comparison of the sd-wan vendor solutions,” [Online]
https://www.netmanias.com/en/post/oneshot/12481/sd-wan-sdn-nfv/
comparison-of-the-sd-wan-vendor-solutions, [Accessed: 2021-02-08].
[39] M. B. Lehocine and M. Batouche, “Flexibility of managing vlan filtering
and segmentation in sdn networks,” in 2017 International Symposium
on Networks, Computers and Communications (ISNCC), 2017. doi:
10.1109/ISNCC.2017.8071999 pp. 1–6.
[40] W. Odom, CCNA 200-301 Official Cert Guide, Volume 1, ser. Official
Cert Guide. Pearson Education, 2019. ISBN 9780135768471. [Online].
Available: https://books.google.it/books?id=LyCtDwAAQBAJ
[41] A. Mehdizadeh, K. Suinggi, M. Mohammadpoor, and H. Harun, “Virtual
local area network (vlan): Segmentation and security,” in Proceedings
of the Third International Conference on Computing Technology and
Information Management, ser. ICCTIM2017. Research Gate, 2017.
ISBN 978-1-941968-45-1 pp. 78–89.
[42] P. Samana, “Data breach a cyber security issue in cloud,”
08 2019. [Online]. Available: https://www.researchgate.net/
profile/Samana-Paudel/publication/335243297_Data_Breach_a_
Cyber_Security_Issue_in_Cloud/links/5d5aefba458515210252202f/
Data-Breach-a-Cyber-Security-Issue-in-Cloud.pdf
[43] Cisco Meraki, “Layer 7 visibility and control,” [Online] https://meraki.
cisco.com/product-collateral/layer-7-visibility-and-control-whitepaper/
?file, [Accessed: 2021-02-12].
[44] National Institute of Standards and Technology, “The nist definition
of cloud computing,” [Online] https://nvlpubs.nist.gov/nistpubs/Legacy/
SP/nistspecialpublication800-145.pdf, [Accessed: 2021-02-08].
[45] Cisco, “Cloud onramp for microsoft 365 white paper,” [Online] https:
//www.cisco.com/c/en/us/solutions/collateral/enterprise-networks/
sd-wan/white_paper-c11-741353.html, [Accessed: 2021-02-08].
[46] Cisco, “Policies configuration guide for vedgerouters, ciscosd-wan
releases 19.1,19.2,and 19.3,” [Online] https://www.cisco.com/c/en/us/
84 | REFERENCES

td/docs/routers/sdwan/configuration/policies/vedge/policies-book.pdf,
[Accessed: 2021-03-06].
[47] S. Rajagopalan, “An overview of sd-wan load balancing for wan
connections,” in 2020 4th International Conference on Electronics,
Communication and Aerospace Technology (ICECA), 2020. doi:
10.1109/ICECA49313.2020.9297574 pp. 1–4.
[48] S. Troia, F. Sapienza, L. Varé, and G. Maier, “On deep
reinforcement learning for traffic engineering in sd-wan,” IEEE
Journal on Selected Areas in Communications, pp. 1–1, 2020. doi:
10.1109/JSAC.2020.3041385
[49] T. Semong, T. Maupong, S. Anokye, K. Kehulakae, S. Dimakatso,
G. Boipelo, and S. Sarefo, “Intelligent load balancing techniques
in software defined networks: A survey,” Electronics, vol. 9,
no. 7, 2020. doi: 10.3390/electronics9071091. [Online]. Available:
https://www.mdpi.com/2079-9292/9/7/1091
[50] E. Vollset, “Maelstrom: Transparent error correction for lambda
networks,” in 5th USENIX Symposium on Networked Systems Design and
Implementation (NSDI 08). San Francisco, CA: USENIX Association,
Apr. 2008. [Online]. Available: https://www.usenix.org/conference/
nsdi-08/maelstrom-transparent-error-correction-lambda-networks
[51] D. Radcliffe, E. Furey, and J. Blue, “An sd-wan solution
assuring business quality voip communication for home based
employees,” in 2019 International Conference on Smart Applications,
Communications and Networking (SmartNets), 2019. doi:
10.1109/SmartNets48225.2019.9069755 pp. 1–6.
[52] P. C. Fonseca and E. S. Mota, “A survey on fault management in software-
defined networks,” IEEE Communications Surveys Tutorials, vol. 19,
no. 4, pp. 2284–2321, 2017. doi: 10.1109/COMST.2017.2719862
[53] F. Botelho, A. Bessani, F. M. V. Ramos, and P. Ferreira, “On
the design of practical fault-tolerant sdn controllers,” in 2014 Third
European Workshop on Software Defined Networks, 2014. doi:
10.1109/EWSDN.2014.25 pp. 73–78.
[54] P. Fonseca, R. Bennesby, E. Mota, and A. Passito, “A replication
component for resilient openflow-based networking,” in 2012
 REFERENCES | 85

IEEE Network Operations and Management Symposium, 2012.

doi: 10.1109/NOMS.2012.6212011 pp. 933–939.
[55] Cisco, “Stacking on cisco catalyst 2960 family
switches,” [Online] https://www.google.com/url?
sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=
2ahUKEwjT4MrEoZzvAhXBAmMBHeyaADQQFjAMegQIBxAD&
url=https%3A%2F%2Fwww.data-holic.com%2Fuploads%2Fproducts_
files%2Fdata_holic_5747049_2019-04-04-09-43-09.pdf&usg=
AOvVaw0PvfSSWJV7btmlF1HYrdtz, [Accessed: 2021-03-06].
[56] I. Irawati, S. Hadiyoso, and Y. Hariyani, “Link aggregation control
protocol on software defined network,” International Journal of
Electrical and Computer Engineering (IJECE), vol. 7, p. 2706, 10 2017.
doi: 10.11591/ijece.v7i5.pp2706-2712
[57] J. Zheng, H. Xu, X. Zhu, G. Chen, and Y. Geng, “Sentinel:
Failure recovery in centralized traffic engineering,” IEEE/ACM
Transactions on Networking, vol. 27, no. 5, pp. 1859–1872, 2019. doi:
10.1109/TNET.2019.2931473
[58] M. Shojaee, M. Neves, and I. Haque, “Safeguard: Congestion and
memory-aware failure recovery in sd-wan,” in 2020 16th International
Conference on Network and Service Management (CNSM), 2020. doi:
10.23919/CNSM50824.2020.9269119 pp. 1–7.
[59] M. Vanamoorthy, V. Chinnaiah, and H. Sekar, “A hybrid approach for
providing improved link connectivity in sdn,” The International Arab
Journal of Information Technology, vol. 17, pp. 250–256, 02 2020. doi:
10.34028/iajit/17/2/13
[60] D. Zad Tootaghaj, F. Ahmed, P. Sharma, and M. Yannakakis, “Homa:
An efficient topology and route management approach in sd-wan
overlays,” in IEEE INFOCOM 2020 - IEEE Conference on Computer
Communications, 2020. doi: 10.1109/INFOCOM41043.2020.9155503
pp. 2351–2360.
[61] M. Obadia, M. Bouet, J. Leguay, K. Phemius, and L. Iannone, “Failover
mechanisms for distributed sdn controllers,” in 2014 International
Conference and Workshop on the Network of the Future (NOF), vol.
Workshop, 2014. doi: 10.1109/NOF.2014.7119795 pp. 1–6.
86 | REFERENCES

[62] K. Basu, A. Hamdullah, and F. Ball, “Architecture of a cloud-

based fault-tolerant control platform for improving the qos
of social multimedia applications on sd-wan,” in 2020 13th
International Conference on Communications (COMM), 2020. doi:
10.1109/COMM48946.2020.9142038 pp. 495–500.
[63] Cisco Meraki, “Mx warm spare - high availability pair,” [Online]
https://documentation.meraki.com/@api/deki/pages/1430/pdf/
MX%2bWarm%2bSpare%2b-%2bHigh%2bAvailability%2bPair.
pdf?stylesheet=default, [Accessed: 2021-04-07].
[64] Cisco Meraki, “Mx load balancing and flow preferences,” [Online]
https://documentation.meraki.com/@api/deki/pages/1462/pdf/
MX%2bLoad%2bBalancing%2band%2bFlow%2bPreferences.pdf?
stylesheet=default, [Accessed: 2021-04-15].
TRITA-EECS-EX-2021:847

www.kth.se