You are on page 1of 148

NTNU

Norwegian University of Science and Technology


Faculty of Information Technology and Electrical
Engineering
Dept. of Information Security and Communication
Master’s thesis
Technology

June 2020
Simen Haga

Supervisor: Katina Kralevska, IIK


Master’s thesis in Communication Technology
WireGuard and Open Source MANO
Towards 5G network slice isolation with
Simen Haga

Towards 5G network slice isolation


with WireGuard and Open Source
MANO

Master’s thesis in Communication Technology


Supervisor: Katina Kralevska, IIK
June 2020

Norwegian University of Science and Technology


Faculty of Information Technology and Electrical Engineering
Dept. of Information Security and Communication Technology
Title: Towards 5G network slice isolation with WireGuard
and Open Source MANO
Student: Simen Haga

Problem description:

5G is expected to enable various verticals to thrive on the mobile network and satisfy
a myriad of performance requirements. These requirements range from ultra-low
latency to increased bandwidth and connections of a massive amount of IoT devices.
To solve these challenges, 5G must be extremely flexible and be able to adapt to new
and upcoming technologies, which have yet to be invented.

Infrastructure Providers will use enabling technologies such as Software Defined


Networking, Network Function Virtualization, and Management and Orchestration
to create multiple logically separated network slices in the network. These slices will
provide the underlying network fit to meet the requirements of specific services.

Due to the high volatility of the 5G mobile network, it will be important to ensure
secure communication between virtual endpoints. Therefore this thesis will investigate
how Open Source MANO can be used to manage and orchestrate the Virtual
Private Network technology WireGuard to secure the communication between Virtual
Network Functions residing in all domains of the mobile network architecture.

Responsible professor: Katina Kralevska, IIK


Supervisor: Katina Kralevska, IIK, Danilo Gligoroski, IIK
Abstract

Supporting vertical industry segments and service creation has been a


priority during the development of the 5G mobile network. Through
new business models and enabling technologies such as Network Function
Virtualization and Software-Defined Networking, every component of
the mobile network will be regarded as a service. Logically separated
network segments will co-exist on the physical network infrastructure
to support specific use cases. While the trend of sharing an underlying
physical infrastructure promises innovation and agility, it introduces new
threat vectors. Therefore, proper isolation between logically separated
networks is essential to ensure correct service delivery and protection of
sensitive data. Over-The-Top isolation techniques, such as Virtual Private
Network solutions, are expected to be utilized to provide traffic-isolation
and mitigate some of these threat vectors.

This thesis contributes to the growing management and orchestration


community in the 5G research field. Through building our network
service that provides a VPN-as-a-Service with WireGuard and Open
Source MANO, we aim to give the Network Service Providers the means
to integrate additional network slice isolation in their infrastructure. As
part of a larger project to adopt Open Source MANO and build end-to-end
network slices between NTNU Trondheim and Gjøvik, the resulting Proof
of Concept is a starting point for future work to design a fully functional
and automated end-to-end VPN network service with WireGuard. To
validate the network service we tested its Key Performance Indicators,
including service creation time, throughput, and latency.

With our work we prove that by combining Open Source MANO and
WireGuard, it is possible to create a network service that can provide VPN-
as-a-Service. Furthermore, our results show that WireGuard outperforms
OpenVPN in terms of latency and throughput while remaining simple to
implement and maintain.
Sammendrag

Å støtte vertikale industri segmenter og tjeneste-utvikling har vært en


prioritet under utviklingen av 5G mobilnettet. Ved hjelp av nye business
modeller og teknologier som Virtuelle Nettverksfunksjoner og Program-
varedefinerte Nettverk vil hver komponent i det mobile nettet ansees
som en tjeneste. Logisk separerte nettverkssegmenter vil sameksistere
på den fysiske nettverksunfrastrukturen for å støtte spesifike tjenester.
Selv om trenden av å dele en underliggende fysisk infrastruktur bidrar
til innovasjon og mobilitet, introduserer den nye trussel vektorer. Derfor
er isolasjon mellom logisk separerte nettverk essensielt for å forsikre seg
om korrekt tjenestelevering og beskyttelse av sensitiv data. Over-Toppen
isolasjonsteknikker som Virtuelle Private Nettverk forventes å bli brukt
til å tilby isolasjon av trafikk og mitigere noen av disse trussel vektorene.

Denne oppgaven bidrar til det voksende ledelse og orkestrerings samfunnet


innen 5G forskning. Ved å undersøke måter å bygge en nettverkstjeneste
som tilbyr VPN-som-en-Tjeneste med WireGuard og Open Source MANO.
Sikter vi på å gi leverandører av nettverkstjenester verktøy til å integrere
ytterligere isolasjon av nettverkssegmenter i deres infrastruktur. Som
del av et større prosjekt for å benytte seg av Open Source MANO til å
bygge nettverskstykker mellom NTNU Trondheim og Gjørivk, resulterte
arbeidet våres i et enkelt konseptbevis, som fungerer som et startpunkt
for fremtidig arbeid en funksjonell og automatisert ende-til-ende VPN
nettverkstjeneste med WireGuard. For å validere nettverkstjenesten vår
testet vi dens nøkkeltallsindikatorer som inkluderer opprettingstid for
tjeneste, gjennomstrøming, og forsinkelse.

Med våres arbeid har vi vist at ved å kombinere Open Source MANO
og WireGuard kan en skape en nettverkstjenese som tilbyr VPN-som-en-
Tjeneste. Ved å teste WireGuards ytelse opp mot OpenVPN fastslo vi
også at WireGuard utkonkurrerer OpenVPN når det kommer til gjen-
nomstrømming og forsinkelse samtidig som den er enklere å implementere
og vedlikeholde.
Preface

This thesis is the final deliverable of an M.Sc. in Communication Tech-


nology with a specialization in Information Security at the Norwegian
University of Science and Technology (NTNU).

This thesis’s work is part of a larger project at the Department of Infor-


mation Security and Communication Technology to research end-to-end
network slices between NTNU Trondheim and Gjøvik. The thesis is a
result of Simen Haga’s work, with supervisors Katina Kralevska, Danilo
Gligoroski, and Ali Esmaeily.
Contents

List of Figures xi

List of Tables xv

Listings xvii

List of Acronyms xix

1 Introduction 1
1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Research Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Hypothesis Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.7 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.8 Tools and Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.8.1 Open Source MANO . . . . . . . . . . . . . . . . . . . . . . . 5
1.8.2 WireGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.8.3 OpenStack . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.8.4 Kubernetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.8.5 Juju . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.8.6 Cloud-init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.9 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Methodology 9
2.1 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Design Science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Background and Related Work 15


3.1 Background Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.1 5G Mobile Network . . . . . . . . . . . . . . . . . . . . . . . 15
3.1.2 Network Slicing . . . . . . . . . . . . . . . . . . . . . . . . . . 17

vii
3.1.3 Network Function Virtualization . . . . . . . . . . . . . . . . 18
3.1.4 Open Source MANO . . . . . . . . . . . . . . . . . . . . . . . 19
3.1.5 WireGuard . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 Towards Service-Oriented 5G: Virtualizing the Networks for
Everything-as-a-Service . . . . . . . . . . . . . . . . . . . . . 24
3.2.2 Dynamic Setup of IPsec VPNs in Service Function Chaining 24
3.2.3 WireGuard: Next Generation Kernel Network Tunnel . . . . 25
3.2.4 A Comparative Study on Virtual Private Networks for Future
Industrial Communication Systems . . . . . . . . . . . . . . . 26
3.2.5 SecMANO: Towards Network Functions Virtualization (NFV)
Based Security MANagement and Orchestration . . . . . . . 26
3.2.6 Services Orchestration Within 5G Networks — Challenges and
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.7 Open Multi-Vendor NFV Showcase . . . . . . . . . . . . . . . 28
3.2.8 On End-to-End Approach for Slice Isolation in 5G Networks.
Fundamental challenges . . . . . . . . . . . . . . . . . . . . . 29
3.2.9 On the Challenges and KPIs for Benchmarking Open-Source
NFV MANO Systems: OSM vs ONAP . . . . . . . . . . . . . 30
3.2.10 Providing Strong 5G Mobile Network Slice Isolation for Highly
Sensitive Third-Party Services . . . . . . . . . . . . . . . . . 30
3.3 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4 Development, Tests and Results 33


4.1 Testing WireGuard with Virtual Machines . . . . . . . . . . . . . . . 33
4.1.1 Technical Requirements . . . . . . . . . . . . . . . . . . . . . 34
4.1.2 Network Topology Setup . . . . . . . . . . . . . . . . . . . . . 34
4.1.3 Configuring WireGuard . . . . . . . . . . . . . . . . . . . . . 36
4.2 Building the WireGuard Network Service . . . . . . . . . . . . . . . 39
4.2.1 Technical Requirements . . . . . . . . . . . . . . . . . . . . . 39
4.2.2 Testbed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2.3 Building the Network Service . . . . . . . . . . . . . . . . . . 40
4.3 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.3.1 Measuring Service Creation Time . . . . . . . . . . . . . . . . 46
4.3.2 Measuring Throughput and Latency . . . . . . . . . . . . . . 47

5 Discussion 49
5.1 Testing WireGuard with Virtual Machines . . . . . . . . . . . . . . . 49
5.2 VPN-as-a-Service PoC . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.3 Analysis discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4 OSM-WireGuard lifecycle . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4.1 Day-0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4.2 Day-1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
5.4.3 Day-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.5 Application of WireGuard in mobile scenarios . . . . . . . . . . . . . 63
5.6 Challenges & Limitations . . . . . . . . . . . . . . . . . . . . . . . . 66

6 Conclusion and Future Work 69


6.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
6.1.1 WireGuard Network Slice Implementation . . . . . . . . . . . 69
6.1.2 Automatic Key-Distribution . . . . . . . . . . . . . . . . . . . 69
6.1.3 Multi-VIM on an end-to-end platform, 5GIIK . . . . . . . . . 69
6.1.4 WireGuard Network Service Performance Limitations . . . . 70
6.1.5 VNF Migration Without Service Interruption . . . . . . . . . 70
6.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

References 73

Appendices
A VPN-as-a-Service PoC 77

B OpenVPN Configuration 93

C Logs 97

D Research Paper 111


List of Figures

2.1 Design Cycle edited with relevant steps to the master thesis. . . . . . . 11
2.2 Empirical Cycle as part of the engineering cycle. . . . . . . . . . . . . . 13

3.1 The Figure from [OAL+ 17] illustrates the vertical relationship between
Infrastructure Providers and their Tenants. The Infrastructure Provider
builds a Virtual Network, which runs a service, and rents it to a Tenant
which uses it to serve an End User. . . . . . . . . . . . . . . . . . . . . . 17
3.2 Network slices will co-exist on a shared physical infrastructure to provide
services to vertical industry segments or end users. Slices of various types
will provide different user experiences. . . . . . . . . . . . . . . . . . . . 18
3.3 The ETSI NFV Architectural framework adapted from [Com19e]. Through
the OSS/BSS the administrator interacts with the MANO component,
which manages and orchestrates the other NFV components. . . . . . . 19
3.4 A network slice is the composition of multiple network services, specified
in the Network Slice Template, composed of multiple Virtual Network
Functions linked according to the Network Service Descriptor. The
Management and Orchestration entity instantiates and links the Vir-
tual Network Functions from the Virtual Network Function Catalogue
according to Network Service Descriptions or Network Slice Templates. 20
3.5 The NBI of OSM makes use of the extensive Information Model, to
minimize integration efforts of network operators. Proxy charms are
used to configure Virtual pr Physical Network Functions deployed on the
Physical Infrastructure, or Network Function Virtualization Infrastructure
[Com19e] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.6 The cryptographic key routing table 1a, 1b, and 2a [Don18]. . . . . . . . 23
3.7 Everything-as-a-Service, as presented by Chang et al. [CZZ+ 18] . . . . . 25
3.8 Bandwidth and latency measurements show how WireGuard performs
against well-established VPN technologies. . . . . . . . . . . . . . . . . . 26
3.9 Threat taxonomy for Network Function Virtualization layers presented
by Pattarantakul et al. [PHMZ16]. Displaying threats and vulnerabilities
in each layer, as well as relevant attacks and countermeasures. . . . . . 27

xi
3.10 Operational flow of the secMANO framework [PHMZ16], shows how the
new security component is appended to the European Telecommunica-
tions Standards Institute (ETSI) Network Function Virtualization (NFV)
Management and Orchestration (MANO) framework. . . . . . . . . . . 28
3.11 The complete virtualized LTE Packet Core built to showcase a multi-
vendor Virtual Network Function (VNF) scenario [Whi19]. VNF packages
from NG4T, OpenAirInterface, Mobileum, and Fortinet were combined
to build the network slice which OSM managed and orchestrated on top
of OpenStack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.12 The operations which lead to time-latency in deployment and orchestration
of network services [GMY19]. . . . . . . . . . . . . . . . . . . . . . . . . 31

4.1 The network topology used during the first design cycle shows two Wire-
Guard gateways connecting subnet Data-East and Data-West. . . . . . 34
4.2 The network topology used during the first design cycle, shows the in-
stantiated WireGuard tunnel between WG-West and WG-East. . . . . . 38
4.3 The software stack used during the development of the VPN-as-a-Service
Proof of Concept. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.4 The WireGuard network service is composed of two Virtual Network
Functions connected through Virtual Links and Connection Points. The
Connection Points coloured in blue, are also shown as yellow in Figure 4.5. 44
4.5 OSM’s graphical interface shows that the WireGuard Virtual Network
Function is composed of one Virtual Deployment Unit and three Connec-
tion Points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.6 OpenStack’s network topology display shows the two WireGuard gateways
connected to the tunnel network, and the test VMs. . . . . . . . . . . . 45
4.7 Throughput of WireGuard and OpenVPN between the gateway nodes,
averaged over 30 minutes with iperf. . . . . . . . . . . . . . . . . . . . 48
4.8 Average latency, after 1000 ICMP requests between a gateway and a
peering subnet’s interface on the gateway. . . . . . . . . . . . . . . . . 48

5.1 The sequence-diagram shows the process of generating and distributing


configuration files, which include the WireGuard keys, during the first
design cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5.2 The sequence-diagram shows the ideal process of generating and distribut-
ing keys and configuration files. . . . . . . . . . . . . . . . . . . . . . . . 52
5.3 The process of instantiating the WireGuard network service through the
use of the Open Source MANO (OSM)-North Bound Interface (NBI). . 54
5.4 The config-primitives of the WireGuard network service. . . . . . . . . . 55
5.5 The operator uses the OSM-NBI to interact with OSM which manages and
orchestrates the network service through a set of operations, categorized
as Day-0, Day-1, and Day-2 operations. . . . . . . . . . . . . . . . . . . 60
5.6 Initial-config-primitive used by OSM to performe initial configurations in
a programmable and reproducible manner. . . . . . . . . . . . . . . . . 61
5.7 Network Service Descriptor of the WireGuard network service with Virtual
Link Descriptors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
5.8 Network Service Descriptor of the WireGuard network service with
connection-points and ip-profiles. . . . . . . . . . . . . . . . . . . . . . . 64
5.9 Possible architecture that makes use of the WireGuard VPN-as-a-Service
solution with OSM in 5G networks and beyond. . . . . . . . . . . . . . . 65
List of Tables

1.1 Network Function Virtualization threat vectors [HCP18]. . . . . . . . . 2

2.1 Important keywords used to find for relevant research. . . . . . . . . . . 10

3.1 The User Experience requirements table from [All15] illustrates that 5G
is expected to have many operational performance characteristics. . . . 16

4.1 VM flavor for the first design cycle. . . . . . . . . . . . . . . . . . . . . . 33


4.2 Network allocation plan during the first design cycle. . . . . . . . . . . . 34
4.3 Hardware and operative system used for the testbed during the second
design cycle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
4.4 List of actions/primitives used by the Juju charm framework. . . . . . . 42
4.5 Service creation time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.6 Network service action primitives and their execution time . . . . . . . . 47

5.1 A speed test was performed with openssl to add context to the difference
in latency and throughput between the results. . . . . . . . . . . . . . . 59

xv
Listings

4.1 Enable forwarding and ARP proxy. . . . . . . . . . . . . . . . . . . . 35


4.2 Assign IP addresses and configure interfaces. . . . . . . . . . . . . . 35
4.3 Configure interfaces on gateways. . . . . . . . . . . . . . . . . . . . . 36
4.4 WG-East configuration. . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.5 WG-West configuration. . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.6 Distribution of configuration files. . . . . . . . . . . . . . . . . . . . . 37
4.7 Configure WireGuard on the gateway. . . . . . . . . . . . . . . . . . 38
4.8 WireGuard connectivity confirmed. . . . . . . . . . . . . . . . . . . . 39
4.9 Build Virtual Network Function package with OSM. . . . . . . . . . 41
4.10 Install Juju charms and create a skeleton-charm. . . . . . . . . . . . 41
4.11 All actions in the actions folder must be executable. . . . . . . . . . 42
4.12 Build and validate the charm. . . . . . . . . . . . . . . . . . . . . . . 42
4.13 Create the network service package, base directory. . . . . . . . . . . 43
4.14 Validate the network service package and upload it to the osm platform. 43
4.15 Instantiate the network service through the OSM-NBI. . . . . . . . . 43
4.16 Connect the two WireGuard gateways with the use of OSMs ns-action
feature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.17 Viewing the LifeCycle Management unit’s logs with microk8s. . . . . 46
5.1 Configure WireGuard on the gateway . . . . . . . . . . . . . . . . . 60
5.2 Multi-VIM network service configuration . . . . . . . . . . . . . . . . 66
A.1 testbed setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
A.2 action.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
A.3 layer.yaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
A.4 metadata.yaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
A.5 actions.yaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
A.6 parameters.yaml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
A.7 vWireGuard_nsd.yaml . . . . . . . . . . . . . . . . . . . . . . . . . . 82
A.8 vWireGuard_vnfd.yaml . . . . . . . . . . . . . . . . . . . . . . . . . 84
A.9 wg-cloud-init . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
A.10 wg_charm.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
B.1 openvpn-server.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
B.2 client.ovpn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

xvii
C.1 lcm.log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
List of Acronyms

5G PPP The 5G Infrastructure Public Private Partnership.

AC Access Control.

API Application Programming Interface.

BSS Business Support System.

CLI Command-Line Interface.

CP Connection Point.

DPD Deployment Process Delay.

eMBB Enhanced Mobile Broadband.

ETSI European Telecommunications Standards Institute.

IaaS Infrastructure-as-a-Service.

IM Information Model.

InP Infrastructure Provider.

KNF Kubernetes-based Network Function.

KPI Key Performance Indicator.

LCM LifeCycle Management.

LDAP Lightweight Directory Access Protocol.

MANO Management and Orchestration.

MDM Mobile Device Management.

xix
mMTC Massive Machine Type Communication.

MNO Mobile Network Operator.

MON Monitoring Module.

NBI North Bound Interface.

NFV Network Function Virtualization.

NFVI Network Function Virtualization Infrastructure.

NGMN Next Generation Mobile Network.

NIC Network Interface Card.

NSD Network Service Descriptor.

NSI Network Slice Instance.

NSP Network Service Provider.

NST Network Slice Template.

OPD On-boarding Process Delay.

OPEX Operating Expenses.

OSM Open Source MANO.

OSS Operating Support System.

OTT Over-The-Top.

PKI Public Key Infrastructure.

PoC Proof of Concept.

POL Policy Module.

QOD Quality of Decision.

RBAC Role-Based Access Control.

RO Resource Orchestrator.

ROD Runtime Orchestration Delay.

SDN Software Defined Networking.


UE User Equipment.

URLLC Ultra-Reliable and Low Latency Communication.

VCA VNF Config and Abstraction.

VDU Virtual Deployment Unit.

VIM Virtual Infrastructure Manager.

VL Virtual Link.

VLD Virtual Link Descriptor.

VM Virtual Machine.

VNF Virtual Network Function.

VNFD Virtual Network Function Descriptor.

VNFM Virtual Network Function Manager.

VPN Virtual Private Network.

VPNaaS VPN-as-a-Service.

XaaS Everything-as-a-Service.
Chapter
Introduction
1
1.1 Background and Motivation

The 5G mobile network is expected to bring significant changes to how mobile


networks are viewed in the future and lay the grounds for innovation like never before
[RSI+ 19, CKS+ 17, ZGF+ 18]. With new business models where everything will be
regarded as a service, a plethora of different services is expected to be provided on
the new mobile network. Additionally, 5G must be able to handle new types of
services and requirements that have yet to be explored [All15].

Preparing the 5G mobile network for unknown types of services is possible with the aid
of enabling technologies such as NFV, Software Defined Networking (SDN), and cloud
computing which through a decoupling of the control and data planes, virtualization,
and flexible function allocation provide a basis for rapid development of virtual
network services that constitute the network slices [MSG+ 16, JSS+ 14, ATS+ 18].
These technologies lay the grounds for Everything-as-a-Service (XaaS) [CZZ+ 18],
which will enable multiple business models centered around third-parties providing
specialized networks and exposing the configuration of the service through open
Application Programming Interfaces (APIs), which Network Service Providers (NSPs)
use to provide end-users with new services.

The new services must satisfy requirements for a myriad of different use cases such
as Ultra-Reliable and Low Latency Communication (URLLC), Massive Machine
Type Communication (mMTC), and Enhanced Mobile Broadband (eMBB). These
three main categories of use cases have different levels of security, depending on the
type of data and service it provides. Thus, concerns have been raised due to the
underlying shared infrastructure, and multiple threats have been identified against
the upcoming network slicing concept in 5G mobile networks [Gro16]. Some of these
threats include denial of service to other slices, differences in security protocols or
policies, cross slice side-channel attacks, or insufficient isolation between slices when
a User Equipment (UE) is attached to several slices.

1
2 1. INTRODUCTION

Threat Vector Description Impact


VNF Service Flooding Attackers can flood the service or the Availability
network interface using attacks such as
DNS lookup, multiple authentication
failure attempts resulting in denial-of-
service (DoS) in signaling plane and
data plane.
Application Crashing Attackers can send malformed packets Availability
to the services running in NFV environ-
ments and cause network service dis-
ruption (e.g, buffer overflow exploit).
Eavesdropping Attackers can eavesdrop on sensitive Confidentiality
data and control plane messages.
Data Ex-filtration Unauthorized access to sensitive data Confidentiality
such as user profiles.
Data and Traffic Mod- Attacker can perform Man-in-the- Integrity
ification Middle attack on the network traffic
in transit, perform DNS redirection or
modify sensitive data on network ele-
ments.
Control Network and The attacker can exploit protocol vul- Control
Network Elements nerabilities or implementation flaws to
compromise a network. Additionally,
attackers can exploit vulnerabilities on
the management interface to take con-
trol of network elements.

Table 1.1: Network Function Virtualization threat vectors [HCP18].

More specifically, some of the threat vectors target NFV, which will be central to the
development of new network services. As shown in Table 1.1, provided by Huang et al.
[HCP18], there are multiple threat vectors which impact availability, confidentiality,
integrity, and control in the NFV ecosystem. Since NFV will constitute a major part
of the 5G mobile network infrastructure, it is essential to investigate how to reduce
or mitigate the risks associated with these threats.

Multiple solutions that promise to solve the issue of management and orchestration
of the NFV components in 5G have appeared in recent years [MLM+ 19]. These tools,
which are in line with the NFV MANO specification, aim to manage and orchestrate
network slices, network services, and physical and virtual network functions that
1.2. PROBLEM DESCRIPTION 3

co-exist on the 5G mobile network infrastructure [Com14b]. The NFV MANO tools
will be a central technology that enables service providers to efficiently manage
and orchestrate the complex environment of services provided by third parties form
network slices. The ETSI backed MANO project OSM [Com19d] has recently gained
much traction in the NFV community, and might become one of the leading projects
for MANO in 5G. The simplicity and availability of OSM will allow both Infrastructure
Providers (InPs) and NSPs to make good use of the XaaS business model, by lending
third-party network service solutions amongst each other, enable rapid development
and service creation, and satisfy a large amount of the expected use cases.

As the 5G mobile network will allow InP to provide Infrastructure-as-a-Service (IaaS)


to NSPs, allowing the NSPs to share the physical infrastructure, an attribute used
to determine the security will be the level of isolation associated with the network
slice. Multiple technologies will provide isolation in different levels of the 5G network
stack. One such solution is Virtual Private Networks (VPNs) solutions which will be
used to isolate traffic between VNFs in a network slice in an Over-The-Top (OTT)
manner. Since Next Generation Mobile Network (NGMN) expects multiple types of
services on the 5G mobile network, an important factor when choosing a VPN is how
well it performs in certain scenarios. WireGuard is a new VPN technology, operating
on the network layer, which provides a simple yet efficient solution to protecting
data between endpoints. The inclusion of WireGuard in Linux kernel 5.6, and the
performance it brings for all types of devices, has made it a strong contender as a
VPN solution that could contribute to network slice isolation.

1.2 Problem Description


NFV is becoming more prominent and will be an enabling technology for 5G. However,
due to threat vectors identified in by Huang et al., it is crucial to provide network
operators with the means to protect both their own and their customers’ data. The
confidentiality of data and control plane messages becomes harder to ensure because
of the volatility and complexity of the 5G mobile network’s architecture. As VNFs
and network services move closer to the user, it becomes more challenging to identify
the threat landscape. Hence, practical and efficient security solutions need to be
available to InPs and NSPs.

1.3 Research Scope


The 5G mobile network ecosystem is more complicated than any of its predecessors,
partly due to all the new use cases it is supposed to accommodate, and the change in
the technology that it utilizes. Network slicing is also a rather new and developing
field with a myriad of components. Thus, to limit our research scope and ensure that
we can provide a meaningful contribution to the research community, our primary
4 1. INTRODUCTION

focus is on the development, management, and orchestration of network services.


Therefore, even though we talk about network slices and their requirements, our
research is focused on VNFs and network services, which constitute the network
slice. Initially, the aim was to use our WireGuard network service in a network slice
of the type eMBB, which we would instantiate on a testbed built by Esmaeily et
al. [esm20]. However, due to issues we explain later, we ended up showcasing the
Proof of Concept (PoC) on a minimal testbed, and run it as a single network service
— not as part of a network slice. The focus was therefore shifted to management
and orchestration of VNFs, which are core components of the network service, and
investigating methods OSM could interact with the WireGuard service running on
said VNFs.

1.4 Contribution
The research scope is in line with the contributions we aim to provide to the research
community. First and foremost, we aim to contribute to the ETSI MANO project by
creating a simple PoC which can be posted on their list of PoCs1 . Along with the
PoC, we aim to propose a WireGuard-OSM lifecycle that will be a significant step
towards ensuring traffic isolation in 5G orchestrated networks. Our proposed lifecycle
and PoC will include the Juju charm we designed, which handle configuration of
WireGuard. The PoC also includes our designs for the Virtual Network Function
Descriptor (VNFD) and Network Service Descriptor (NSD), available on Github2
used to describe our network service’s infrastructure. We also aim to take part in
the adoption of WireGuard, which we believe is a strong candidate for future VPN
services. By providing a step by step guide of the installation, instantiation, and
configuration of our network service, we also lay the foundation for future work on
the development of an isolated end-to-end network slice. In addition to the thesis,
our work has resulted in a paper which is currently a work in progress, included
in Appendix D. The paper VPN-as-a-Service for 5G Networks:Proof-of-Concept
summarizes our main contributions.

1.5 Hypothesis Statement


During the thesis, we worked with the hypothesis that OSM can be used to manage
and orchestrate the secure communication between VNFs in a mobile network, where
VNFs are instantiated and re-locate at a higher pace than previous mobile networks.
We also hypothesized that with the aid of OSM and WireGuard; it is possible to
create an overlay network that can contribute to network slice isolation in a 5G
mobile network.
1 https://osm.etsi.org/wikipub/index.php/OSM_PoCs - Sourced: 2020-02-09
2 https://github.com/simehag/wg-osm-ns - Accesed: 2020-06-28
1.6. RESEARCH OBJECTIVES 5

1.6 Research Objectives


RO1 Design a VPN gateway based on WireGuard.

RO2 Use OSM to manage and orchestrate a VPN network service with WireGuard.

1.7 Research Questions


RQ1 Is it possible to use OSM to manage and orchestrate a WireGuard VPN and
provide network slice isolation?

RQ2 How will the use of WireGuard affect Key Performance Indicators (KPIs)
between nodes in a simple network slice?

RQ3 How will the service creation time of our network service be compared to the
goals of The 5G Infrastructure Public Private Partnership (5G PPP)?

RQ4 What is the current state-of-the-art in terms of OSM, and how is it used to
manage and orchestrate VPN technologies?

1.8 Tools and Resources


During the thesis, we utilized a handful of tools to complete our work. Below is a
summarized description of the tools we used the most during the thesis.

1.8.1 Open Source MANO


OSM3 developed through a collaboration of ETSI partners, is used to manage and
orchestrate the NFVs technology. In this thesis, OSM will be used to deploy VNFs,
network services, and network slices, and to manage and orchestrate WireGuard.
More details about OSM are provided in Section 3.1.4.

1.8.2 WireGuard
WireGuard4 is a cross-platform VPN tunnel which utilizes state-of-the-art cryptogra-
phy in a simple and efficient manner. In this thesis it will be deployed on our VNFs
and provide the secure network tunnel. More information about WireGuard will be
given in Section 3.1.5. Along with WireGuard we used the tools wg-quick5 , and
wireguard-mesh-generator6 , which we discuss in more detail in Section 4.1.3.
3 https://osm.etsi.org/- Sourced: 2020-03-15
4 https://www.wireguard.com/ - Sourced: 2020-03-15
5 https://git.zx2c4.com/wireguard-tools/about/src/man/wg-quick.8 - Sourced: 2020-02-27
6 https://github.com/k4yt3x/wireguard-mesh-configurator - Sourced: 2020-04-20
6 1. INTRODUCTION

1.8.3 OpenStack
OpenStack7 is a cloud operating system used to provide IaaS orchestration, and
fault and service management to operators. For the case of this thesis, it hosts
the base images, refered to as Virtual Deployment Units (VDUs), on which the
VNFs are instantiated, and the virtual network infrastructure connecting the VNFs.
OSM handles the rest of the orchestration, configuration, and management through
OpenStack’s NBI.

1.8.4 Kubernetes
Kubernetes8 is a container orchestration engine which manages the lifecycle of
containers. The tool allows operators to distribute containers within pods and
control their interconnections, deployment, and scaling. During this project we used
a lightweight version of Kubernetes, Microk8s9 , which is used by ETSI to manage all
the connected pods that constitute OSM.

1.8.5 Juju
Juju10 is one of the main Virtual Network Function Managers (VNFMs) for OSM.
It works by interacting with proxy charms that act as structured VNF managers
for VNFs on the Network Function Virtualization Infrastructure (NFVI). A charm
holds all the necessary hooks to manage the lifecycle of a VNF, such as deployment,
configuration, and exposing services to external processes.

1.8.6 Cloud-init
cloud-init11 is a tool maintained by Canonical, used to ensure that a set of Virtual
Machines (VMs) based on the same cloud-init configuration are identically configured
during instantiation. OSM has included support for cloud-init, which means the we
can describe the packages, the hostname, passwords e.g. which should be installed
and configured on our VNFs.

1.9 Thesis Structure


We have divided the rest of the thesis into five chapters. Chapter 2 presents the
scientific method applied during the thesis work. Chapter 3 introduces the required
background knowledge and presents selected papers we deem state-of-the-art. Chapter
7 https://www.openstack.org/software/ - Sourced: 2020-05-21
8 https://kubernetes.io/ - Sourced: 2020-05-21
9 https://microk8s.io/ - Sourced: 2020-06-08
10 https://jaas.ai/ - Sourced: 2020-05-21
11 https://cloud-init.io/- Sourced: 2020-04-20
1.9. THESIS STRUCTURE 7

4 goes through the setup of our platform and network service and presents our analysis
and results. In Chapter 5, we discuss the platform and network service, and our
results. Chapter 6 presents future work and concludes the thesis.
Chapter
Methodology
2
This chapter presents the methodology used in the project and the reasoning behind
our choice. To be able to work in a scientifically sound manner, we chose to adopt a
methodology that is best suited to our needs and provides a high possibility that we
would succeed in our search for new knowledge. Thus, we chose to use a combination
of Design Science with a Single-case mechanism study, and a Literature review to
reach our goals.

2.1 Literature Review

We started with the literature review to discover relevant state-of-the-art research,


and make sure we had a profound understanding of the topic. Searching for relevant
articles and papers requires the right tools. Therefore, we mostly used Google
Scholar1 and IEEE Explorer2 to search the web. The Open-Source MANO site,
mentioned in Section 1.4, also provided a good overview of the current PoCs projects
which focused on displaying OSM’s applicability in the modern mobile network
infrastructure.

To best be able to find relevant sources, we chose to narrow the search by deciding
which attributes of a research project were of most importance to us. Since Open-
Source MANO and NFV, in general, have developed a lot during the recent years —
release date was a crucial factor in choosing which papers to read. Relevant keywords
were also necessary to decide the papers to read and include in the thesis. The most
important keywords can be found in Table 2.1 in order of importance.

While considering the relevant papers, we also focused on attributes such as the
number of citations, references, and peer-reviews. We were less inclined to include a
paper if it lacked either trustworthy references or peer-reviews.

1 https://scholar.google.com/ - Sourced: 2020-04-19


2 https://ieeexplore.ieee.org/Xplore/home.jsp - Sourced: 2020-04-19

9
10 2. METHODOLOGY

Keyword Description
OSM As the utilization of OSM is the primary topic, and the
framework has only existed for four years, it was essen-
tial to see where it was applied and which obstacles and
open research problems had previously been identified.
WireGuard WireGuard itself was an essential keyword since it was
essential to see if any previous work had been done with
WireGuard in combination with 5G technology.
Network Slice Isolation Finding relevant work in terms of network slice isolation
was necessary to get an overview of which attributes of
isolation were already covered.
5G Network Slicing Since an implicit goal of this thesis is to explore current
and possible solutions to problems in 5G with regards
to network slicing, it was important to assign a high
value to the presence of the keyword in a paper.
NFV/NFVI NFV is a core component of our thesis. Therefore
we wanted to understand better which solutions were
already available and what the current state-of-the-art
is.
Security We used the security keyword to filter out irrelevant
research when the number of papers available in either
5G Network Slicing or NFV/NFVI was too large.

Table 2.1: Important keywords used to find for relevant research.

2.2 Design Science

Designing and investigating artifacts in context is the object of study in the Design
Science framework. Artifacts are objects designed and created by humans, such as
software, hardware, methods, or techniques. In our case, artifacts are regarded as
the software WireGuard and OSM, while the context is the design, development,
maintenance, goal, desire, and use of the two artifacts working in tandem [Wie14b]
to provide network slice isolation. These goals and desires can be linked to the goal
of enhancing network slice isolation between nodes in an SDN/NFV based network
environment and design of a VPN-as-a-Service (VPNaaS) solution. It is not the
artifact or context itself, which solves a problem; instead, it is the interaction between
artifacts and the context that contributes to solving the problem. Investigation and
the design of artifacts in context are the primary problem-solving activities in design
science. Each activity is treated as a cycle, and they both contribute to a larger
cycle.

Design science consists of two types of cycles, which form the engineering cycle. The
2.2. DESIGN SCIENCE 11

Figure 2.1: Design Cycle edited with relevant steps to the master thesis.

design cycle consists of four phases, but for this thesis, only three of them are relevant,
and hence the fourth, Treatment Implementation, is excluded from our illustration in
Figure 2.2. During the design cycle, the objective is to solve design problems. First,
one investigates stakeholders and their goals, which is used to deduce the background
and motivation for the thesis. In our research, there are multiple stakeholders, all
with different goals. Network administrators working for a network service provider,
who will interact with the OSM-NBI to manage their network resources, would desire
a framework that is easy to use and implement and does not add much complexity
to the network environment. Simplicity with regards to integration is, thus, of high
importance. Network administrators would also have a goal of securing network
resources and look at performance trade-offs. Therefore, verifying if management
and orchestration of WireGuard with OSM can contribute to the network isolation of
a network slice will be among their goals and verify that using WireGuard between
nodes does not affect the network’s performance in a significant matter. NSPs have
the goal of acquiring a management solution which allows for simpler interoperability
with other NSPs, and efficient integration of network services. As the ETSI OSM
group has the desire to increase the adoption of OSM in the research community, and,
in the end, by NSPs. ETSI OSM has a continuous goal to showcase the effectiveness
of OSM. Hence, their goal is primarily to provide samples of successful projects with
OSM. If this thesis is successful, it would be of interest to the OSM ETSI group
to display the PoC on their website 3 . InPs would aim to use the tool for simpler
3 https://osm.etsi.org/wikipub/index.php/OSM_PoCs - Sourced: 2020-02-25
12 2. METHODOLOGY

Operating Support System (OSS)/Business Support System (BSS) solutions, which


OSM is already able to provide through its multi-tenant capabilities. Simpler and
efficient MANO is also of interest to InPs to provide XaaS. During the design cycle,
requirements for the artifacts interacting with the problem context are specified,
available solutions are discovered, and, if necessary, a new artifact is designed. The
combined product of WireGuard and OSM can be defined as a single artifact that
interacts with a problem context.

Gathering and analyzing empirical data is also essential to a research project, as it


allows us to answer knowledge questions. The design science framework, therefore,
includes the empirical cycle. Figure 2.2 illustrates the five key points of the empirical
cycle.

Since design science by itself is not a scientific methodology, but a framework in


which methods can be applied better to suit the need of a software engineer’s research,
using the appropriate validation methodology during the empirical cycle is necessary.
For our case, in which we are supposed to answer the research questions in Section
1.7, and conduct an implementation evaluation of our PoC, a Single-Case Mechanism
Experiment [Wie14a] suits best. With it, we investigated how the use of WireGuard
affected the throughput and latency between two subnets, compared to OpenVPN
4
, and how OSM can perform based on the Operational KPIs mentioned in Section
3.2.9.

Together, these cycles form the basis of the design science framework. By iterating
through each cycle when necessary, researchers can, in a systematic way, answer both
knowledge and design questions. This is also the reason why this framework was
chosen for our thesis, as it would allow us to perform small iterations on our testing
platform, answer questions, and apply the new knowledge to the next iteration. The
small iterations allowed us to focus on what we deemed as important design and
knowledge problems from the start.

4 https://openvpn.net/ - Sourced:2020-05-25
2.2. DESIGN SCIENCE 13

Figure 2.2: Empirical Cycle as part of the engineering cycle.


Background and Related Work
Chapter

3
The following chapter will explain topics related to the work outlined in our thesis,
and iterate on relevant state-of-the-art articles. Through our presented literature, we
aim to illustrate the relevance of a WireGuard network service that provides VPNaaS
in the expected fifth-generation mobile network.

3.1 Background Theory


The NFV architecture involves numerous components, and 5G is a complex ecosystem.
Understanding the current background theory needed to fully grasp the concept of
NFV can be cumbersome. Therefore, we have gathered and condensed the most
relevant background theory and information required to understand the remainder
of our thesis and present it in the following section.

3.1.1 5G Mobile Network


“5G is an end-to-end ecosystem to enable a fully mobile and connected society.
It empowers value creation towards customers and partners, through existing and
emerging use cases, delivered with consistent experience, and enabled by sustainable
business models.” [All15]

In their vision for 5G, NGMN indicates that 5G is expected to enable services for both
existing and emerging use cases. This requires the 5G mobile network to be prepared
for network utilization that we are unable to predict today. That is why 5G must be
able to adapt quickly to changes in technology and allow new services to easily be
integrated with the mobile network platform without affecting other services. The
5G PPP envisioned a 5G infrastructure which let NSPs "[reduce] the average service
creation time cycle from 90 hours to 90 minutes" [5G-]. Setting the focus on the
virtualization of network functions and the use of XaaS to enable rapid deployment
and development of network services. The NGMN whitepaper [All15] on 5G present
a non-exhaustive list of 25 use cases, summarized to 14 user experience requirements,

15
16 3. BACKGROUND AND RELATED WORK

Table 3.1: The User Experience requirements table from [All15] illustrates that 5G
is expected to have many operational performance characteristics.

displayed in Table 3.1, which shows the need for flexibility in 5G. It also illustrates
that 5G must adhere to a myriad of different performance requirements, ranging
from high broadband utilization to ultra-reliable and low latency communication.

5G’s business models are expected to expand the support of multiple types of
customers and partnerships [OAL+ 17]. Operators will support vertical industries
by contributing to the mobilization of industries and their processes. Through
providing XaaS, operators will be able to provide real-time sharing of their network
infrastructure to third parties. This will enable a relationship between the InP,
tenant and user which looks similar to the one in Figure 3.1. In this diagram, one
can see that the tenant, who requires a network fit for a certain use case, rents a
3.1. BACKGROUND THEORY 17

Figure 3.1: The Figure from [OAL+ 17] illustrates the vertical relationship between
Infrastructure Providers and their Tenants. The Infrastructure Provider builds a
Virtual Network, which runs a service, and rents it to a Tenant which uses it to serve
an End User.

slice of the network from the infrastructure provider. The network slice is then used
by the tenant to provide a network service to the end user. In this case, the tenant is
a Mobile Network Operator (MNO), which either uses the service to provide mobile
connectivity to subscribers or rents it out as virtual infrastructure to other tenants.

3.1.2 Network Slicing


To be able to provide onboarding of new vertical market segments, such as those
illustrated in Figure 3.2, the 5G mobile network will make use of network slicing. The
intention of a network slice is "to provide only the traffic treatment that is necessary
for the use case and avoid all other unnecessary functionality" [All15]. Thus, at least
one slice will exist on the InP’s physical network for each use case or service satisfied
by NSPs who are tenants of said InP.

Figure 3.1 illustrates that a network slice is the creation of multiple logical networks
on shared network infrastructure. Each slice will have its own set of dedicated
resources, which preferably are logically isolated [GK19]. As we have illustrated in
Figure 3.4, a network service is a set of VNFs, chained together through what is
referred to as Service Function Chaining. The network slice is formed by providers
linking the network service instances that exist on the NFVI.
18 3. BACKGROUND AND RELATED WORK

Tenants
eMBB mMTC URLLC

Infrastructure
Shared Physical Infrastructure
Provider

Figure 3.2: Network slices will co-exist on a shared physical infrastructure to


provide services to vertical industry segments or end users. Slices of various types
will provide different user experiences.

3.1.3 Network Function Virtualization


In their whitepaper from 2012 [Com12], ETSI proposed moving network functions to
virtual environments, which meant lower dependence on commodity hardware, higher
agility, and rapid deployment of new network functionality. The network functions can
be firewalls, load-balancers, proxy servers, and intrusion detection. Complemented
with SDN the networks will then become more programmable [FKGG19, KGFG19,
Før19]. The goal of NFV is to allow operators to have a unified and generic
virtualization infrastructure, which is compatible with any vendor’s VNF. NFV
is based on three primary standards: the NFV Architectural Framework, NFV
Infrastructure Overview, and NFV Management and Orchestration [Com14a, Com15,
Com14b]. Figure 3.3 illustrates the three functional blocks of the NFV Architectural
Framework, and the relations among them. Each block has its own responsibility
which are listed below.

– Network Function Virtualization Infrastructure and Virtual Infras-


tructure Manager. The Infrastructure block, composed of the NFVI and
Virtual Infrastructure Manager (VIM), is responsible for providing the vir-
tualization environment for the VNFs. The VIM is responsible for lifecycle
management of virtual resources, such as VMs, storage, networking and con-
nectivity.

– Network Functions. This block, contains the VNFs which are instantiated on
3.1. BACKGROUND THEORY 19

MANO
Network Functions

NFVI + VIM

Figure 3.3: The ETSI NFV Architectural framework adapted from [Com19e].
Through the OSS/BSS the administrator interacts with the MANO component,
which manages and orchestrates the other NFV components.

the underlying NFVI. Together, the VNFs form network services and network
slices managed and orchestrated by the MANO entity.

– Magement and Orchestration. The third block of the NFV Architectural


Framework is the MANO component, which controls the lifecycle of the VNFs,
network services, and network slices. It also covers the general resource orches-
tration and service lifecycle of the service it manages.

3.1.4 Open Source MANO


The OSM project aim to provide the functionality of the MANO block, mentioned
in Section 3.1.3 and displayed in Figure 3.3 [Com19e]. In the ETSI NFV group
specification [Com19b], the authors specified the functional requirements for NFV
20 3. BACKGROUND AND RELATED WORK

Figure 3.4: A network slice is the composition of multiple network services, specified
in the Network Slice Template, composed of multiple Virtual Network Functions linked
according to the Network Service Descriptor. The Management and Orchestration
entity instantiates and links the Virtual Network Functions from the Virtual Network
Function Catalogue according to Network Service Descriptions or Network Slice
Templates.
3.1. BACKGROUND THEORY 21

Figure 3.5: The NBI of OSM makes use of the extensive Information Model, to
minimize integration efforts of network operators. Proxy charms are used to configure
Virtual pr Physical Network Functions deployed on the Physical Infrastructure, or
Network Function Virtualization Infrastructure [Com19e]

MANO. OSM is adherent to this standard, and aims to support the widest range of
NFVIs, VIMs, VNFs, network services, and network slices [Com19e].

By making use of its extensive Information Model (IM) 1 , aligned with the ETSI NFV
specification [Com19c, Com19a], OSM is able to model and automate the lifecycle of
VNFs, network services, and network slices, as seen in Figure 3.5. Through an internal
unit known as the Resource Orchestrator (RO)2 , OSM manages and orchestrates
the necessary network resources. The RO is connected to the NBI of the registered
VIM, and through the NBI it manages and orchestrates the NFVI. Configuration
and instantiation of VNFs happen through the VNF Config and Abstraction (VCA)
unit which instantiates charms that connect directly to the VNFs through SSH or
exposed APIs, as Illustrated in Figure 3.5.

With VNFDs OSM describes and onboards all VNFs according to a unified VNF cat-
alogue. VNFs are packed into VNF packages, which include the VNFD, configuration
1 https://osm.etsi.org/wikipub/index.php/OSM_Information_Model - Sourced: 2020-02-12
2 Not to be confused with the Research Objectives, referred to as RO#.
22 3. BACKGROUND AND RELATED WORK

scripts, and other artifacts such as charms and instantiation scripts. VNF providers
upload their NSDs to the OSM platform describing network services which combine
VNFs from the VNF catalogue to build network services. The network services can
be instantiated to function by themselves, or, through the use of a Network Slice
Template (NST) it can be part of a Network Slice Instance (NSI). The relationship is
illustrated in Figure 3.4, where the VNFs are connected through Virtual Links (VLs)
to form a network service, and each network service exposes an external Connection
Point (CP), which is connected to a VL to form a network slice.

The process of development and deployment of a VNF package, as defined by ETSI


MANO, is known as the onboarding process [ETS]. It consists of three stages that
the ETSI OSM group refer to as Day-0, Day-1, and Day-2, which focus on the full
lifecycle of a single VNF package and the network service it is part of.

– Day-0: The goal of this phase is to provide all the necessary elements of a
VNF package for it to be instantiated successfully, and set up its management
so it can be configure later.

– Day-1: During Day-1 the VNF is instantiated and configured to enable the
continuous delivery of its services.

– Day-2: The last phase occupies the majority of the VNF’s lifecycle. The goal of
Day-2 is to provide the necessary functionality for the VNF package to be fully
operational throughout its lifecycle. Day-2 operations include reconfiguration,
monitoring of KPI, and automatic scaling based on the status of KPI.

3.1.5 WireGuard
WireGuard is a secure network tunnel that aims to replace IPsec and most TLS-
based solutions such as OpenVPN while remaining more secure, efficient, and more
straightforward to use [Don18]. It operates solely on Layer 3 of the network stack
and works by associating public keys with IP addresses, and identifying peers strictly
by their public key. As of Linux kernel 5.6, WireGuard will be available by default3 .
Making it easier for network administrators to use the tool for their VPN solutions.

When the sender transmits a packet out the WireGuard interface, it consults the
cryptographic key routing table, shown in Figure 3.6, to determine which public
key to use for encryption. In the example from [Don18], a packet destined for
10.192.122.4 would be encrypted with the TrMv. . .WXX0 public key. The process
also works for receiving packets; if the interface wg0 receives a packet, after decryption
and authentication, it will only accept the packet if the IP resolves in the table to
3 https://github.com/torvalds/linux/commit/bd2463ac7d7ec51d432f23bf0e893fb371a908cd -

Sourced: 2020-02-17
3.1. BACKGROUND THEORY 23

Figure 3.6: The cryptographic key routing table 1a, 1b, and 2a [Don18].

the public key used in the secure session for decrypting it. This process simplifies
firewall rules from an administrative point of view because any packet accepted on
the WireGuard interface will have a reliably authentic source IP.

The initial configuration of WireGuard might seem static, and unfit for roaming
purposes, such as those required in a mobile network environment. However, the
designers of WireGuard took this in mind during the implementation of the secure
network tunnel. The following example highlights how WireGuard solves this issue.
Given the two cryptographic key routing tables 1a and 1b, shown in Figures 3.6, where
the peer identified by the public key gN65. . .z6EA has configured the cryptographic
key routing table to contain an endpoint for the peer HIgo. . .8ykw and changed
the IP and port of the WireGuard interface to 192.95.5.64:21841. The host,
gN65. . .z6EA, sends an encrypted packet to HIgo. . .8ykw at 192.95.5.69:41414.
When HIgo. . .8ykw receives the packet, it learns that the endpoint for sending a reply
is changed. Thus, it updates the table to 1b and ensures that the reply reaches the
new endpoint.

There is a recent trend in proposing 5G private networks with resources dedicated


24 3. BACKGROUND AND RELATED WORK

to a specific enterprise to address critical wireless communication requirements in


the industry. Private 5G networks will enable and foster Industry 4.0 [LKS19].
Lackorzynski et al. investigate and compare various VPN solutions for future
industrial systems. WireGuard is one of the investigated solutions, and the reported
results show that it outperforms IPsec in terms of throughput and latency on
different hardware platforms. In their paper [PWA+ 19], Plaga et al. discuss the use
of WireGuard as an open source solution to secure communications on the transport
layer in terms of confidentiality, integrity, and authenticity for future decentralized
industrial IoT infrastructures.

3.2 Related Work


5G and NFV have seen much progress in the last couple of years. Therefore, it is
beneficial to gather an overview of the current progress. The following are a set of
articles we deem state-of-the-art and related to the work in our thesis.

3.2.1 Towards Service-Oriented 5G: Virtualizing the Networks


for Everything-as-a-Service
In their paper [CZZ+ 18], Chang et al. proposed that every component of the mobile
network infrastructure can be viewed as a service. This requires that the network
infrastructure is virtualized and provided to whoever requires it in a sense that it is
no longer limited to hardware or software, and both hardware and software are owned
by different dedicated network infrastructure operators. Thus, they introduced the
concept as XaaS, depicted in Figure 3.7, which they believe will light the way towards
the design of the service-oriented wireless 5G networks. We discussed this in Section
3.1, where we presented the vertical relationship between InP and their tenants.
However, Chang et al. expand the notion to practically any part of the mobile
network service infrastructure, instead of solely focusing on separating infrastructure
and service.

3.2.2 Dynamic Setup of IPsec VPNs in Service Function


Chaining
Gunleifsen et al. [GKG19] propose the Software Defined Security Associations (SD-
SA) component, designed explicitly for NFV environments, as an alternative to
IPsec-IKE. The SD-SA will be used to automate the establishment of dynamic VPNs
in an NFV domain since Gunleifsen et al. identified a lack of per-flow encryption
in NFV Service Function Chaining. One of the points Gunleifsen et al. made is
that end-users are unable to end-to-end encrypt the traffic passing through multiple
service providers. Gunleifsen et al. previously suggested solutions to this issue
by introducing what they called a hop-by-hop encryption per IP flow, which was
3.2. RELATED WORK 25

Figure 3.7: Everything-as-a-Service, as presented by Chang et al. [CZZ+ 18]

done by deploying an encryption application in front of every VNF within a Service


Function Chain [GK17, GGK18]. However, they identified a problem with such an
architecture which needed a key exchange mechanism that was not dependent on
point-to-point bidirectional communication channels. Gunleifsen et al. showed that
there is a significant need for encryption between VNFs. However, their solution
relied mostly on SDN and did not include an NFV MANO unit such as OSM. Also,
their solution is aimed at being dynamic, while ours currently is more static with
little automation.

3.2.3 WireGuard: Next Generation Kernel Network Tunnel


In his paper from 2017 [Don17], Donenfeld proposed a new secure network tunnel,
operating at the IP network layer. WireGuard aims to replace IPsec and TLS-based
solutions for most use cases, while it demonstrates that it is possible to implement a
secure network tunnel, using state-of-the-art cryptography in less than 4000 lines of
code. Similar to IPsec and OpenVPN, WireGuard’s approach to key distribution is
agnostic and relies on out-of-band mechanisms. At the same time, as it maintains a
small codebase, WireGuard outshines both IPsec and OpenVPN, as shown in Figure
3.8 in terms of latency and throughput. Hence, it will be interesting to see if OSM
can aid in the configuration and key distribution of WireGuard in a modern mobile
26 3. BACKGROUND AND RELATED WORK

Figure 3.8: Bandwidth and latency measurements show how WireGuard performs
against well-established VPN technologies.

network topology.

3.2.4 A Comparative Study on Virtual Private Networks for


Future Industrial Communication Systems
Lackorzynski et al. [LKS19] compared various VPN technologies in an environment
aimed to resemble the future industrial networks, which they stated will be a
combination of new hybrid cloud systems and legacy components. Their throughput
and latency comparisons of both well-established solutions such as OpenVPN and
IPsec, and new upcoming ones such as MACsec and WireGuard — led them to
the conclusion that WireGuard and MACsec, should be preferred in the future
whenever it is possible. It is worth noting that the study investigated solutions
for future industrial networks, and thus favored VPN solutions which performed
better with smaller frame sizes. However, it will not impact the results of the overall
performance measured by the study, and as of 2020, it is the one of few public study
that has made a performance comparison of major VPN solutions against WireGuard
— except Donenfeld himself. That is why we believe the paper highlights WireGuard’s
relevance in a 5G cloud environment as well.

3.2.5 SecMANO: Towards Network Functions Virtualization


(NFV) Based Security MANagement and Orchestration
In their paper [PHMZ16], Pattarantakul et al. present a threat analysis in the context
of NFV based network services, a comprehensive threat taxonomy, summarized in
3.2. RELATED WORK 27

Figure 3.9: Threat taxonomy for Network Function Virtualization layers presented
by Pattarantakul et al. [PHMZ16]. Displaying threats and vulnerabilities in each
layer, as well as relevant attacks and countermeasures.

Figure 3.9, and a conceptual design framework for NFV based security management
and service orchestration illustrated in Figure 3.10, based on ETSI’s NFV MANO
framework. The authors also provide an overview of state-of-the-art with regards
to the NFV MANO framework. They conclude that even though there are several
ETSI NFV MANO implementations, there is not enough focus on NFV security
management and orchestration.
28 3. BACKGROUND AND RELATED WORK

Figure 3.10: Operational flow of the secMANO framework [PHMZ16], shows how
the new security component is appended to the ETSI NFV MANO framework.

3.2.6 Services Orchestration Within 5G Networks — Challenges


and Solutions
Sienkiewicz et al. [SLK18] discuss the trends in service orchestration, as of 2018,
and point out challenges network operators face while implementing orchestration
functionality in their networks. The authors conclude that neither NFV nor SDN will
bring the expected benefits until operators’ OSS/BSS systems are able to support
them. They also stated that a complete MANO system must be provided to ensure
full automation of configuration processes, error management, and invoicing.

3.2.7 Open Multi-Vendor NFV Showcase


During the ETSI NFV Plugtests Programme, a whitepaper [Whi19] was released
which showcase how a set of standardized models and descriptors can be used to create
VNFD and NSD packages which OSM use to instantiate the vEPC network service
depicted in Figure 3.11. The authors highlighted the need for a horizontal NFV
model, as it will significantly improve automation, deployment time, and contribute
3.2. RELATED WORK 29

Figure 3.11: The complete virtualized LTE Packet Core built to showcase a
multi-vendor VNF scenario [Whi19]. VNF packages from NG4T, OpenAirInterface,
Mobileum, and Fortinet were combined to build the network slice which OSM
managed and orchestrated on top of OpenStack.

to higher flexibility. With a horizontal NFV model, it will be possible for operators
to pick NFVs according to their needs from a public trusted repository to build their
network services with a modular approach.

3.2.8 On End-to-End Approach for Slice Isolation in 5G


Networks. Fundamental challenges
By presenting and examining the capabilities and approaches for the realization of
network slicing in 5G, Kotulski et al. [KNS+ 17] propose a set of challenges and
outline recent trends for slice isolation. Kotulski et al. state that in 5G, there will
be a portfolio of isolation technologies available, rather than a single one. Several
technologies that provide network slicing isolation are also presented; among them
are Tag-based, VLAN-based, and VPN-based network slice isolation. Kutolski et al.
also provide a set of definitions of isolation parameters, which should be considered
when isolating network slices. Isolation of traffic will ensure that data does not
flow between slices. Isolation of bandwidth ensures that a slice does not utilize the
bandwidth of a neighboring slice. Isolation of processing is also essential as the
30 3. BACKGROUND AND RELATED WORK

processing of a packet should be independent of all other slices. To accomplish


isolation of storage, data related to a particular slice should be stored separately from
other slices. Kutolski et al. conclude with the proposition that to ensure end-to-end
isolation on a particular strength level and to introduce an adequate security policy,
it is necessary to identify isolation attributes and to create an abstraction layer. The
paper highlights the need for multiple types of isolation techniques in the upcoming
5G network architecture and shows the importance of testing if WireGuard in tandem
with OSM is suitable for VPN-based network slice isolation.

3.2.9 On the Challenges and KPIs for Benchmarking


Open-Source NFV MANO Systems: OSM vs ONAP
In their paper [GMY19], Yilma et al. propose a set of NFV-MANO KPIs. Among
them is the Operational KPIs, which characterize the runtime operations by measuring
the time-latency of an action or task performed by the LifeCycle Management (LCM)
unit. The operational KPIs are depicted in Figure 3.12, and illustrate the process
of on-boarding, deployment, and runtime orchestration of a network service. On-
boarding Process Delay (OPD) is the time it takes to boot-up a virtualized network
function image. Deployment Process Delay (DPD) is the time-delay introduced
by deploying and instantiating the VNF inside the VM, to produce an operational
network service. Runtime Orchestration Delay (ROD) is the delay introduced by
management actions performed during run-time, while Quality of Decision (QOD)
is the MANO system’s effectiveness in scaling and migration of VNFs. Yilma et
al.’s definition of the on-boarding seems to be different from the definition used by
ETSI MANO, as the ETSI MANO group defines it as "[t]he complete onboarding
process implies producing a VNF Package that will be part of the OSM catalogue for
its inclusion in a Network Service" [ETS]. Yilma et al. focus on the time it takes to
boot the machine, not the time it takes to develop the VNFD or upload the package
to OSM’s storage. Thus, it is crucial to know that while we measure the KPIs later
in this thesis, we measure them for the process as defined by Yilma et al., not the
onboarding as it is defined by ETSI MANO.

3.2.10 Providing Strong 5G Mobile Network Slice Isolation for


Highly Sensitive Third-Party Services
Schneider et al. [SMK18] introduce novel provisioning models for third-party slices
and discussed their isolation properties. With their work, Schneider et al. show
that industry verticals utilizing public 5G networks or network slices are unable to
easily enforce isolation and security for their traffic and their devices. They must
instead rely on other parties, such as NSPs, InP, or hardware and software vendors.
In contrast to Kotulski et al., Schneider et al. state that isolation through OTT
does not contribute to resource isolation but can provide confidentiality and integrity
3.3. PREPARATION 31

Figure 3.12: The operations which lead to time-latency in deployment and orches-
tration of network services [GMY19].

protection of the tenant’s traffic. Hence, while disagreeing with the definitions of
isolation, both papers agree on the role OTT plays in securing network slice traffic.

3.3 Preparation
Before beginning the experimentation with both WireGuard and OSM, we allocated
time to gain experience with the tools themselves and the networking setup. Since
we expected to use Canonical’s cloud-init tool with OSM — experiments with
cloud-init were done to understand the potential of the tool and acquire ideas on
how we could leverage it throughout the project. During the same period, we also
experimented with the configuration of interfaces on Ubuntu server images, through
the ip4 tool.

After we decided to go forth with OSM, two weeks were spent assisting Esmaeily et
al. with the integration of the vEPC on the 5GIIK testbed, which is explained in
further detail in [esm20]. The goal was to get to know OSM and OpenStack before
beginning with the WireGuard network service, as well as lay the grounds for the last
iteration of the design cycle — where we wanted to apply the WireGuard network
service to the network slice in which the vEPC existed. Besides the weeks allocated
to assisting with the development of the vEPC, one week was spent at the OSM-MR8
Hackfest5 , hosted by the OSM’s main contributors. The hackfest, which took place
4 https://linux.die.net/man/8/ip - Sourced: 2020-04-12
5 https://osm.etsi.org/wikipub/index.php/OSM-MR8_Hackfest - Sourced: 2020-04-12
32 3. BACKGROUND AND RELATED WORK

remotely during the second week of March, was aimed for network engineers who
needed to use OSM to create network slices. During the hackfest, new features of
OSM were presented, such as Kubernetes-based Network Functions (KNFs), which
we will explain in further detail during the discussion.

Finally, to better understand the state-of-the-art of OSM, we contacted Pål Grønsund,


Vice-Chair of the ETSI OSM project6 , and arranged a meeting to discuss cases which
users of OSM were currently trying to solve. During the meeting, Pål confirmed that
using WireGuard in combination with OSM to create a network service which could
provide VPNaaS was a relevant use case. He also presented some topics of his own,
which included how OSM could be used to manage multi-layer security in a network
where access to subnets should be restricted in a more dynamic sense.

3.4 Summary
In this chapter, we have presented the most relevant background knowledge required
to understand the remainder of the thesis. The focus has mostly been on the essential
technologies for this thesis, such as OSM, NFV, and WireGuard. We have also
summarized state-of-the-art work, which we deem relevant to our project. Much
of the presented work has shown that there is a demand for MANO and NFV in
upcoming 5G networks, and that implementations are currently still being agreed
upon. The related work underlines the point that there is currently a lack of options
and a high potential for security in network slicing.

6 https://osm.etsi.org/about/osm-leaders - Sourced: 2020-04-12


Development, Tests and Results
Chapter

4
To verify if OSM was eligible for creating a VPN network service with WireGuard,
we performed two design cycles with incremental complexity. During the first cycle,
we investigated how Wireguard could be used to create a VPN tunnel between
two Virtual Machines. The purpose of this test was to gain knowledge about how
Wireguard functioned, and which problems might occur while deploying the test on
a larger platform. The second cycle was longer and more complex. The purpose was
to investigate how, and if OSM could manage and orchestrate Wireguard, to create
a VPN tunnel between VNFs in a network slice, and verify if OSM in combination
with WireGuard could be used to provide VPNaaS. Thus, the objective of the second
design cycle was to complete RO2, and provide a basis for answering RQ1, RQ2, and
RQ3.

4.1 Testing WireGuard with Virtual Machines

The objective of the first design cycle was to complete RO1 and gain insight on how
the WireGuard gateways would be launched and handled by OSM. Thus, we had
to create a simple network of VMs, as seen in Figure 4.2, where two VMs would
function as WireGuard gateways, while the rest would emulate other network services
utilizing the VPN tunnel.

Ubuntu Server 18.04 Bionic


Memory 1024 MB
CPU 1 vCPU
Storage 10 GB
Table 4.1: VM flavor for the first design cycle.

33
34 4. DEVELOPMENT, TESTS AND RESULTS

Name Subnet
Data-East 192.168.10.0/24
Data-West 192.168.20.0/24
Tunnel-Network 192.168.30.0/24
Management 192.168.56.0/24
Table 4.2: Network allocation plan during the first design cycle.

Figure 4.1: The network topology used during the first design cycle shows two
WireGuard gateways connecting subnet Data-East and Data-West.

4.1.1 Technical Requirements


The experiment used five VMs with the set up from Table 4.1. Our network topology
used during the experimentation can be found in Table 4.2 and viewed in Figure 4.1.

4.1.2 Network Topology Setup


To complete the setup of our simple WireGuard network topology, we followed a set
of reproducible steps outlined below.

1. Use Virtualbox to create an instance of an Ubuntu Server 18.04 Virtual Machine


with one vCPU, 1024 MB Memory, and 10 GB storage.

2. Assign three interfaces to the instance, one for management, one for data, and
one for the tunnel.

3. Create three new internal subnets, dedicated to Data-East, Data-West, and


the tunnel.

4. Create two clones of the clean installation of US1804.


4.1. TESTING WIREGUARD WITH VIRTUAL MACHINES 35

5. Download and install the WireGuard package to get the official WireGuard
binary and the accompanying tool wg-quick.

6. Enable IPv4 forwarding on the WireGuard VMs and enable proxy ARP, as
shown in Listing 4.1.

7. Create two clones of the WireGuard VM with Virtualbox.

8. Start the two edge VMs (VM-East and VM-West), assign IP addresses, and
enable their interfaces, as shown in Listing 4.2.

9. Start the two WireGuard gateway VMs, assign IP addresses and enable the
interfaces, as shown in Listing 4.3.

10. Start the Management VM, which has WireGuard installed, and assign IP
addresses and enable its respective interfaces. The network should now be
similar to the topology displayed in Figure 4.2.

$ echo net.ipv4.ip_forward=1 | sudo tee -a /etc/sysctl.conf


$ echo net.ipv4.conf.all.proxy_arp=1 | sudo tee -a /etc/sysctl.conf
$ sudo sysctl -p /etc/sysctl.conf

Listing 4.1: Enable forwarding and ARP proxy.

# Configure enp0s8 interface on VM-East


$ ip addr add 192.168.10.101/24 dev enp0s8
$ ip link set dev enp0s8 up

# Configure enp0s8 interface on VM-West


$ sudo ip addr add 192.168.20.202/24 dev enp0s8
$ ip link set dev enp0s8 up

Listing 4.2: Assign IP addresses and configure interfaces.


36 4. DEVELOPMENT, TESTS AND RESULTS

# Configure data interface on WG-EAST


$ ip addr add 192.168.10.11/24 dev enp0s8
$ ip link set dev enp0s8 up

# Configure tunnel interface on WG-EAST


$ ip addr add 192.168.30.11/24 dev enp0s9
$ ip link set dev enp0s9 up

# Configure management interface on WG-EAST


$ ip addr add 192.168.56.11/24 dev enp0s10
$ ip link set dev enp0s10 up

Listing 4.3: Configure interfaces on gateways.

4.1.3 Configuring WireGuard

WireGuard’s design allows the configuration of endpoints to be efficient and straight-


forward. Thus, the original process of enabling Wireguard and configuring WG-East
and WG-West is only a matter of a few commands accompanied by a small configura-
tion file. However, to further simplify and automate the process, we still chose to use
the tool wireguard-mesh-generator, which allowed us to generate configuration
files for all our nodes quickly, and then use wg-quick to finish the setup of the tunnel.
wg-quick handles the additional configuration of the WireGuard interface wg0, such
as adding IP routes, and setting the IP address associated with wg, according to
what is specified in wg0.conf.

To generate the initial configuration files, which are displayed in Listing 4.4 and 4.5,
we used the management node connected to WG-East and WG-West, as illustrated
in Figure 4.2. On the management node, we used the wireguard-mesh-generator
to generate the config-files and keys.
4.1. TESTING WIREGUARD WITH VIRTUAL MACHINES 37

# WG-EAST: wg0.conf
[Interface]
PrivateKey = WGK49XsZKwfEbO0p4AWuq3f/JX1DZ0Eg0hjJ4ZBQoHU=
Address = 10.0.0.1/24
ListenPort = 51820

[Peer]
PublicKey = gjxy6d/S3s7AcV72sAhZsKrQ02ya6HtrfHxueNbrlkQ=
AllowedIPs = 10.0.0.2/32, 192.168.20.0/24
Endpoint = 192.168.30.22:51820

Listing 4.4: WG-East configuration.

# WG-WEST: wg0.conf
[Interface]
PrivateKey = 2L6IQZYq5MvLEX/pLmdnWA1vHg2QvVwxDo+dM02jhmM=
Address = 10.0.0.2/24
ListenPort = 51820

[Peer]
PublicKey = higo9v0fAYbJfCGThRhsfLhsKUeiKAEUgXB73Lt5kwA=
AllowedIPs = 10.0.0.1/32, 192.168.10.0/24
Endpoint = 192.168.30.11:51820

Listing 4.5: WG-West configuration.

To distribute the keys and configuration files to each WireGuard gateway, we used
SSH, shown in Listing 4.6, as it would emulate the same technique as OSM’s proxy
charms use to interact with the VNF in the network it is managing and orchestrating.
The files were named wg0.conf to make the configuration of each gateway more
streamlined, while it is also in line with WireGuard’s interface naming convention.

$ scp wg-east.conf ubuntu@192.168.56.11:wg0.conf


$ scp wg-west.conf ubuntu@192.168.56.22:wg0.conf

Listing 4.6: Distribution of configuration files.

To finish the configuration of WireGuard on each gateway, we ran wg-quick passing


wg0.conf as an argument, which can be seen in Listing 4.7. Wg-quick will then parse
the configuration file, and look for wg-quick specific variables, such as the address,
38 4. DEVELOPMENT, TESTS AND RESULTS

Figure 4.2: The network topology used during the first design cycle, shows the
instantiated WireGuard tunnel between WG-West and WG-East.

and then configure the wg0 interface and add IP routes according to wg0.conf,
before handing on the configuration to WireGuard itself. When the configuration of
WireGuard is complete, the topology will resemble the one in Figure 4.2

# Create a configuration folder for wireguard and move the


,→ configuration file
$ sudo mkdir /etc/wireguard
$ sudo mv wg0.conf /etc/wireguard

# Run wg-quick for WG-EAST


$ sudo wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip address add 10.0.0.1/24 dev wg0
[#] ip link set mtu 1420 dev wg0
[#] ip link set wg0 up
[#] ip route add 10.0.0.2/24 dev wg0
[#] ip route add 192.168.20.1/24 dev wg0

Listing 4.7: Configure WireGuard on the gateway.

Verifying the connectivity and correct forwarding behavior, between the WireGuard
peers, was done by checking that a handshake had taken place, and traffic had been
sent or received, as seen in Listing 4.8. WireGuard allows the administrator to do
this easily with wg show. By starting a ping session between VM-West and VM-East,
one could also see that the number of received and sent bytes increased.
4.2. BUILDING THE WIREGUARD NETWORK SERVICE 39

Host OS Ubuntu 18.04 Bionic


Memory 32 GB
CPU Intel i7-4790
NIC NetXtreme BCM5722
Table 4.3: Hardware and operative system used for the testbed during the second
design cycle.

$ wg show
interface: wg0
public key: gjxy6d/S3s7AcV72sAhZsKrQ02ya6HtrfHxueNbrlkQ=
private key: (hidden)
listening port: 51820

peer: higo9v0fAYbJfCGThRhsfLhsKUeiKAEUgXB73Lt5kwA=
endpoint: 192.168.30.11:51820
allowed ips: 10.0.0.1/32, 192.168.10.0/24
latest handshake: 2 minute, 17 seconds ago
transfer: 221.80 KiB received, 70.35 KiB sent

Listing 4.8: WireGuard connectivity confirmed.

4.2 Building the WireGuard Network Service

During the second design cycle, we performed the primary iteration of our project. The
purpose was to find answers to our research questions by designing and implementing
a VPN tunnel with WireGuard, and see if it could be managed and orchestrated by
OSM to provide VPNaaS.

In Figure 4.3, we show the relationship between the major components of our software
stack. The illustration is based on the ETSI NFV MANO reference architecture
[Com14b] we mentioned in Section 3.1.3, which illustrates the primary components
at play in an NFV/SDN based network.

4.2.1 Technical Requirements


Initially, we planned to do the test on the testbed proposed by Esmaeily et al. [esm20].
However, it was moved to a minimal testbed set up as an emergency solution because
of time constraints. The technical specifications of the new testbed are listed in
Table 4.3.
40 4. DEVELOPMENT, TESTS AND RESULTS

Figure 4.3: The software stack used during the development of the VPN-as-a-Service
Proof of Concept.

4.2.2 Testbed
The testbed is a basic setup of a charmed OSM environment, and the major compo-
nents are displayed in Figure 4.3. The installed VIM was MicroStack1 , a minimal
version of OpenStack, with only the core components installed. To run OSM and the
proxy charms, we used Microk8s, a ligthweight version of Kubernetes, as we stated
in Section 1.8. The commands used during the installation and configuration of the
testbed can be found in Appendix A.1.

4.2.3 Building the Network Service


The process of building the VNF and network service packages was based on the
onboarding guidelines provided by ETSI OSM [ETS]. In the following section, we
provide a set of reproducible steps to build and instantiate the WireGuard network
service. Parts of Figure 3.4 illustrate what is achieved during this section.
1 https://microstack.run/ Sourced: 2020-06-12
4.2. BUILDING THE WIREGUARD NETWORK SERVICE 41

A minimal Ubuntu Server 18.04 image was used, along with the flavor described in
Table 4.1, to create a baseline for the VDU. While logged on to the host with OSM
installed, we created the VNF package through the Command-Line Interface (CLI),
as shown in Listing 4.9.

$ osm package-create --base-directory ~/vWireGuard --image wg-base--


,→ vcpu 1 --memory 1024 --storage 10 --interfaces 3 vnf
,→ vWireGuard

Listing 4.9: Build Virtual Network Function package with OSM.

We replaced the vWireGuard_vnfd.yaml, which is the VNFD that describes our


VNF as we mentioned in Section 3.1.4, with the one in the Appendix A.8. Then we
put the cloud-init file from Appendix A.9 in the folder vWireGuard/vWireGuard_-
vnf/cloud-init.

To create the Juju charm, we prepared the environment by installing charm on the
development platform and set the path variables.

$ snap install charm --classic


$ mkdir -p ~/charms/layers
$ export JUJU_REPOSITORY=~/charms
$ export LAYER_PATH=$JUJU_REPOSITORY/layers
$ cd $LAYER_PATH
$ charm create wg_charm
$ cd wg_charm

Listing 4.10: Install Juju charms and create a skeleton-charm.

We replaced the layer.yaml with Appendix A.3. layer.yaml is responsible for


describing the functional layers which should be present in the charm. Then we put
the content of Appendix A.4 in metadata.yaml. Finally we replaced the content in
actions.yaml, which defined the primitive tasks that will run against our VNFs,
with Appendix A.5. Afterwards, we created a new folder inside wg_charm, named
actions. For each action in Table 4.4, we created a new file inside actions, with
the same name as the table entry, and placed the content of Appendix A.2 in it.

To make all files in the actions directory executable we made use of the script in
Listing 4.11.
42 4. DEVELOPMENT, TESTS AND RESULTS

Initial Config Primitives Description


configure Health check operational status and enable
forwarding on the gateway
generate-keys Uses the WireGuard binary to generate public
and private keys
generate-config Generates the config file for the wg0 interface
wireguard-up Instantiates the service with the given configu-
ration
Config Primitives Description
add-peer Adds a peer to the WireGuard configuration
of the wg0 interface
del-peer Removes a peer from the WireGuard configu-
ration of the wg0 interface
Table 4.4: List of actions/primitives used by the Juju charm framework.

#!/bin/bash
FILES=$LAYER_PATH/wg_charm/actions/*
for f in $FILES
do
chmod +x $f
done

Listing 4.11: All actions in the actions folder must be executable.

We placed the content of Appendix A.10 in the reactive folder, which is where
our own functionality is defined. Then we built the charm, exported it to the VNF
package, and verified that the VNFD was valid. The process is displayed in Listing
4.12.

$ cd ~/charms/layers/wg_charm
$ sudo -E charm build
$ cp -r ~/charms/builds/wg_charm ~/vWireGuard/vWireGuard_vnf/charms/
$ osm package-validate vWireGuard/vWireGuard_vnf/

Listing 4.12: Build and validate the charm.

To create the network service package which includes the VNF, we used to OSM-NBI
package-create, as shown in Listing 4.13.
4.2. BUILDING THE WIREGUARD NETWORK SERVICE 43

$ osm package-create --base-directory ~/vWireGuard ns vWireGuard

Listing 4.13: Create the network service package, base directory.

We replace the vWireGuard_nsd.yaml file with the one in Appendix A.7, and val-
idated the NSD. Then we compressed the packages and upload them to OSM’s
VNF-catalouge. The process is shown in Listing 4.14

$ osm package-validate vWireGuard/vWireGuard_ns/


$ cd ~/vWireGuard
$ tar -cvzf vWireGuard_vnfd.tar.gz vWireGuard_vnf/
$ tar -cvzf vWireGuard_nsd.tar.gz vWireGuard_ns/
$ osm vnfd-create vWireGuard_vnfd.tar.gz
$ osm nsd-create vWireGuard_nsd.tar.gz

Listing 4.14: Validate the network service package and upload it to the osm
platform.

After the VNFD and NSD were onboarded, the OSM GUI displays them as a graph,
seen in Figure 4.4 and 4.5. The graphs show the CPs exposed by the WireGuard
VDU, which combined with the VDU itself constitute the VNF. The same CPs are
displayed in the network service graph, connecting the VNF to the Virtual Link
Descriptors (VLDs) which describe the link between each VNF — creating the
network service. Figure 4.4 also somewhat resembles Figure 3.4, where one of the
network services is a composition of multiple VNFs.

The network service is instantiated with the command shown in Listing 4.15. Where
the gateway IPs are set to 10.0.9.1 and 10.0.9.2. The rest of the configurations
are static, but can easily be overwritten at instantiation time with the use of
additionalParamsForVnf.

Listing 4.15: Instantiate the network service through the OSM-NBI.


$ osm ns-create --ns_name vWireGuard_ns --nsd_name vWireGuard_nsd --
,→ vim_account microstack --config ’{additionalParamsForVnf: [{"
,→ member-vnf-index": "1", additionalParams: { gateway_ip:
,→ 10.0.9.1} }, {"member-vnf-index": "2", additionalParams: {
,→ gateway_ip: 10.0.9.2} } ] }’

When OSM creates an instance of the network service, OSM’s RO component will
instruct the connected VIM to spin up the described NFVI. The outcome of the
44 4. DEVELOPMENT, TESTS AND RESULTS

Figure 4.4: The WireGuard network service is composed of two Virtual Network
Functions connected through Virtual Links and Connection Points. The Connection
Points coloured in blue, are also shown as yellow in Figure 4.5.

Figure 4.5: OSM’s graphical interface shows that the WireGuard Virtual Network
Function is composed of one Virtual Deployment Unit and three Connection Points.
4.2. BUILDING THE WIREGUARD NETWORK SERVICE 45

Figure 4.6: OpenStack’s network topology display shows the two WireGuard
gateways connected to the tunnel network, and the test VMs.

process is the network topology displayed in Figure 4.6. It is worth noting that the
two extra VMs on the image are not deployed by OSM. They were manually created
to emulate machines connected to the subnets networks, protected by the WireGuard
network service. Also, all machines are connected to the management network to
simplify the administrator’s access.

The final step in instantiating the network service is to connect the two WireGuard
gateways. This is done by adding the peer’s public key to the configuration file
for the wg0 interface, along with the peer’s IP on the VPN, the subnet we want to
connect to, and finally the public endpoint connected top the tunnel-network created
by the VIM. The process of using the ns-action to add a peer is shown in Listing
4.16.
46 4. DEVELOPMENT, TESTS AND RESULTS

$ osm ns-action --action_name add-peer --vnf_name 1 --params ’ { peer


,→ -publickey: "/PBK/W4SFp65yKNrF3CfS5obi0tmAh25YNn7MT8Adm4=",
,→ gateway-ip: "10.0.9.2", subnet-behind-tunnel:
,→ "192.168.71.0/24", public-endpoint: "192.168.69.22" } ’
,→ vWireGuard_ns
45b5d327-b399-4951-84ae-3a19d76ac7b5

$ osm ns-action --action_name add-peer --vnf_name 2 --params ’ { peer


,→ -publickey: "ZQixtd9hTHQ1DfVB3OajhsikAM8pypS0jpZ0asHkFHk=",
,→ gateway-ip: "10.0.9.1", subnet-behind-tunnel:
,→ "192.168.70.0/24", public-endpoint: "192.168.69.11" } ’
,→ vWireGuard_ns
d85069fa-8d2e-467d-b02e-8f597cbb8f04

Listing 4.16: Connect the two WireGuard gateways with the use of OSMs ns-action
feature.

4.3 Performance Evaluation


We ran a series of tests to validate the applicability of the WireGuard VPN in
our network service. The first set of tests were based on the Operational KPIs
mentioned in Section 3.2.9, while the second tested the latency and throughput of
our WireGuard network service.

4.3.1 Measuring Service Creation Time


As one of the goals of 5G PPP is to "[reduce] the average service creation time cycle
from 90 hours to 90 minutes." [5G-], measuring our own service creation time was of
interest to us. To measure the service creation time, we viewed the logs from the
LCM, with the command from Listing 4.17, which include timestamps on when a task
or action is initiated and completed. The complete logs can be found in Appendix C.

$ microk8s.kubectl logs -n osm service/lcm-k8s

Listing 4.17: Viewing the LifeCycle Management unit’s logs with microk8s.

A summary of the measured OSM testbed’s KPIs can be found in Table 4.5, which
shows that the network service’s OPD is 159 second, while the DPD ended at 107
seconds. This means that the total time from we initiated our service until it was
ready to provide the required service was 266 seconds. The DPD involved the
4.3. PERFORMANCE EVALUATION 47

NVF-MANO KPI Time


On-boarding Process Delay 159 s
Deployment Process Delay 107 s
Table 4.5: Service creation time.

Action Time
add-peer 60 s
del-peer 51 s
Table 4.6: Network service action primitives and their execution time

action add-peer, but excludes the time it took for the admin to SSH onto the
gateways and extract the public keys — which means that the delay introduced by
the initial-config-primitives was 47 seconds.

4.3.2 Measuring Throughput and Latency


As throughput and latency are of a priority for some of the use cases proposed by
NGMN shown in Table 3.1, we tested WireGuard’s throughput and latency. To
provide context to our measurements, instead of comparing WireGuard against
the internal speeds of a Virtual Network between two VMs, we included a test of
OpenVPN2 . The configurations we used can be found in Appendix B.

To test the throughput of our network service, we used iperf3 between the gateway
nodes and averaged the results over 30 minutes. The tests were first done while
WireGuard was active on the two gateways. Then, we stopped the WireGuard service,
set up an OpenVPN server on one gateway, and used the other as a client, before
we ran iperf between the nodes again. The results illustrated in Figure 4.7, show
that with our configurations, WireGuard has a 5.3 times higher throughput than
OpenVPN.

Measuring latency was done by using the ping4 tool to send 1000 ICMP Requests,
and record the average latency between the gateway and a subnetwork on the opposite
side of the VPN tunnel. As Figure 4.8 illustrates, packets transferred through the
WireGuard VPN have a latency 41 % lower than those from OpenVPN.

2 https://openvpn.net/ - Sourced:2020-05-25
3 https://iperf.fr/iperf-doc.php
- Sourced: 2020-05-27
4 http://www.skbuff.net/iputils/ - Sourced: 2020-05-27
48 4. DEVELOPMENT, TESTS AND RESULTS

Figure 4.7: Throughput of WireGuard and OpenVPN between the gateway nodes,
averaged over 30 minutes with iperf.

WireGuard

OpenVPN

0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00 2.25
Througput (Gbps)

Figure 4.8: Average latency, after 1000 ICMP requests between a gateway and a
peering subnet’s interface on the gateway.

WireGuard

OpenVPN

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Latency (ms)
Chapter
Discussion
5
The following chapter presents our discussion regarding the project. We begin
by elaborating on our thoughts and experiences during the first two design cycles
before we move on to discuss the results from Section 4.3.2. Finally, we iterate on
our proposed OSM-WireGuard lifecycle and present our challenges and limitations
throughout the project.

5.1 Testing WireGuard with Virtual Machines


During the first design cycle, the goal was to complete RO1. We also wanted to
discover WireGuard’s capabilities to create a secure tunnel between two VMs. It is
worth noting that even though the results resemble a site-to-site VPN scenario, it is
done because of how the NFV ecosystem’s vendor lock-in often works. As VNFs are
often pre-compiled before they are published to public repositories, or sold as a product
by the vendor — the only way to configure them is through their exposed API. This
means it depends on the provider if the internals of the VNF can be modified. That
is why we place a VNF in front of the VNF we aim to secure. This might harm the
performance, introduce unwanted complexity, and drive Operating Expenses (OPEX)
up due to increased resource demands, as it requires cloud-based NSPs to rent more
storage, CPU, and RAM than necessary on their cloud platforms to accommodate
the new gateways. To circumvent this problem, NSPs can try to make use of open
source VNFs, which allows them to install WireGuard on the VNFs before packaging
and deploying it. However, distributing the WireGuard VPN functionality across
multiple VNFs, that likely are non-homogeneous in terms of OS version, and virtual
hardware introduces new complexities. As developers must build a service to handle
lifecycle events such as updates, restarts, and reconfiguration of various types of
machines, management, orchestration, and development becomes a complex challenge.
Utilizing cross-platform management and automation tools such as Ansible 1 or Juju
charms will then be necessary to support the non-homogeneous network stack. OSM
1 https://www.ansible.com/ - Sourced: 2020-06-12

49
50 5. DISCUSSION

supports both Ansible playbooks and Juju charms, and using either in combination
with OSM is, therefore a valid option with regards to management, orchestration,
and automation in a non-homogeneous network-stack. Another possibility to limit
OPEX and avoid complexities with the distributed installation, is utilizing KNFs,
which we will come back to in Section 5.2.

The simplicity and efficiency of WireGuard became apparent during the first design
cycle as the simple command-line interface made it easy to understand how most of
the process of launching a tunnel network with WireGuard was going to proceed.
One thing to note is that since WireGuard is designed to be simple, it also becomes
a necessity to script or use external tools in combination with WireGuard to solve
cases with increased complexity. However, the simplicity is an intended feature to
avoid cases such as IPsec, which has myriads of configurations the administrator can
choose from — and therefore increase the probability of introducing vulnerabilities
through misconfigurations. WireGuard itself has a to-do page2 , which includes several
projects which will extend the functionality of WireGuard by providing scripts and
external tools. The development of a tool that allows users to create a dynamic mesh
network is listed on WireGuard’s to-do page. It is written that with WireGuard
as a building block, it is possible to develop a tool where peers can discover each
other, taking previously developed authentication and trust properties included into
consideration. Wg-dynamic3 will be the new tool which includes these features, and
combining it with OSM, will be an exciting topic for future work.

Design decisions regarding key and configuration file distribution, during the first
design cycle, between the management and WireGuard nodes, were done to save
time and were not something we envisioned would be present in the later design
phases. The way it was handled in the first cycle did not follow the best practices of
Public Key Infrastructure (PKI) since both private and public keys were generated
at a single point, and then distributed, as illustrated in Figure 5.1.

However, it helped us understand where each component needed to be generated in


the topology and how it could be done. Hence, it allowed us to come up with the idea
for the next iteration of the design cycle. As Figure 5.2 illustrates, the generation
of keys and configuration files will be handled by Juju proxy charms, which is done
after the instantiation of the VM, which constitutes the VNF, itself. Our use of Juju
proxy charms was discussed in more detail in Section 3.1.

A caveat with regards to the sequence diagram in Figure 5.2 is how public keys
are distributed throughout the network. With the way our design is envisioned for
the second design cycle, it is clear that there has to be some mechanism on each

2 https://www.wireguard.com/todo/ - Sourced: 2020-04-20


3 https://git.zx2c4.com/wg-dynamic - Sourced: 2020-04-20
5.1. TESTING WIREGUARD WITH VIRTUAL MACHINES 51

Figure 5.1: The sequence-diagram shows the process of generating and distributing
configuration files, which include the WireGuard keys, during the first design cycle.

VM, pushing the respective public keys to the management node in an asynchronous
fashion. Authentication and trust should also be handled at the management level,
before distributing keys further onto the network. During the first design cycle, this
was handled manually by the administrator and wireguard-mesh-generator, which
could simultaneously generate keys and establish trust in a local domain. In the
second design cycle, where we will use the OSM-NBI to configure VMs in a VNF,
it will be the task of the network administrator to distribute the keys. This will
lead to a more static and manual approach, more adherent to the original process
of how WireGuard was meant to be used. However, it will be unfeasible to assume
that this manual process could be followed for more than a handful of nodes without
increasing the DPD significantly.
52 5. DISCUSSION

Figure 5.2: The sequence-diagram shows the ideal process of generating and
distributing keys and configuration files.
5.2. VPN-AS-A-SERVICE POC 53

5.2 VPN-as-a-Service PoC

During the design and implementation phase of the WireGuard network service,
we followed the OSM onboarding guidelines. The process is straightforward as it
separates the tasks of deployment, configuration, and management into three days,
as explained in Section 3.1. In the end, we managed to use our results from the
first design cycle, where we addressed RO1, and created a simple PoC VNF which
provided the basic functionality of a WireGuard gateway. We also verified that
it is possible to use OSM to manage and orchestrate a VPN network service with
WireGuard, which was what we wanted to answer in RQ1.

The second design cycle sheds light on various requirements that need to be addressed
for OSM to properly manage and orchestrate a WireGuard VPN network. First of
all, it was clear that OSM, by itself, cannot provide the required functionality to
create a VPN solution with WireGuard. That being said, it is not something we
should have expected of a framework with the ambition of OSM. The goal of OSM is
instead to manage and orchestrate components and their interconnections, which
constitute a more extensive network infrastructure [Com19e]. Thus, it makes sense
that VNFMs, such as Juju proxy charms must be used alongside OSM to handle the
configuration and instantiations of applications on the VDUs. This does not mean
that OSM failed in solving our problem, as the OSM-NBI in combination with Juju
proxy charms allowed us to manage our WireGuard VPN infrastructure through
ns-actions.

The process of instantiating the WireGuard network service now resembles what
we depicted in Figure 5.2. The new process is shown in Figure 5.3, and it includes
the deployment of the underlying infrastructure done by the VIM. The process
begins when the operator uses the OSM-NBI to call ns-create. OSM then logs
the instantiation parameters in its database, and the internal RO unit utilizes the
OpenStack API to deploy the necessary NFVI. When the deployment of the NFVI is
complete, the RO updates the status on OSM’s shared message bus, initiating the
configuration of the VNFs. The Juju controller, which acts as the VCA unit, now
executes the primitives from the initial-config-primitive section to instantiate
the network service. After observing that the VNFs are configured, the operator
extracts the public keys from each VNF and uses it while calling the ns-action
add-peer, initiating the process of adding a peer to the VNFs in WireGuard’s
configuration. At this point, the network service instance is operational and ready to
serve connected network services in the network slice.

With regards to the lifecycle of OSM, Day-1 and Day-2 are the most relevant to
discuss, as they directly impact the configuration of WireGuard in our network service.
Adding VNF configuration with OSM requires that we first define our configuration
54 5. DISCUSSION

Figure 5.3: The process of instantiating the WireGuard network service through
the use of the OSM-NBI.
5.2. VPN-AS-A-SERVICE POC 55

Figure 5.4: The config-primitives of the WireGuard network service.

primitives in the VNFD before we specify them in detail in our proxy charm. Adding
primitives to the VNFD makes the process and oversight of the generated API easy
to verify and manage, as it provides us with a detailed list that can be read as
documentation of the configuration primitives and their instantiation parameters, as
seen in Figure 5.4. OSM’s NBI allows the administrator to configure these primitives
when instantiating the network service through additional configuration files, making
it easier to automate the process from the administrator’s point of view. In our
project’s case, we used the feature to define static IP addresses for our gateways in
the overlay network. Future iterations of this project can use the feature to include
a pre-shared-key generated by WireGuard on the management node to secure the
confidentiality of sensitive traffic better.

Using Juju proxy charms, as we illustrated in Figure 3.5, allowed us to define an


API that streamlined the essential operations required to instantiate the WireGuard
56 5. DISCUSSION

network service. The charms made it easy to expose functions to the administrator
that would let us programmatically define tasks such as key generation, adding
and deleting peers, starting and stopping the WireGuard service, and gathering
performance metrics. In our case, the administrator will use the OSM-NBI to
configure peering relationships between gateways, and due to the Role-Based Access
Control (RBAC) features of OSM access to such administration functionality can
easily be controlled through roles by the MANO platform.

To cut down on the DPD, introduced by the manual key configuration, an automatic
key distribution scheme, and peer discovery should be used to connect peers. Wire-
Guard itself does not provide key distribution, as the architects behind WireGuard
think that WireGaurd’s key distribution should be agnostic, and done by an out-
of-band process, similar to OpenSSH [Don18]. The most obvious way to implement
key distribution and peer discovery would be through adding a central repository
of public keys to the network. The repository would contain tuples of public keys
and the subnet they belong to, while WireGuard gateways will read or write to the
repository.

With a central repository of keys, Access Control (AC) must be discussed, as forgery
of keys can compromise the confidentiality, integrity, and availability of the network.
A scenario where a malicious actor can compromise the integrity of the repository
could, for instance, lead to the compromise of availability where packets are encrypted
with the wrong key. It could also allow the attacker to decrypt traffic successfully
during MiTM attacks and compromise the integrity or confidentiality of the packets.
Thus, some form of AC and trust is essential. Lightweight Directory Access Protocol
(LDAP) would be a fitting protocol to implement as it is designed to provide AC
to distributed services. By pre-installing certificates in the VNFs — read and write
privileges can be controlled to ensure that no unauthorized party gains access to the
repository, allowing an out-of-band process to manage key distribution among peers.

The 10 GB of storage used by our WireGuard VNFs, we specified in Table 4.1,


is worth further discussing. To simplify the development process, we used the 10
GB storage as it would not have any effect on our measurements. These VNFs
should be slimmed down to minimal size when deploying them to production-grade
networks, making them the size of approximately 2 GB 4 . If the amount of interfaces
connected to untrusted networks is high, and the network operator must cut costs,
another alternative to the slimmed-down images would be KNFs. With release
SEVEN5 of OSM, support for Kubernetes has been added. Meaning it will be
possible to use containerized network functions. This can further aid in reducing

4 https://wiki.ubuntu.com/Minimal - Sourced: 2020-06-01


5 https://osm.etsi.org/news-events/news/61-etsi-osm-unveils-release-seven - Sourced: 2020-06-
01
5.3. ANALYSIS DISCUSSION 57

the resource overhead of each network function. However, it is worth noting the
increased complexity of integrating another type of VIM to the infrastructure. As it
essentially becomes a multi-VIM scenario where traffic between VNFs in the original
VIM and the new KNFs might not be as isolated as one first thinks, due to potential
traffic leaks between the two sites. Also, this will potentially require more resources
than using the existing VIM with small VMs and introduce new unnecessary security
risks to the infrastructure.

Another alternative that does not introduce the added complexity of a new VIM
would be to use the WireGuard gateway as a site-to-site VPN instead of a peer-to-
peer VPN. In some cases, this might not be applicable, as multiple tenants could
share the VIM, and the point of using the WireGuard VPN is to provide network
slice isolation from other tenants. Nonetheless, in a scenario where the tenant only
requires confidentiality to outbound services, a single WireGuard VNF can reside on
the edge of the site. To accommodate for the increased load on the single VNF, the
network service provider can make use of OSM’s auto-scaling capabilities, allowing
the VNF’s designer to specify thresholds and scaling actions in the VNFD. OSM’s
Policy Module (POL) and Monitoring Module (MON) units will then create and
configure the alarms attached to the thresholds. If a threshold is crossed, a scaling
action will be triggered.

5.3 Analysis discussion


Our analysis of OSM’s operational KPIs made it clear that OSM combined with
WireGuard can be used to provide VPNaaS while still being in line with NGMN’s
goal of "From 90h to 90min" — thus answering RQ3. The total time OSM used
to instantiate our network service with two VNFs was 266 seconds, and it can be
reduced by about 102 seconds if the operator prepares images used in the VMs with
a pre-installed and up to date version of WireGuard before the on-boarding process.
By pre-configuring the images, it is also possible to omit actions performed by the
Juju charms such as those enabling forwarding. Alternatively, the cloud-init API can
be used to create the public and private keys on each VNF as well. Which would be
faster than using the VCA unit to create proxy charms, SSH onto each VNF, and
instruct it to generate keys. However, as a design decision, it makes more sense to
utilize the Juju charms for the generation of keys, as it allows network services that
require regeneration of keys to move the primitive to the standard config primitives,
making it simple to use the primitive during Day-2 operations.

Table 4.6 shows that performing an action through the OSM-NBI takes an unexpect-
edly large amount of time. One would think that sending one simple instruction to a
VNF through SSH would at most take a couple of seconds, but the measurements
show that it takes up to a minute. The logs show these timestamps as the time the
58 5. DISCUSSION

events were reported back to the LCM from the VCA, that executed the action in
an atomic fashion, which is why it took extra time. To investigate the behaviour of
the VNFs during the ns-action, we logged on to the VNF executed the ns-action
through the OSM-NBI and could see the change in the WireGuard config in less
than five seconds. Which confirmed our suspicion that the action is completed a long
time before the message is shared on OSM’s message bus. The difference might come
from the asynchronous nature of the configuration by the VCA, as it instantiates
a set of containers for each individual VNF, which executes the primitives on the
VNFs. Therefore, there might be several points in the configuration sequence where
the sender must wait for a reply, which introduces the delay. Thus, if one requires
low-latency synchronous messages in, for instance, an URLLC network slice relying
on the timestamps from the communication between the LCM and VCA would
not be sufficient. The timestamps from the events triggered by the RO during the
OPD were, however, more in line with what we recovered from the internal logs of
OpenStack.

Measuring the throughput of WireGuard and OpenVPN gave us results that resembled
those of Figure 3.8. It is worth noting that there is a discrepancy in the results as
the throughput ratio between WireGuard and OpenVPN is higher in our results
than the originals. As WireGuard’s throughput in our results is 5.3 times as high as
OpenVPN’s while it is 3.9 times as high in the original. The reason for the difference
is likely that the throughput between the VNFs is much higher than that of the
testbed in [Don17]. The throughput of our testbed is measured at about 30 Gbps,
while Donenfeld’s was 1 Gbps, which means that the throughput of WireGuard, in
our case, is not bottle-necked by our Network Interface Card (NIC), but more likely
the available vCPU. It is also reasonable to assume that since our OpenVPN server
uses HMAC-SHA512 for message authentication instead of HMAC-SHA2-256, which
was used in by Donenfeld, contributed to the discrepancy.

The same pattern was present in our results from the latency test displayed in Figure
4.8. The latency introduced by WireGuard is lower than that of OpenVPN, but the
ratio is different from what we see in Donenfeld’s tests. Again this difference likely
comes from the available vCPU and the use of message authentication. By performing
a simple speed test with OpenSSL on one of our VNFs, see Table 5.1, where each
hashing algorithm was used to hash as much data as possible over a total of three
seconds with varying from 16 B to 16 KB, we verified that SHA512 was indeed faster
on our VNFs. This adds validity to the assumption that the difference between the
sets of results is partly affected by our difference in message authentication.

The latency added by using WireGuard between two VNFs makes it applicable
for network services or slices. Since the additional latency was only about 150 ms
between the gateways, using it in succession between multiple tenant infrastructures
5.4. OSM-WIREGUARD LIFECYCLE 59

Type 64 B 256 B 1024 B 8192 B 16384 B


SHA512 207543.91k 375815.25k 485403.31k 655351.81k 662459.73k
SHA2-256 182713.19k 325111.72k 413648.55k 452132.86k 448088.70k

Table 5.1: A speed test was performed with openssl to add context to the difference
in latency and throughput between the results.

while service function chaining in a network slice, is reasonable for most services
which do not provide URLLC. However, due to WireGuard’s performance, it is still
more applicable than both IPSec and OpenVPN, and should thus be considered a
prime candidate for URLLC network slices which require network slice isolation by
means of VPN.

5.4 OSM-WireGuard lifecycle


As we have previously discussed, we identified that OSM can be used with WireGuard
to create a VPN-as-a-Service for network providers. The lifecycle for a scenario
where the WireGuard network service is used to provide confidentiality to traffic
passing between two nodes will consist of a set of steps based on the VNF onboarding
guidelines explained in Section 3.1. A simplified version of the process, based on ETSI
OSM’s illustration at [ETS], is shown in Figure 5.5, which portrays the interaction
between components in the stack for each day of the life cycle.

5.4.1 Day-0
The administrator must first retrieve the VNF and network service package, ensure
the hardware requirements are met, upload it to the OSM platform through the
NBI, and make sure that a compatible ubuntu image exists on the connected VIMs.
The administrator will then use the OSM-NBI to launch an instance of the network
service, as explained in Section 4.2.3, and make the VNFs manageable from OSM.

5.4.2 Day-1
During this process, as illustrated by the Day-1 arrow in Figure 5.5, OSM will use
Juju charms, defined by OSM as initial-config-primitives to instantiate the
network service, according to the parameters given at instantiation time. The process
is also depicted in Figure 3.5, and the initial-config-primitive used in our case
can be seen in Figure 5.6. The extra configurations included with –config is used
to specify instantiation parameters, which makes it simpler to effectively launch
the service and customize the network service’s attributes according to the network
service provider’s needs.
60 5. DISCUSSION

Figure 5.5: The operator uses the OSM-NBI to interact with OSM which manages
and orchestrates the network service through a set of operations, categorized as
Day-0, Day-1, and Day-2 operations.

5.4.3 Day-2
When the network service is instantiated, and all services are running, the network
service is ready to be operated at runtime. By this point, the administrator can use
the OSM-NBI and our provided actions add-peer and del-peer to manage peering
relationships between new WireGuard VNFs while running Day-2 operations. During
this time, the OSM KPI monitoring framework can also be used to monitor the
performance of the WireGuard gateways, and if there is a need for more performance,
OSM can scale the gateways accordingly.

$ osm ns-action wireguard --vnf_name [vnf-index] --action_name add-


,→ peer --params ’{peer-publickey: "<value>", gateway-ip: "<value
,→ >", subnet-behind-tunnel: "<value>", public-endpoint: "<value
,→ >"}

Listing 5.1: Configure WireGuard on the gateway


5.4. OSM-WIREGUARD LIFECYCLE 61

Figure 5.6: Initial-config-primitive used by OSM to performe initial configurations


in a programmable and reproducible manner.
62 5. DISCUSSION

Figure 5.7: Network Service Descriptor of the WireGuard network service with
Virtual Link Descriptors.
5.5. APPLICATION OF WIREGUARD IN MOBILE SCENARIOS 63

To further connect the WireGuard network service with other network services,
forming a network slice, the administrator will use the CPs and VLDs, which are part
of the Network Slice Template. With the use of CPs, the WireGuard network service
can easily be integrated with the existing infrastructure to provide confidentiality
where needed. The exposed CPs and VLDs also allow infrastructure designers to
easily create the chain of network services, by adding new networks, which either
exists on the VIM or need to be instantiated, to the VLDs and connect the respective
VNF by using its member index. In Figure 5.7 it is easy to get an overview of the
current infrastructure by looking at the VLDs, and which CPs they are connected to.
Additionally the NFV-MANO IM allows the network service developer to define static
IP addresses to CPs linked to a VLD. Which simplifies the process of infrastructure
operators by providing a clear and reproducible approach to define their infrastructure.

In Figure 5.8 we define the network service’s external CPs which are used to connect
it to other network services in a NST. This way, it is easy to integrate the service
itself into other network slices. Similarly to what is done between VNFs in a network
service.

5.5 Application of WireGuard in mobile scenarios


Even though we have not tested it, we speculate that in a scenario where the network
service or VNF protected by our WireGuard network service migrates to a new
location, our VNFs should manage to follow, and continue to provide network slice
isolation by making use of OpenStack’s live-migration feature and WireGuard’s
ability to handle roaming. The OpenStack live-migration feature was tested in
a paper by Garcia-Aviles et al. [GAGS+ 20], where one of the objectives was to
showcase adaptive allocation of VNFs using a VNF mobility concept. Garcia-Aviles
et al. managed to migrate the VNF without service interruption through either
block live-migration or shared storage live-migration. The downside they discovered
was that block live-migration requires a high bandwidth since the complete image
is transmitted directly, and the shared storage live-migration introduces several
complexities with regards to node placement, security, and added latency.

WireGuard’s inherent roaming features stem from the property of the public key,
which uniquely identifies a WireGuard gateway, and the outer external source IP
of encrypted packets will be used to identify the remote endpoint of peers, allowing
peers to roam between external IP addresses. This comes in handy for NSPs, as
it allows UE connected to a slice to seamlessly change IP while moving between
base stations — if a WireGuard client is installed and configured on the device. An
illustration of this can be seen in Figure 5.9, where a mobile phone has a WireGuard
client installed and is using it to encrypt packets sent to the data center. An obvious
caveat of this solution is enforcing the installation of the WireGuard client on the
64 5. DISCUSSION

Figure 5.8: Network Service Descriptor of the WireGuard network service with
connection-points and ip-profiles.
5.5. APPLICATION OF WIREGUARD IN MOBILE SCENARIOS 65

Figure 5.9: Possible architecture that makes use of the WireGuard VPN-as-a-Service
solution with OSM in 5G networks and beyond.

connected UE. However, using the WireGuard VPN on the UE is likely something
that will only be done by users who are well aware of the benefits, and might need it
due to the sensitivity of their traffic. Additionally, if the mobile phones are part of
an organization’s standard equipment, installation of WireGuard can be achieved
through Mobile Device Management (MDM) by the administrator of the service
[LMS10]. Another option, to avoid installing WireGuard clients on the UE, is to
run a WireGuard VNFs on the gNB, which will at least protect the traffic between
the gNB and the edge data center. This is a more realistic scenario, with regards to
implementations, and will make it simpler for NSPs to manage and orchestrate their
WireGuard VPN network service as they have more control over the versions of the
WireGuard binaries running on each device.

WireGuard’s roaming feature might be seen as a vulnerability that will affect the
network’s availability since an attacker could modify the unauthenticated external
source IP. However, the attacker would not be able to compromise the confidentiality
66 5. DISCUSSION

or integrity of the payload. Thus, it only affects the availability of the network —
while the confidentiality, which is what we aim to preserve, remains unaffected.

A caveat with our design, which is worth mentioning, is that our network service’s
static nature might make it hard to add additional WireGuard gateways to a network
that is already instantiated. This is a scenario we believe should be investigated
further. However, we now hypothesize that adding an additional WireGuard gateway
to an already instantiated WireGuard network service is done by following the steps
in Section 4.2.3. While specifying the required amount of VNFs and making sure that
the tunnel-network defined in the NSD is the same as the one specified in the already
instantiated network service. If that is successful, the only thing which remains is
adding the WireGuard gateways through the OSM-NBI with ns-action.

One point we wish to iterate with regards to our tests is that even though our test was
conducted on a single VIM — OSM allows users to instantiate the network service
across multiple VIMs. This means that in a scenario where VIM 1 is located at Data
Center 1 and VIM 2 is located at Data Center 2, protecting the confidentiality of
the communication passing between the data centers will be done by specifying the
VIM location of each VNF in the WireGuard network service. Thus, we believe our
PoC also applies to multi-VIM infrastructures.

--config ’{vnf: [ {member-vnf-index: "1", vim_account: vim1}, {member


,→ -vnf-index: "2", vim_account: vim2} ], vld: [ {name: datanet,
,→ vim-network-name: {vim1: netVIM1, vim2: netVIM2} } ] }’

Listing 5.2: Multi-VIM network service configuration

5.6 Challenges & Limitations

The original plan for this project was to launch a regular network slice of the type
eMBB on the end-to-end platform 5GIIK built by Esmaeily et al. [esm20], and
then integrate the WireGuard network service with the network slice to show the
modularity and simplicity introduced by NFV. However, due to the end-to-end
testbed not being fully operational by the time our WireGuard network service was
ready, we decided to set up an OSM testbed based on Canonical’s Charmed OSM
platform. The new testbed would allow us to verify most of the same functionality,
which was planned for the original testbed, excluding testing of end-to-end operations
between two VIMs. It also introduced a delay to our plans and shift of focus away
from the original network slice, to just the network service. The reason we chose to
stop focusing on the network slice, was that from our understanding, the operator
would have to include the WireGuard network service in as a connected network
5.6. CHALLENGES & LIMITATIONS 67

service in the NST. Thus, we wanted to prioritize the research questions regarding
OSM’s interaction with WireGuard.

During the setup of the Charmed OSM testbed a bug, which ended up being tracked
here6 , slowed down the development significantly as it required us to redo most of
the configuration of the testbed. When this issue is fixed, the Charmed OSM testbed
will be easy to set up and use, and is thus recommended as a small-scale testbed on
limited hardware resources.

6 https://osm.etsi.org/bugzilla/show_bug.cgi?id=1026
Conclusion and Future Work
Chapter

6
6.1 Future work
Since our work was focused on providing a simple PoC there are still plenty poten-
tial paths available when it comes to utilizing OSM to manage and orchestrate a
WireGuard VPN network service and to provide network slice isolation.

6.1.1 WireGuard Network Slice Implementation


As network slice isolation is part of the goal in our thesis — investigating the
applicability of our WireGuard network service in a network slice should be prioritized.
Besides the network slice tested by Esmaeily et al., there are several network slice
templates1,2,3 which the reader can attempt to include our network service in.

6.1.2 Automatic Key-Distribution


Designing an architecture for the VPNaaS that includes automatic key-distribution
among connected peers through a central public key repository is also a valid option
for future work. It would entail research of appropriate database technologies, AC
methods, and secure key-distribution protocols. We previously suggested LDAP as
part of the solution.

6.1.3 Multi-VIM on an end-to-end platform, 5GIIK


As mentioned in Chapter 5, it is possible to specify the VIM on which a network
function should be instantiated. Therefore future work can include experiments
where the network service is tested on a multi-VIM infrastructure, to better align
with realistic mobile infrastructure scenarios of core and edge data centers. We
1 https://github.com/facebookincubator/magma - Sourced: 2020-06-15
2 OSM 6th Hackfest - Sourced: 2020-06-15
3 OSM 9th Hackfest - Sourced: 2020-06-15

69
70 6. CONCLUSION AND FUTURE WORK

believe this work will be done in the next iteration of this project on the 5GIIK
testbed which is hosted in Gjøvik and Trondheim.

6.1.4 WireGuard Network Service Performance Limitations


In our tests we used a single node in the subnet connected to each WireGuard gateway.
From a performance point of view, it would therefore be of interest to experiment
with different numbers of connected devices in a site-to-site design, to see the limits
of what the WireGuard gateway is capable of. Similarly it would be interesting to
document how the network service behaves if the number of WireGuard gateways
is controlled by OSM’s monitoring and auto-scaling feature to accommodate the
increasing number of connected devices. Trade-offs between site-to-site design and
peer-to-peer where each device in the subnet or VIM is assigned a WireGuard gateway
instead. This would obviously mean that the MNO must make trade-offs with regards
to complexity or performance, because of the increased number of connected nodes
in a peer-to-peer network. However, using a single gateway introduces the possibility
of a single point of failure, which if not properly handled might introduce several new
vulnerabilities. It would also be a matter of the amount of resources available on
the VIM, as instantiating multiple WireGuard gateway instances will imply that the
MNO should take care to use a VDU with as little a resource footprint as possible.
This can be combined with the use of KNFs which will use Kubernetes instead of
OpenStack as a VIM, and reduce the amount of resources per gateway by using
containers instead of VMs.

6.1.5 VNF Migration Without Service Interruption


Previously we mentioned that OSM would be able to keep up with VNFs which
migrated to other locations in the mobile network. We believe this is a path worth
exploring to fully enable the WireGuard network service to operate. Combining the
live-migration feature with a public key repository might also be a project worth
investigating. If the new wg-dynamic feature mentioned in Section 5.1 is finished,
this might also be an interesting path to explore.

6.2 Conclusion
This thesis touched upon the management and orchestration of network services
in 5G, focusing on utilizing WireGuard to provide network slice isolation, and
presenting a PoC network service. We believe we were partially able to confirm
our initial hypothesis "that OSM can be used to manage and orchestrate the secure
communication between VNFs in a mobile network, where VNFs are instantiated
and re-locate at a higher pace than previous mobile networks". Since we reserved
the validation of live-migration for future work, we can not say that it has been
6.2. CONCLUSION 71

fully confirmed that the WireGuard network service will function properly in such
a scenario. However, we can confirm our second hypothesis statement "that with
the aid of OSM and WireGuard; it is possible to create an overlay network that
can contribute to network slice isolation in a 5G mobile network". We did so by
designing a VPN network service with WireGuard and OSM, which encrypted traffic
between VNFs at the edges of the network service. With our own VNFD and NSD
we built a simple PoC which could be placed between network services in a NST
providing network slice isolation. Configuring the WireGuard service running on
each VNF was done through our Juju proxy charms. The charms API allowed us to
programmatically declare operations on the VNF needed to instantiate and configure
the WireGuard network service properly. We verified the applicability of WireGuard
in our network service by testing its latency and throughput, which showed that it
outshines OpenVPN in terms of performance.

Additionally, we discussed WireGuard’s applicability in mobile networks due to its


inherent roaming features. By showcasing WireGuard’s simplicity and efficiency,
we showed that it is a strong contender for OTT isolation of network slices, which
provide low latency or high throughput communication. By measuring the MANO
KPIs, we also verified that OSM can instantiate our network service within the goal
of the 5G PPP.

Along with the PoC we provided a step-by-step instruction to re-create our WireGuard
network service, as well as the source4 for the VNF and network service package, and
discussed how it could be iterated upon to build a fully automatic VPNaaS solution.

By designing and building a network service which provides network isolation between
VNFs, and proving the simplicity in combining WireGuard and OSM, we have
hopefully contributed to driving telecommunication operators towards being more
open for multi-vendor NFV solutions and provided NSPs and InPs with the additional
tools to reduce or mitigate threat vectors in 5G.

4 https://github.com/simehag/wg-osm-ns
References

[5G-] 5G-PPP, https://5g-ppp.eu/about-us/. Vision and Mission. Accessed: 2020-05-


25.

[All15] NGMN Alliance. 5G whipe paper. Standard Issue 1, Next Generation Mobile
Networks, 2015.

[ATS+ 18] I. Afolabi, T. Taleb, K. Samdanis, A. Ksentini, and H. Flinck. Network Slicing
and Softwarization: A Survey on Principles, Enabling Technologies, and Solutions.
IEEE Comm Surv Tut, 20:2429–2453, 2018.

[CKS+ 17] I. P. Chochliouros, A. Kostopoulos, A. S. Spiliopoulou, A. Dardamanis, I. Neokos-


midis, T. Rokkas, and L. Goratti. Business and market perspectives in 5G
networks. In 2017 Internet of Things Business Models, Users, and Networks,
pages 1–6, 2017.

[Com12] ETSI Community. Network Function Virtualisation. Standard Issue 1, European


Telecommunications Standards Institute, Sophia Antipolis, France, 2012.

[Com14a] ETSI Community. Network Functions Virtualisation (NFV); Architectural Frame-


work. Standard v1.2.1, European Telecommunications Standards Institute, Sophia
Antipolis, France, 2014.

[Com14b] ETSI Community. Network Functions Virtualisation (NFV); Management and Or-
chestration. Standard v1.1.1, European Telecommunications Standards Institute,
Sophia Antipolis, France, 2014.

[Com15] ETSI Community. Network Functions Virtualisation (NFV); Infrastructure


Overview. Standard v1.1.1, European Telecommunications Standards Institute,
Sophia Antipolis, France, 2015.

[Com19a] ETSI Community. ETSI GS NFV-IFA 014. Standard 3.3.1, European Telecom-
munications Standards Institute, Sophia Antipolis, France, 2019.

[Com19b] ETSI Community. Network Functions Virtualisation (NFV); Management and Or-
chestration; Functional Requirements Specification. Standard Release 3, European
Telecommunications Standards Institute, Sophia Antipolis, France, 2019.

73
74 REFERENCES

[Com19c] ETSI Community. Network Functions Virtualisation (NFV) Release 3; Manage-


ment and Orchestration; VNF Descriptor and Packaging Specification. Stan-
dard 3.3.1, European Telecommunications Standards Institute, Sophia Antipolis,
France, 2019.

[Com19d] ETSI Community. OSM Release FIVE Technical Overview. Standard RELEASE
FIVE, European Telecommunications Standards Institute, Sophia Antipolis, FR,
2019.

[Com19e] ETSI Community. OSM Scope, Functionality, Operation and Integration Guide-
lines. Standard Issue 1, European Telecommunications Standards Institute, Sophia
Antipolis, FR, 2019.

[CZZ+ 18] Z. Chang, Z. Zhou, S. Zhou, T. Chen, and T. Ristaniemi. Towards service-
oriented 5g: Virtualizing the networks for everything-as-a-service. IEEE Access,
6:1480–1489, 2018.

[Don17] Jason A. Donenfeld. Wireguard: Next generation kernel network tunnel. 01 2017.

[Don18] Jason A. Donenfeld. Wireguard: Next generation kernel network tunnel. Technical
Report revision 416d63b, 2018.

[esm20] A Cloud-based SDN/NFV Testbed for End-to-End Network Slicing in 4G/5G,


author=Esmaeily, Ali and Kralevska, Katina and Gligoroski, Danilo. arXiv
preprint arXiv:2004.10455, 2020.

[ETS] ETSI OSM, https://osm.etsi.org/docs/vnf-onboarding-guidelines/. OSM VNF


Onboarding. Accessed: 2020-04-20.

[FKGG19] M. K. Forland, K. Kralevska, M. Garau, and D. Gligoroski. Preventing DDoS


with SDN in 5G. In 2019 IEEE Globecom Workshops (GC Wkshps), pages 1–7,
2019.

[Før19] Mathias Kjølleberg Førland. SDN-a crucial security component towards 5G.
Master’s thesis, NTNU, 2019.

[GAGS+ 20] Gines Garcia-Aviles, Marco Gramaglia, Pablo Serrano, Francesco Gringoli, Ser-
gio Fuente-Pascual, and Ignacio [Labrador Pavon]. Experimenting with open
source tools to deploy a multi-service and multi-slice mobile network. Computer
Communications, 150:1 – 12, 2020.

[GGK18] Håkon Gunleifsen, Vasileios Gkioulos, and Thomas Kemmerich. A tiered control
plane model for service function chaining isolation. Future Internet, 10(6), 2018.

[GK17] H. Gunleifsen and T. Kemmerich. Security requirements for service function


chaining isolation and encryption. In 2017 IEEE 17th International Conference
on Communication Technology (ICCT), pages 1360–1365, 2017.

[GK19] Danilo Gligoroski and Katina Kralevska. Expanded combinatorial designs as tool
to model network slicing in 5g. IEEE Access, 7:54879–54887, 2019.
REFERENCES 75

[GKG19] Håkon Gunleifsen, Thomas Kemmerich, and Vasileios Gkioulos. Dynamic setup
of ipsec vpns in service function chaining. Computer Networks, 160:77 – 91, 2019.

[GMY19] Vincenzo Sciancalepore Xavier Costa-Perez Girma M. Yilma, Faqir Zarrar Yousaf.
On the Challenges and KPIs for Benchmarking Open-Source NFV MANO Systems:
OSM vs ONAP. Arxiv, 2019.

[Gro16] NGMN 5G Security Group. 5g security recommendations package 2: Network


slicing. April 2016.

[HCP18] Dijiang Huang, Ankur Chowdhary, and Sandeep Pisharody. SDN and NFV
Security: From Theory to Practice, pages 127–149. CRC Press, 12 2018.

[JSS+ 14] Manar Jammal, Taranpreet Singh, Abdallah Shami, Rasool Asal, and Yiming Li.
Software defined networking: State of the art and research challenges. Computer
Networks, 72:74 – 98, 2014.

[KGFG19] Katina Kralevska, Michele Garau, Mathias Førland, and Danilo Gligoroski.
Towards 5G Intrusion Detection Scenarios with OMNeT++. In Proceedings of 6th
International OMNeT++ Community Summit 2019, volume 66 of EPiC Series
in Computing, pages 44–51. EasyChair, 2019.

[KNS+ 17] Z. Kotulski, T. Nowak, M. Sepczuk, M. Tunia, R. Artych, K. Bocianiak, T. Osko,


and J. Wary. On end-to-end approach for slice isolation in 5g networks. fun-
damental challenges. In 2017 Federated Conference on Computer Science and
Information Systems (FedCSIS), pages 783–792, 2017.

[LKS19] T. Lackorzynski, S. Köpsell, and T. Strufe. A comparative study on virtual


private networks for future industrial communication systems. In 2019 15th IEEE
International Workshop on Factory Communication Systems (WFCS), pages 1–8,
2019.

[LMS10] L. Liu, R. Moulic, and D. Shea. Cloud service portal for mobile device management.
In 2010 IEEE 7th International Conference on E-Business Engineering, pages
474–478, 2010.

[MLM+ 19] L. Mamushiane, A. A. Lysko, T. Mukute, J. Mwangama, and Z. D. Toit. Overview


of 9 open-source resource orchestrating etsi mano compliant implementations: A
brief survey. In 2019 IEEE 2nd Wireless Africa Conference (WAC), pages 1–7,
2019.

[MSG+ 16] R. Mijumbi, J. Serrat, J. Gorricho, N. Bouten, F. De Turck, and R. Boutaba.


Network Function Virtualization: State-of-the-Art and Research Challenges.
IEEE Communications Surveys Tutorials, 18(1):236–262, 2016.

[OAL+ 17] J. Ordonez-Lucena, P. Ameigeiras, D. Lopez, J. J. Ramos-Munoz, J. Lorca, and


J. Folgueira. Network Slicing for 5G with SDN/NFV: Concepts, Architectures,
and Challenges. IEEE Communications Magazine, 55(5):80–87, May 2017.
76 REFERENCES

[PHMZ16] M. Pattaranantakul, R. He, A. Meddahi, and Z. Zhang. SecMANO: Towards


Network Functions Virtualization (NFV) Based Security MANagement and Or-
chestration. In 2016 IEEE Trustcom/BigDataSE/ISPA, pages 598–605, Aug
2016.
[PWA+ 19] Sven Plaga, Norbert Wiedermann, Simon Duque Anton, Stefan Tatschner, Hans
Schotten, and Thomas Newe. Securing future decentralised industrial iot in-
frastructures: Challenges and free open source solutions. Future Generation
Computer Systems, 93:596 – 608, 2019.
[RSI+ 19] B. Rusti, H. Stefanescu, M. Iordache, J. Ghenta, C. Patachia, P. Gouvas,
A. Zafeiropoulos, E. Fotopoulou, Q. Wang, and J. A. Calero. 5g smart city
vertical slice. In 2019 IFIP/IEEE Symposium on Integrated Network and Service
Management (IM), pages 13–19, 2019.
[SLK18] K. Sienkiewicz, W. Latoszek, and P. Krawiec. Services orchestration within 5g
networks — challenges and solutions. In 2018 Baltic URSI Symposium (URSI),
pages 265–268, May 2018.
[SMK18] P. Schneider, C. Mannweiler, and S. Kerboeuf. Providing strong 5g mobile network
slice isolation for highly sensitive third-party services. In 2018 IEEE Wireless
Communications and Networking Conference (WCNC), pages 1–6, 2018.
[Whi19] Whitestack. Open Multi-Vendor NFV Showcase. Technical Report 1st Edition,
ETSI, 2019.
[Wie14a] Roel J. Wieringa. Single-Case Mechanism Experiments, pages 247–267. Springer
Berlin Heidelberg, Berlin, Heidelberg, 2014.
[Wie14b] Roel J. Wieringa. What Is Design Science? Springer Berlin Heidelberg, Berlin,
Heidelberg, 2014.
[ZGF+ 18] A. Zafeiropoulos, P. Gouvas, E. Fotopoulou, G. Tsiolis, T. Xirofotos, J. Bonnet,
G. Carrozzo, S. Rizou, A. Gavras, M. J. Barros, X. Costa-Perez, A. Prasad,
M. Gramaglia, A. Tzanakaki, D. Simeonidou, J. Cosmas, M. Fallgren, R. Muñoz,
and R. Vilalta. Enabling vertical industries adoption of 5g technologies: A
cartography of evolving solutions. In 2018 European Conference on Networks and
Communications (EuCNC), pages 1–9, 2018.
Appendix
VPN-as-a-Service PoC
A
Listing A.1: testbed setup
sudo snap install juju --classic
sudo snap install osmclient --beta
sudo snap connect osmclient:juju-client-observe
sudo snap connect osmclient:ssh-public-keys
sudo snap connect osmclient:network-control
juju bootstrap localhost osm-lxd
sudo snap install microk8s --classic
sudo usermod -a -G microk8s $USER
newgrp microk8s
microk8s.status --wait-ready
microk8s.enable storage dns
sudo snap install microstack --classic --beta
sudo microstack.init --auto
wget https://cloud-images.ubuntu.com/bionic/current/bionic-server-
,→ cloudimg-amd64.img
microstack.openstack image create --public --disk-format qcow2 --
,→ container-format bare --file bionic-server-cloudimg-amd64.img
,→ ubuntu1804
juju bootstrap microk8s osm-k8s
juju add-model osm
osmclient.overlay
export VCA_APIPROXY=‘cat vca-overlay.yaml | grep vca_apiproxy | awk ’
,→ {print $2}’‘
export VCA_HOST=‘cat vca-overlay.yaml | grep vca_host | sort -u | awk
,→ ’{print $2}’‘
sudo apt install netfilter-persistent -y
sudo iptables -t nat -A PREROUTING -p tcp -m tcp -d $VCA_APIPROXY --
,→ dport 17070 -j DNAT --to-destination $VCA_HOST

77
78 A. VPN-AS-A-SERVICE POC

sudo netfilter-persistent save


juju deploy osm --overlay vca-overlay.yaml
while [ "$(juju status | grep kubernetes | grep -c "active")" != "12"
,→ ];
do
sleep 5;
done
export OSM_HOSTNAME=‘juju status nbi-k8s | grep kubernetes | awk ’{
,→ print $9}’‘
echo "export OSM_HOSTNAME=$OSM_HOSTNAME" >> ~/.bashrc
source ~/.bashrc
sudo snap alias osmclient.osm osm
osm vim-create --name microstack \
--user admin \
--password keystone \
--auth_url http://$VCA_APIPROXY:5000/v3 \
--tenant admin \
--account_type openstack \
--config=’{security_groups: default,
keypair: microstack,
project_name: admin,
user_domain_name: default,
region_name: microstack,
insecure: True,
availability_zone: nova,
version: 3,
use_floating_ip: true}’
79

Listing A.2: action.py


#!/usr/bin/env python3
import sys
sys.path.append(’lib’)

from charms.reactive import main


from charms.reactive import set_state
from charmhelpers.core.hookenv import action_fail, action_name

"""
‘set_state‘ only works here because it’s flushed to disk inside the ‘
,→ main()‘
loop. remove_state will need to be called inside the action method.
"""
set_state(’actions.{}’.format(action_name()))

try:
main()
except Exception as e:
action_fail(repr(e))

Listing A.3: layer.yaml


"includes":
- "layer:options"
- "layer:basic"
- "layer:sshproxy"
- "layer:vnfproxy"
"exclude": [".travis.yml", "tests", "tox.ini", "test-requirements.txt
,→ "]
"options":
"basic":
"use_venv": !!bool "false"
"packages": []
"python_packages": []
"include_system_packages": !!bool "false"
"sshproxy": {}
"vnfproxy": {}
"wg_charm": {}
"is": "wg_charm"
80 A. VPN-AS-A-SERVICE POC

Listing A.4: metadata.yaml


"name": "wg_charm"
"summary": "wireguard charm"
"maintainer": "Simen"
"description": |
This is an example of a proxy charm deployed by Open Source Mano.
"tags":
# Replace "misc" with one or more whitelisted tags from this list:
# https://jujucharms.com/docs/stable/authors-charm-metadata
- "misc"
- "osm"
- "vnf"
- "nfv"
"series":
- "bionic"
- "trusty"
- "xenial"
"subordinate": !!bool "false"

Listing A.5: actions.yaml


"run":
"description": "Run an arbitrary command"
"params":
"command":
"description": "The command to execute."
"type": "string"
"default": ""
"required":
- "command"
"generate-ssh-key":
"description": "Generate a new SSH keypair for this unit. This will
,→ replace any\
\ existing previously generated keypair."
"verify-ssh-credentials":
"description": "Verify that this unit can authenticate with server
,→ specified by\
\ ssh-hostname and ssh-username."
"get-ssh-public-key":
"description": "Get the public SSH key for this unit."
"start":
"description": "Stop the service on the VNF."
81

"stop":
"description": "Stop the service on the VNF."
"restart":
"description": "Stop the service on the VNF."
"reboot":
"description": "Reboot the VNF virtual machine."
"upgrade":
"description": "Upgrade the software on the VNF."
"configure":
"description": "Touch a file on the VNF."
"required":
- "filename"
"generate-keys":
"description": "Touch a file on the VNF."
"generate-config":
"description": "Touch a file on the VNF."
"wireguard-up":
"description": "Touch a file on the VNF."
"add-peer":
"description": "Run an arbitrary command"
"params":
"peer-publickey":
"description": "The command to execute."
"type": "string"
"default": ""
"gateway-ip":
"description": "The command to execute."
"type": "string"
"default": ""
"subnet-behind-tunnel":
"description": "The command to execute."
"type": "string"
"default": ""
"required":
- "peer-publickey"
- "gateway-ip"
- "subnet-behind-tunnel"
"del-peer":
"description": "Run an arbitrary command"
"params":
"peer-publickey":
82 A. VPN-AS-A-SERVICE POC

"description": "The command to execute."


"type": "string"
"default": ""
"subnet-behind-tunnel":
"description": "The command to execute."
"type": "string"
"default": ""
"required":
- "peer-publickey"
- "subnet-behind-tunnel"

Listing A.6: parameters.yaml


additionalParamsForVnf:
- member-vnf-index: ’1’
additionalParams:
gateway_ip: ’10.0.9.1’
- member-vnf-index: ’2’
additionalParams:
gateway_ip: ’10.0.9.2’

Listing A.7: vWireGuard_nsd.yaml


nsd:nsd-catalog:
nsd:
- id: vWireGuard_nsd
name: vWireGuard_nsd
short-name: vWireGuard_nsd
description: Generated by OSM package generator
vendor: OSM
version: ’1.0’

constituent-vnfd:
- member-vnf-index: 1
vnfd-id-ref: vWireGuard_vnfd
- member-vnf-index: 2
vnfd-id-ref: vWireGuard_vnfd
ip-profiles:
- name: tunnel-net
description: network for the wg tunnel
ip-profile-params:
ip-version: ipv4
subnet-address: 192.168.69.0/24
83

dhcp-params:
enabled: true
- name: data-net-east
description: network for the wg data
ip-profile-params:
ip-version: ipv4
gateway-address: 192.168.70.1
subnet-address: 192.168.70.0/24
dhcp-params:
enabled: true
- name: data-net-west
description: network for the wg data
ip-profile-params:
ip-version: ipv4
gateway-address: 192.168.71.1
subnet-address: 192.168.71.0/24
dhcp-params:
enabled: true

connection-point:
- name: nsd_cp_mgmt
vld-id-ref: nsd_vnfd_vld_mgmt
- name: nsd_cp_data1
vld-id-ref: nsd_vnfd_vld_data1
- name: nsd_cp_data2
vld-id-ref: nsd_vnfd_vld_data2

vld:
- id: management
name: management
short-name: management
type: ELAN
mgmt-network: ’true’
vnfd-connection-point-ref:
- member-vnf-index-ref: 1
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-mgmt
- member-vnf-index-ref: 2
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-mgmt
- id: tunnel
84 A. VPN-AS-A-SERVICE POC

name: tunnel
short-name: tunnel
type: ELAN
ip-profile-ref: tunnel-net
vnfd-connection-point-ref:
- member-vnf-index-ref: 1
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-tunnel
ip-address: 192.168.69.11
- member-vnf-index-ref: 2
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-tunnel
ip-address: 192.168.69.22
- id: data-east
name: data-east
short-name: data-east
type: ELAN
ip-profile-ref: data-net-east
vnfd-connection-point-ref:
- member-vnf-index-ref: 1
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-data
ip-address: 192.168.70.1
- id: data-west
name: data-west
short-name: data-west
type: ELAN
ip-profile-ref: data-net-west
vnfd-connection-point-ref:
- member-vnf-index-ref: 2
vnfd-id-ref: vWireGuard_vnfd
vnfd-connection-point-ref: vnf-data
ip-address: 192.168.71.1

Listing A.8: vWireGuard_vnfd.yaml


vnfd:vnfd-catalog:
vnfd:
- id: vWireGuard_vnfd
name: vWireGuard_vnfd
short-name: vWireGuard_vnfd
description: WireGuard Gateway
85

vendor: OSM
version: ’1.0’

# Management interface
mgmt-interface:
cp: vnf-mgmt
# To create IP subnets of your own choosing use the section
,→ below
# Combine it with vld descriptors.
# Atleast one VDU need to be specified
vdu:
# Additional VDUs can be created by copying the
# VDU descriptor below
- id: vWireGuard_vnfd-VM
name: vWireGuard_vnfd-VM
description: vWireGuard_vnfd-VM
count: 1

# Flavour of the VM to be instantiated for the VDU


vm-flavor:
vcpu-count: 1
memory-mb: 1024
storage-gb: 10

# Image including the full path


image: ’ubuntu1804’
cloud-init-file: wg-cloud-init

interface:
# Specify the external interfaces
# There can be multiple interfaces defined
- name: eth0
type: EXTERNAL
virtual-interface:
type: PARAVIRT
external-connection-point-ref: vnf-mgmt
mgmt-interface: true
- name: eth1
type: EXTERNAL
virtual-interface:
type: PARAVIRT
86 A. VPN-AS-A-SERVICE POC

external-connection-point-ref: vnf-data
- name: eth2
type: EXTERNAL
virtual-interface:
type: PARAVIRT
external-connection-point-ref: vnf-tunnel
monitoring-param:
- id: "wg_cpu_util"
nfvi-metric: "cpu_utilization"
- id: "wg_memory_util"
nfvi-metric: "average_memory_utilization"
# The connection points which are exposed to the Network
,→ Service
connection-point:
- name: vnf-mgmt
type: VPORT
- name: vnf-data
type: VPORT
- name: vnf-tunnel
type: VPORT

vnf-configuration:

# To configure the VNF, OSM will connect to the global


,→ management connectionpoint.

# These actions are run automatically when the vnf is


,→ instantiated;
# Initial primitives must include a primitive named config
,→ that passes information for OSM VCA
# to be able to authenticate and run further primitives
,→ into the VNF.
# The config primitive should provide, at least, the
,→ following parameters:
initial-config-primitive:
- seq: ’1’
name: config
parameter:
- name: ssh-hostname
value: <rw_mgmt_ip> # Auto populated
- name: ssh-username
87

value: ubuntu
- seq: ’2’
name: enable-forwarding
- seq: ’3’
name: configure-interfaces
- seq: ’4’
name: generate-keys
- seq: ’5’
name: generate-config
parameter:
- name: gateway-ip
data-type: STRING
value: <gateway_ip> # Defined in parameters.yaml
- seq: ’6’
name: start-wg
config-primitive:
- name: add-peer
parameter:
- name: peer-publickey
data-type: STRING
default-value: ’’
- name: gateway-ip
data-type: STRING
default-value: ’10.0.9.1’
- name: subnet-behind-tunnel
data-type: STRING
default-value: ’192.168.0.0/16’
- name: public-endpoint
data-type: STRING
default-value: ’192.168.30.1/24’
- name: del-peer
parameter:
- name: peer-publickey
data-type: STRING
default-value: ’null’
- name: subnet-behind-tunnel
data-type: STRING
default-value: ’192.168.0.0/16’
juju:
charm: wg_charm
monitoring-param:
88 A. VPN-AS-A-SERVICE POC

- id: "wg_vnf_cpu_util"
name: "wg_vnf_cpu_util"
aggregation-type: AVERAGE
vdu-monitoring-param:
vdu-ref: "vWireGuard_vnfd-VM"
vdu-monitoring-param-ref: "wg_cpu_util"

- id: "wg_vnf_memory_util"
name: "wg_vnf_memory_util"
aggregation-type: AVERAGE
vdu-monitoring-param:
vdu-ref: "vWireGuard_vnfd-VM"
vdu-monitoring-param-ref: "wg_memory_util"

Listing A.9: wg-cloud-init


#cloud-config
password: osm4u
chpasswd: { expire: False }
ssh_pwauth: True
apt:
preserve_sources_list: true
sources:
wireguard:
source: "ppa:wireguard/wireguard"
package_upgrade: true
packages:
- wireguard
write_files:
- content: |
network:
version: 2
renderer: networkd
ethernets:
ens3:
dhcp4: true
ens4:
dhcp4: true
ens5:
dhcp4: true
owner: root:root
permissions: ’0644’
89

path: /etc/netplan/config.yaml

Listing A.10: wg_charm.py


from charmhelpers.core.hookenv import (
action_get,
action_fail,
action_set,
status_set,
)
from charms.reactive import (
clear_flag,
set_flag,
when,
when_not,
)
import charms.sshproxy

@when(’sshproxy.configured’)
@when_not(’simple.installed’)
def install_simple_proxy_charm():
set_flag(’simple.installed’)
status_set(’active’, ’Ready!’)

@when(’actions.config’)
def touch():
err = ’’
try:
filename = action_get(’filename’)
cmd = [’touch {}’.format(filename)]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo sysctl -w net.ipv4.ip_forward=1’]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo sysctl -w net.ipv4.conf.all.proxy_arp=1’]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo ip link set ens4 up’]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo ip link set ens5 up’]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo netplan apply’]
result, err = charms.sshproxy._run(cmd)
except:
90 A. VPN-AS-A-SERVICE POC

action_fail(’command failed:’ + err)


else:
action_set({’outout’: result})
finally:
clear_flag(’actions.config’)

@when(’actions.generate-keys’)
def generate_keys():
err = ’’
try:
cmd = [’wg genkey | sudo tee /etc/wireguard/privatekey | wg
,→ pubkey | sudo tee /etc/wireguard/publickey’]
result, err = charms.sshproxy._run(cmd)
except:
action_fail(’command failed:’ + err)
else:
action_set({’outout’: result})
finally:
clear_flag(’actions.generate-keys’)

@when(’actions.generate-config’)
def generate_config():
err = ’’
try:
gateway_ip = action_get(’gateway-ip’)
cmd = [’echo -e "[Interface]\nAddress = {}\nListenPort =
,→ 51820\nPrivatekey = $(sudo cat /etc/wireguard/
,→ privatekey)" | sudo tee /etc/wireguard/wg0.conf’.format
,→ (gateway_ip)]
result, err = charms.sshproxy._run(cmd)
except:
action_fail(’command failed:’ + err)
else:
action_set({’outout’: result})
finally:
clear_flag(’actions.generate-config’)

@when(’actions.wireguard-up’)
def wireguard_up():
err = ’’
try:
91

cmd = [’sudo wg-quick up wg0’]


result, err = charms.sshproxy._run(cmd)
except:
action_fail(’command failed:’ + err)
else:
action_set({’outout’: result})
finally:
clear_flag(’actions.wireguard-up’)

@when(’actions.add-peer’)
def add_peer():
err = ’’
try:
peer_public_key = action_get(’peer-publickey’)
gateway_ip = action_get(’gateway-ip’)
subnet_behind_tunnel = action_get(’subnet-behind-tunnel’)
public_endpoint = action_get(’public-endpoint’)
cmd = [’sudo wg set wg0 peer {} allowed-ips {},{} endpoint
,→ {}:51820 persistent-keepalive 25’.format(
,→ peer_public_key, gateway_ip,subnet_behind_tunnel,
,→ public_endpoint)]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo ip -4 route add {} dev wg0’.format(gateway_ip)]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo ip -4 route add {} dev wg0’.format(
,→ subnet_behind_tunnel)]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo wg-quick save wg0’]
result, err = charms.sshproxy._run(cmd)
except:
action_fail(’command failed:’ + err)
else:
action_set({’outout’: result})
finally:
clear_flag(’actions.add-peer’)

@when(’actions.del-peer’)
def del_peer():
err = ’’
try:
92 A. VPN-AS-A-SERVICE POC

peer_public_key = action_get(’peer-publickey’)
subnet_behind_tunnel = action_get(’subnet-behind-tunnel’)
cmd = [’sudo wg set wg0 peer {} remove’.format(peer_public_key
,→ )]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo ip -4 route del {} dev wg0’.format(
,→ subnet_behind_tunnel)]
result, err = charms.sshproxy._run(cmd)
cmd = [’sudo wg-quick save wg0’]
result, err = charms.sshproxy._run(cmd)
except:
action_fail(’command failed:’ + err)
else:
action_set({’outout’: result})
finally:
clear_flag(’actions.del-peer’)
Appendix
OpenVPN Configuration
B
Listing B.1: openvpn-server.conf
local 192.168.69.22
port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh.pem
auth SHA512
tls-crypt tc.key
topology subnet
server 10.8.0.0 255.255.255.0
push "redirect-gateway def1 bypass-dhcp"
ifconfig-pool-persist ipp.txt
push "dhcp-option DNS 192.168.71.2"
push "dhcp-option DNS 192.168.69.1"
push "dhcp-option DNS 192.168.222.2"
keepalive 10 120
cipher AES-256-CBC
user nobody
group nogroup
persist-key
persist-tun
status openvpn-status.log
verb 3
crl-verify crl.pem
explicit-exit-notify

93
94 B. OPENVPN CONFIGURATION

Listing B.2: client.ovpn

client
dev tun
proto udp
remote 192.168.69.22 1194
resolv-retry infinite
nobind
route-nopull
route 192.168.69.0 255.255.255.0
route 192.168.71.0 255.255.255.0
persist-key
persist-tun
remote-cert-tls server
auth SHA512
cipher AES-256-CBC
ignore-unknown-option block-outside-dns
block-outside-dns
verb 3
<ca>
-----BEGIN CERTIFICATE-----
MIIDQjCCAiqgAwIBAgIUQSQG4S0s2pX7dKY0s4ahh5ZL8DAwDQYJKoZIhvcNAQEL
BQAwEzERMA8GA1UEAwwIQ2hhbmdlTWUwHhcNMjAwNTI1MTIwNTIyWhcNMzAwNTIz
MTIwNTIyWjATMREwDwYDVQQDDAhDaGFuZ2VNZTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBALspEYg3Mg9aBwrNnAy9l0B09tO5mdJBLQsX3dMhhQsJbytd
MNnXqea/8lSu9GNhmF8K/W0vL5/DSvpIWZDcUOlmRKlr+tjb9NgtgeJiBkndaccS
/IDbNZKJCwuIamWCda6DLHxZZ0ASwq6TF/UHaqEpJ1NgL72bT1R8SSupDBquQqJT
C7UxnK61vBm/27VK086EYzmOqRZNxN2GuzmaRQAp5GTQk8HTFWjkckq583dt/tg4
8yMvvpKh8LIANZhuj6kOOKkE4xnZRGH931Gp1VxFD53uUwi0YBvntY2UsgPYE+RE
PIY85HEXryS5CPHsG15cCSnV7Ilbb8TTgE1rNM0CAwEAAaOBjTCBijAdBgNVHQ4E
FgQURwF6Xf7U8GeTa/ao2ujP1/Qc2UgwTgYDVR0jBEcwRYAURwF6Xf7U8GeTa/ao
2ujP1/Qc2UihF6QVMBMxETAPBgNVBAMMCENoYW5nZU1lghRBJAbhLSzalft0pjSz
hqGHlkvwMDAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIBBjANBgkqhkiG9w0BAQsF
AAOCAQEAKKEY6Ep4p77piDz8Rm5xvE6lBVmoPIHoAcxSUP8FNCFk2mNJA0haVeyG
kbJwoOZnfO9F+qyt8rae5k6NhWsfMN8Pjx4Zx/xbCderF+5MIVeq5TM7pm8nanOr
g3MQP1xeANFXdHKSBZcjCDX4oyvlLfpjyRb1qlpihsdJkKHv3zVzNcsrZSWY8eyM
rFh6CMRYEFvR+P8JM/fiBPg8x1HZXEVRcGbssJpgqEBA5Ha+CwMifASdE8Xk3s85
QnDEAx0UgxhBSiDrIiN1zXWJwfR9auo5fZTANaOL7iXOEVJ1LYaeaRcITziV17JE
TAJRCxVm84ainkmrM/1lsgCAIHFi9Q==
-----END CERTIFICATE-----
</ca>
<cert>
95

-----BEGIN CERTIFICATE-----
MIIDTjCCAjagAwIBAgIQJIHvmckUHRZSl182L8C7QDANBgkqhkiG9w0BAQsFADAT
MREwDwYDVQQDDAhDaGFuZ2VNZTAeFw0yMDA1MjUxMjA1MjJaFw0zMDA1MjMxMjA1
MjJaMBExDzANBgNVBAMMBmNsaWVudDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBAMuB3vzRkWz9q1USx8wNcW83a5hwQ8+hTkGcIC7R7k+igpVqt7+Wiy4t
hdctpqlfVGw08BHI6U6I3/J8lgkjkaseMPfKoUGIdYazYxHi3/t5FmI1RJ3D4yJH
/ix96g2DYG7wWAz8R5brk0AOZGrP3RVT3tufGPwDeuv5/XHKzFsW7PI9i6raXdml
QnivyMbgiPXGsgIfiVL+7FLQbWE1m338BVTk7XyUJ0Y4gJ4UKG1e6Ao1H8Pk7xOq
G9K8YWZphxewMvkLaimlLEVusDekhweP6NtQKe2oQqAHmXICww7kBiYXRp+PJvTl
/15KtG/NQZdr98hRxE33JUtRyPOJDasCAwEAAaOBnzCBnDAJBgNVHRMEAjAAMB0G
A1UdDgQWBBSup8ZM7XCz4Q2yzCZ8w3x4DBAbxjBOBgNVHSMERzBFgBRHAXpd/tTw
Z5Nr9qja6M/X9BzZSKEXpBUwEzERMA8GA1UEAwwIQ2hhbmdlTWWCFEEkBuEtLNqV
+3SmNLOGoYeWS/AwMBMGA1UdJQQMMAoGCCsGAQUFBwMCMAsGA1UdDwQEAwIHgDAN
BgkqhkiG9w0BAQsFAAOCAQEAJLqzKgN+tD9YyKXkFjd7hJMBP+JzyEH4vsG/P9md
cxLYgLqlml/c2BXojn36RrVeLi++azhxkXHYgzreINvqb4/0Y0IkiozXFzod1GbX
G9W2cIxsb+YUn/KQlalL8Z1EYs2cxPvhzesQr8VclYWSus57vK5WULYrOfTTHd9r
6mIqgN2UBXqJdtGAjjaANVdlIjHkQi9rYL6uPm+DS/fUpAnKls3XDVAGb3zkSuqx
uf8j84pQ6mQK4dFG0YUW/0Aj4Vqp2WHBmaIMC7srQoRdb8zcp2sInRzkjwrCF7sj
liP6XDvTZgRGRUiCxCJmTa6MbbbQQ4LU/GB2na5se8UwHA==
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQDLgd780ZFs/atV
EsfMDXFvN2uYcEPPoU5BnCAu0e5PooKVare/losuLYXXLaapX1RsNPARyOlOiN/y
fJYJI5GrHjD3yqFBiHWGs2MR4t/7eRZiNUSdw+MiR/4sfeoNg2Bu8FgM/EeW65NA
DmRqz90VU97bnxj8A3rr+f1xysxbFuzyPYuq2l3ZpUJ4r8jG4Ij1xrICH4lS/uxS
0G1hNZt9/AVU5O18lCdGOICeFChtXugKNR/D5O8TqhvSvGFmaYcXsDL5C2oppSxF
brA3pIcHj+jbUCntqEKgB5lyAsMO5AYmF0afjyb05f9eSrRvzUGXa/fIUcRN9yVL
UcjziQ2rAgMBAAECggEADn36HjDiVUufNti2h65xZIUi0WE1QwTuqLdwh0UF3n/p
IrDN0wPqd2hgfplOpAMY5PfEHNAc5S3UHNWIbbAtwGzO+eZhI2SPUf9WT8DDSIov
8OBwOXfopdlQnWR+FWZEqMHO02XnxzSRcN9VgVG6IbmuRh+HTPMGfuV3ifHyKY5Q
0oMtSTvXcfcJwy/uhJ/OAtJtLk1izdtRMWY2QA9lyRbnBpTk1wv1fhwsV/Mdts1o
n3wSsAxRh/8nY79zPVE0SYKiUz4t3lNKivYC/QR5g0rWcytcgn79bJ1rXc+oUk4x
GLnHYpKlgIQiwnUlrr+dLQFs3pgj2yGZNSwKo36q8QKBgQDyL4bnznCOsrB8HoRQ
4loHsyTTcywS1OqPXVrVqzWJWnsHIrMkCJUwauYIczhhH/vA6o85OSjrLe+8QOOT
j2kEp3K3k1LdJ0WHB+jCXH0fndL6HGRtQD1aUMhTuwbIYgwvSy75fH0Uu7TxeYCI
pWf7ta03SHh1Dg8hH7F3+N1bKQKBgQDXHYzoZ628Lv45eg5pi6U+I/ICgIxjR9Kz
eli0RkiYZIWmL1i1p4Of6RvPird78D1j9jIYj7nKEJXt+V0cWPrTlP8DYp8XwqAa
8QcAZRWaOEvI14Fk64n1cO+ESid4dKUwDQzT1lRGYUEcc74KchDNnAxuVAVG87C0
pVT1WEnQswKBgQDZZ36z8maqQpAZK4/dyAPHq7IgOuBSm11giw4yhrnNbjLIqgak
96 B. OPENVPN CONFIGURATION

DyxVxXBwH3Z6AasOd8FQPhaUX604kgt4+CDu854QWyjkrA9G8vuoFjygOB4fNYOV
azhoLeJW/6pMEXAcuGIXxf/Gr+Xzav3OLKWWmHVBDkZL1e4Q7xFxwC59WQKBgHqz
alpuORUsIOqcCnh9pQWq7zIqLOm8YBzr3okFxgbW370JKKqFIHkoOCrLzxks0Xeh
SLXl1W3CxCJ3qHqw7nGdfVBrAqBE7ZowfXPmNMpA7kljU3VDPO/5OUXWHmmA73BX
IUf76xoyAze9dVrz8U22gwizeEceXV7NJ4bCk2gtAoGAc/+O+8XOF1yTNiXi6UXj
orqTZI4UssTzlXJTa7Z9JBrkubOeSy0mWAtWtnr+MxpNsZvbGS2K5IYo2SF5Yf/E
g3KuRmoT+3/QzzvhtkmwnTW53Zcx8hUEAq2nbigYTCM6OtkeSMPF3wSnBZEaoz9b
2x1813A4y/Np3i5Ab1ar2ys=
-----END PRIVATE KEY-----
</key>
<tls-crypt>
-----BEGIN OpenVPN Static key V1-----
fe6ab3b04f306313616330bf80c549e5
243ec5cd5233a301069d02f9119f873d
cc13d1d0c60dbcbd8e5c23dbd1b49b8c
564b37d7d802148fea52449d5eec52b2
f83f9a10528624d5008a5bf3ae8bda02
c102b2b4fcf13f60477044d8370c1d32
5b2da1371f19225078d0c488fb329f98
2eff45159e5cdb1be259f2016ee148c1
e916c2e8b3ebc822534b3ceadd086476
b6bb56625054f5d0495895f4f62c0f4a
208c52449549e93120460f61a56030b4
c6ad889f4c134d7c2a335e4bbe33ce99
18d1f5f1cc49448a5b592493da5f7005
fb37a30226c6d2192d64ea17244e39d9
07a50291140939d153523d134805e4a0
d96e08643928a67642c222a9e17787be
-----END OpenVPN Static key V1-----
</tls-crypt>
Appendix

C Logs

Listing C.1: lcm.log


2020-05-26T18:07:14 DEBUG lcm lcm.py:261 Task kafka_read receives ns instantiate: {’
,→ _admin’: {’created’: 1590516434.1910577, ’modified’: 1590516434.1910577, ’
,→ projects_read’: [’bbfcc6ee-2a9d-47bd-9fbf-59ba1b9d47c7’], ’projects_write’: [’
,→ bbfcc6ee-2a9d-47bd-9fbf-59ba1b9d47c7’]}, ’_id’: ’8dde5240-59e2-488a-aeeb
,→ -83244c3b60ae’, ’detailedStatus’: None, ’errorMessage’: None, ’id’: ’8dde5240
,→ -59e2-488a-aeeb-83244c3b60ae’, ’isAutomaticInvocation’: False, ’
,→ isCancelPending’: False, ’lcmOperationType’: ’instantiate’, ’links’: {’
,→ nsInstance’: ’/osm/nslcm/v1/ns_instances/7a5869db-6a14-45cf-8320-dab77758412e’
,→ , ’self’: ’/osm/nslcm/v1/ns_lcm_op_occs/8dde5240-59e2-488a-aeeb-83244c3b60ae’
,→ }, ’nsInstanceId’: ’7a5869db-6a14-45cf-8320-dab77758412e’, ’operationParams’:
,→ {’additionalParamsForVnf’: [{’additionalParams’: {’gateway_ip’: ’10.0.9.1’},
,→ ’member-vnf-index’: ’1’}, {’additionalParams’: {’gateway_ip’: ’10.0.9.2’}, ’
,→ member-vnf-index’: ’2’}], ’lcmOperationType’: ’instantiate’, ’nsDescription’:
,→ ’default description’, ’nsInstanceId’: ’7a5869db-6a14-45cf-8320-dab77758412e’
,→ , ’nsName’: ’vWireGuard_ns’, ’nsdId’: ’91dc3cd9-1e08-43b5-b7c3-656ac66e7f65’,
,→ ’nsr_id’: ’7a5869db-6a14-45cf-8320-dab77758412e’, ’vimAccountId’: ’8857adfd
,→ -4b54-4087-bc82-5b0024b4b138’}, ’operationState’: ’PROCESSING’, ’
,→ queuePosition’: None, ’stage’: None, ’startTime’: 1590516434.1910274, ’
,→ statusEnteredTime’: 1590516434.1910274}
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1371 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Enter
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1429 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Getting vnfrs
,→ from db
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1452 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Getting vnfd
,→ =00334262-1ea5-4d76-9900-29532573bb37 id=’vWireGuard_vnf’ from db
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1495 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Before
,→ deploy_kdus
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1534 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deploying proxy
,→ and native charms
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1114 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 create execution environment

97
98 C. LOGS

2020-05-26T18:07:14 DEBUG lcm.ns ns.py:1114 Task ns=7a5869db-6a14-45cf-8320-


,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 create execution environment
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:779 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae vnfd=’
,→ vWireGuard_vnf’ member_vnf_index=’1’ created at RO. RO_id=def0b997-4267-43bb-
,→ beca-566c71647b5d
2020-05-26T18:07:14 DEBUG lcm.ns ns.py:779 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae vnfd=’
,→ vWireGuard_vnf’ member_vnf_index=’2’ created at RO. RO_id=80dd5d19-c46f-42a4
,→ -83ad-cef8b1cf4f9c
2020-05-26T18:07:15 DEBUG lcm.ns ns.py:812 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae nsd=
,→ vWireGuard_nsd created at RO. RO_id=561b70df-6607-4047-8107-cd06d4b453d5
2020-05-26T18:07:16 DEBUG lcm.ns ns.py:867 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae ns created at
,→ RO. RO_id=52ac08a1-dbdd-48f4-88aa-975b250ea539
2020-05-26T18:07:16 DEBUG lcm.ns ns.py:873 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Waiting VIM to
,→ deploy ns. RO_ns_id=52ac08a1-dbdd-48f4-88aa-975b250ea539
2020-05-26T18:07:48 DEBUG lcm.ns ns.py:918 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deployed at VIM
Machine destroyed: 1

N2VC 2020-05-26 17:55:16.351341 (+0.001) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_disconnect_model():1381
Disconnecting model dca5fad2-dca6-482a-9bea-9dadd61f4068

N2VC 2020-05-26 17:55:16.353491 (+0.002) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1291
destroying model dca5fad2-dca6-482a-9bea-9dadd61f4068...

N2VC 2020-05-26 17:55:16.451995 (+0.098) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1293
model destroy requested dca5fad2-dca6-482a-9bea-9dadd61f4068

N2VC 2020-05-26 17:55:16.452487 (+0.001) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:17.456077 (+1.003) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:18.460780 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:19.465358 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...
99

N2VC 2020-05-26 17:55:20.470168 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:21.475297 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:22.479928 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:23.491864 (+1.012) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:24.497440 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:25.503007 (+1.006) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:26.509102 (+1.006) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:27.513974 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:28.519497 (+1.006) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:29.523489 (+1.004) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:30.528383 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:31.533504 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:32.538152 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...
100 C. LOGS

N2VC 2020-05-26 17:55:33.542684 (+1.004) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:34.548175 (+1.005) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:35.554993 (+1.007) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:36.561949 (+1.007) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:37.570184 (+1.007) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:38.589042 (+1.02) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:39.595021 (+1.006) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1298
Waiting for model is destroyed...

N2VC 2020-05-26 17:55:39.604265 (+0.009) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_destroy_model():1303
The model dca5fad2-dca6-482a-9bea-9dadd61f4068 (f536f1b6-1f56-46b8-84c0-9eff9e16589c)
,→ was destroyed

N2VC 2020-05-26 17:55:39.604725 (+0.001) coro-7fe44c1e73b8 _delete_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.delete_namespace():561
Namespace .dca5fad2-dca6-482a-9bea-9dadd61f4068 deleted

N2VC 2020-05-26 18:07:14.420812 (+694.816) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():220
Creating execution environment. namespace: .7a5869db-6a14-45cf-8320-dab77758412e.
,→ e62b1cd8-7276-4775-ad10-96a107044bb6, reuse_ee_id: None

N2VC 2020-05-26 18:07:14.421286 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():236
model name: 7a5869db-6a14-45cf-8320-dab77758412e, application name: app-vnf-96
,→ a107044bb6, machine_id: None

N2VC 2020-05-26 18:07:14.421819 (+0.0) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():784
creating machine in model: 7a5869db-6a14-45cf-8320-dab77758412e, existing machine id:
,→ None
101

N2VC 2020-05-26 18:07:14.660470 (+0.239) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():220
Creating execution environment. namespace: .7a5869db-6a14-45cf-8320-dab77758412e
,→ .94847cf8-e3a9-4b14-b231-181994a6f985, reuse_ee_id: None

N2VC 2020-05-26 18:07:14.660928 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():236
model name: 7a5869db-6a14-45cf-8320-dab77758412e, application name: app-vnf-181994
,→ a6f985, machine_id: None

N2VC 2020-05-26 18:07:14.661335 (+0.0) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():784
creating machine in model: 7a5869db-6a14-45cf-8320-dab77758412e, existing machine id:
,→ None

N2VC 2020-05-26 18:07:14.661765 (+0.0) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_get_model():1144
Another coroutine is creating a model. Wait...

N2VC 2020-05-26 18:07:14.663837 (+0.002) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_get_model():1159
Model 7a5869db-6a14-45cf-8320-dab77758412e does not exist. Creating new model...

N2VC 2020-05-26 18:07:15.434962 (+0.771) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_get_model():1171
New model created, name=7a5869db-6a14-45cf-8320-dab77758412e

N2VC 2020-05-26 18:07:15.435650 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():801
Creating a new machine in juju...

N2VC 2020-05-26 18:07:15.467106 (+0.031) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():801
Creating a new machine in juju...

N2VC 2020-05-26 18:07:16.125666 (+0.659) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_machine():106
Waiting for machine completed: 0

N2VC 2020-05-26 18:07:16.128070 (+0.002) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_machine():106
Waiting for machine completed: 1

N2VC 2020-05-26 18:08:02.118610 (+45.99) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():846
2020-05-26T18:08:02 DEBUG lcm.ns ns.py:1176 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 Install configuration Software
2020-05-26T18:08:20 DEBUG lcm.ns ns.py:1176 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 Install configuration Software
102 C. LOGS

2020-05-26T18:09:30 DEBUG lcm.ns ns.py:1872 Task ns=7a5869db-6a14-45cf-8320-


,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 No relations
2020-05-26T18:09:30 DEBUG lcm.ns ns.py:1198 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 Waiting to VM being up and getting IP address
2020-05-26T18:09:40 DEBUG lcm.ns ns.py:1208 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 VM_ip_address=10.20.20.237
2020-05-26T18:09:40 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 execute primitive ’config’ params ’{’ssh-hostname’: ’
,→ 10.20.20.237’, ’ssh-username’: ’ubuntu’, ’ssh-password’: ’osm4u’}’
2020-05-26T18:09:41 DEBUG lcm.ns ns.py:1872 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 No relations
2020-05-26T18:09:41 DEBUG lcm.ns ns.py:1198 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 Waiting to VM being up and getting IP address
2020-05-26T18:09:51 DEBUG lcm.ns ns.py:1208 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 VM_ip_address=10.20.20.2
2020-05-26T18:09:51 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 execute primitive ’config’ params ’{’ssh-hostname’: ’
,→ 10.20.20.2’, ’ssh-username’: ’ubuntu’, ’ssh-password’: ’osm4u’}’
Machine ready at 10.22.46.25

N2VC 2020-05-26 18:08:02.119490 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():259
ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-96a107044bb6.0

N2VC 2020-05-26 18:08:02.120390 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():265
Execution environment created. ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-96
,→ a107044bb6.0, credentials: {’hostname’: ’10.22.46.25’}

N2VC 2020-05-26 18:08:02.122848 (+0.003) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():344
Installing configuration sw on ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-96
,→ a107044bb6.0, artifact path: 00334262-1ea5-4d76-9900-29532573bb37/
,→ hackfest_simplecharm_vnf/charms/simple, db_dict: {’collection’: ’nsrs’, ’
,→ filter’: {’_id’: ’7a5869db-6a14-45cf-8320-dab77758412e’}, ’path’: ’_admin.
,→ deployed.VCA.0.’}

N2VC 2020-05-26 18:08:02.123538 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():359
model: 7a5869db-6a14-45cf-8320-dab77758412e, application: app-vnf-96a107044bb6,
,→ machine: 0

N2VC 2020-05-26 18:08:02.125258 (+0.002) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():972
103

deploying application app-vnf-96a107044bb6 to machine 0, model 7a5869db-6a14-45cf


,→ -8320-dab77758412e

N2VC 2020-05-26 18:08:02.126091 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():973
charm: /app/storage//00334262-1ea5-4d76-9900-29532573bb37/hackfest_simplecharm_vnf/
,→ charms/simple

N2VC 2020-05-26 18:08:04.697281 (+2.571) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():988
waiting for application deployed... app-vnf-96a107044bb6

N2VC 2020-05-26 18:08:04.698274 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_application():126
Waiting for application completed: app-vnf-96a107044bb6

N2VC 2020-05-26 18:08:20.022329 (+15.324) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_create_machine():846
Machine ready at 10.22.46.34

N2VC 2020-05-26 18:08:20.023115 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():259
ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-181994a6f985.1

N2VC 2020-05-26 18:08:20.023861 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.create_execution_environment():265
Execution environment created. ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf
,→ -181994a6f985.1, credentials: {’hostname’: ’10.22.46.34’}

N2VC 2020-05-26 18:08:20.026177 (+0.002) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():344
Installing configuration sw on ee_id: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf
,→ -181994a6f985.1, artifact path: 00334262-1ea5-4d76-9900-29532573bb37/
,→ hackfest_simplecharm_vnf/charms/simple, db_dict: {’collection’: ’nsrs’, ’
,→ filter’: {’_id’: ’7a5869db-6a14-45cf-8320-dab77758412e’}, ’path’: ’_admin.
,→ deployed.VCA.1.’}

N2VC 2020-05-26 18:08:20.026898 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():359
model: 7a5869db-6a14-45cf-8320-dab77758412e, application: app-vnf-181994a6f985,
,→ machine: 1

N2VC 2020-05-26 18:08:20.028621 (+0.002) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():972
deploying application app-vnf-181994a6f985 to machine 1, model 7a5869db-6a14-45cf
,→ -8320-dab77758412e

N2VC 2020-05-26 18:08:20.029344 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():973
charm: /app/storage//00334262-1ea5-4d76-9900-29532573bb37/hackfest_simplecharm_vnf/
,→ charms/simple
104 C. LOGS

N2VC 2020-05-26 18:08:24.263293 (+4.234) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():988
waiting for application deployed... app-vnf-181994a6f985

N2VC 2020-05-26 18:08:24.263862 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_application():126
Waiting for application completed: app-vnf-181994a6f985

N2VC 2020-05-26 18:09:30.009520 (+65.745) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():993
application deployed

N2VC 2020-05-26 18:09:30.010370 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():393
Configuration sw installed

N2VC 2020-05-26 18:09:40.022173 (+10.012) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: config on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-96
,→ a107044bb6.0, params: {’ssh-hostname’: ’10.20.20.237’, ’ssh-username’: ’
,→ ubuntu’, ’ssh-password’: ’osm4u’}

N2VC 2020-05-26 18:09:40.022779 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1068
configuring the application app-vnf-96a107044bb6 -> {’ssh-hostname’: ’10.20.20.237’, ’
,→ ssh-username’: ’ubuntu’, ’ssh-password’: ’osm4u’}

N2VC 2020-05-26 18:09:40.057195 (+0.034) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1070
application app-vnf-96a107044bb6 configured. res={’request-id’: 110, ’response’: {}}

N2VC 2020-05-26 18:09:40.061658 (+0.004) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
ssh-hostname = 10.20.20.237

N2VC 2020-05-26 18:09:40.062110 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
ssh-username = ubuntu

N2VC 2020-05-26 18:09:40.062540 (+0.0) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
ssh-password = osm4u

N2VC 2020-05-26 18:09:40.065367 (+0.003) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1095
Executing action verify-ssh-credentials...

N2VC 2020-05-26 18:09:40.068245 (+0.003) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "verify-ssh-credentials" using params: {}
105

N2VC 2020-05-26 18:09:41.701193 (+1.633) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 1

N2VC 2020-05-26 18:09:41.779100 (+0.078) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_deploy_charm():993
application deployed

N2VC 2020-05-26 18:09:41.779583 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.install_configuration_sw():393
Configuration sw installed

N2VC 2020-05-26 18:09:51.788864 (+10.009) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: config on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf
,→ -181994a6f985.1, params: {’ssh-hostname’: ’10.20.20.2’, ’ssh-username’: ’
,→ ubuntu’, ’ssh-password’: ’osm4u’}

N2VC 2020-05-26 18:09:51.789398 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1068
configuring the application app-vnf-181994a6f985 -> {’ssh-hostname’: ’10.20.20.2’, ’
,→ ssh-username’: ’ubuntu’, ’ssh-password’: ’osm4u’}

N2VC 2020-05-26 18:09:51.847186 (+0.058) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1070
application app-vnf-181994a6f985 configured. res={’request-id’: 138, ’response’: {}}

N2VC 2020-05-26 18:09:51.851214 (+0.004) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
ssh-hostname = 10.20.20.2

N2VC 2020-05-26 18:09:51.851654 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
2020-05-26T18:09:52 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 execute primitive ’touch’ params ’{’filename’: ’/home/
,→ ubuntu/first-touch’}’
2020-05-26T18:10:07 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 execute primitive ’generate-keys’ params ’{}’
2020-05-26T18:10:07 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 execute primitive ’touch’ params ’{’filename’: ’/home/
,→ ubuntu/first-touch’}’
2020-05-26T18:10:16 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 execute primitive ’generate-config’ params ’{’gateway-ip’:
,→ ’10.0.9.1’}’
2020-05-26T18:10:25 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 execute primitive ’wireguard-up’ params ’{}’
106 C. LOGS

2020-05-26T18:10:25 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-


,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 execute primitive ’generate-keys’ params ’{}’
2020-05-26T18:10:34 DEBUG lcm.ns ns.py:1276 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=1 instantiated at VCA
2020-05-26T18:10:35 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 execute primitive ’generate-config’ params ’{’gateway-ip’:
,→ ’10.0.9.2’}’
2020-05-26T18:10:48 DEBUG lcm.ns ns.py:1265 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 execute primitive ’wireguard-up’ params ’{}’
ssh-username = ubuntu

N2VC 2020-05-26 18:09:51.852132 (+0.0) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1076
ssh-password = osm4u

N2VC 2020-05-26 18:09:51.855155 (+0.003) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1095
Executing action verify-ssh-credentials...

N2VC 2020-05-26 18:09:51.858205 (+0.003) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "verify-ssh-credentials" using params: {}

N2VC 2020-05-26 18:09:52.176605 (+0.319) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 2

N2VC 2020-05-26 18:09:52.510905 (+0.334) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:09:52.522197 (+0.011) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1104
Result: completed, output: {’Code’: ’0’, ’output’: ’vwireguard-ns-1-vwireguard-vnfd-
,→ vm-1’, ’verified’: ’True’}

N2VC 2020-05-26 18:09:52.522970 (+0.001) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: touch on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-96
,→ a107044bb6.0, params: {’filename’: ’/home/ubuntu/first-touch’}

N2VC 2020-05-26 18:09:52.526780 (+0.004) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "touch" using params: {’filename’: ’/home/ubuntu/first-touch’}

N2VC 2020-05-26 18:09:52.593674 (+0.067) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 3
107

N2VC 2020-05-26 18:10:07.296158 (+14.703) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:07.330912 (+0.035) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: generate-keys on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-
,→ vnf-96a107044bb6.0, params: {}

N2VC 2020-05-26 18:10:07.333670 (+0.003) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "generate-keys" using params: {}

N2VC 2020-05-26 18:10:07.437472 (+0.104) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:07.439597 (+0.002) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 4

N2VC 2020-05-26 18:10:07.444983 (+0.005) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_configure_application():1104
Result: completed, output: {’Code’: ’0’, ’output’: ’vwireguard-ns-2-vwireguard-vnfd-
,→ vm-1’, ’verified’: ’True’}

N2VC 2020-05-26 18:10:07.445615 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: touch on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf-181994
,→ a6f985.1, params: {’filename’: ’/home/ubuntu/first-touch’}

N2VC 2020-05-26 18:10:07.448166 (+0.003) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "touch" using params: {’filename’: ’/home/ubuntu/first-touch’}

N2VC 2020-05-26 18:10:07.539531 (+0.091) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 5

N2VC 2020-05-26 18:10:16.365829 (+8.826) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:16.402870 (+0.037) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: generate-config on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-
,→ vnf-96a107044bb6.0, params: {’gateway-ip’: ’10.0.9.1’}

N2VC 2020-05-26 18:10:16.405719 (+0.003) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "generate-config" using params: {’gateway-ip’: ’10.0.9.1’}
108 C. LOGS

N2VC 2020-05-26 18:10:16.461920 (+0.056) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 6

N2VC 2020-05-26 18:10:25.338961 (+8.877) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:25.425415 (+0.086) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:25.480186 (+0.055) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: wireguard-up on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf
,→ -96a107044bb6.0, params: {}

N2VC 2020-05-26 18:10:25.481356 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: generate-keys on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-
,→ vnf-181994a6f985.1, params: {}

N2VC 2020-05-26 18:10:25.483098 (+0.002) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "wireguard-up" using params: {}

N2VC 2020-05-26 18:10:25.484289 (+0.001) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "generate-keys" using params: {}

N2VC 2020-05-26 18:10:25.609762 (+0.125) coro-7fe44c1c49e8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 7

N2VC 2020-05-26 18:10:25.611379 (+0.002) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 8

N2VC 2020-05-26 18:10:34.493796 (+8.882) coro-7fe44c1c49e8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:34.982438 (+0.489) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:35.021822 (+0.039) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: generate-config on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-
,→ vnf-181994a6f985.1, params: {’gateway-ip’: ’10.0.9.2’}
109

N2VC 2020-05-26 18:10:35.024650 (+0.003) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "generate-config" using params: {’gateway-ip’: ’10.0.9.2’}

N2VC 2020-05-26 18:10:35.146557 (+0.122) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 9

N2VC 2020-05-26 18:10:48.385843 (+13.239) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
action completed with status: completed

N2VC 2020-05-26 18:10:48.427032 (+0.041) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector.exec_primitive():610
Executing primitive: wireguard-up on ee: 7a5869db-6a14-45cf-8320-dab77758412e.app-vnf
,→ -181994a6f985.1, params: {}

N2VC 2020-05-26 18:10:48.431569 (+0.005) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1031
executing action "wireguard-up" using params: {}

N2VC 2020-05-26 18:10:48.470143 (+0.039) coro-7fe44c1c40f8 instantiate_N2VC()


,→ juju_observer.py::N2VCJujuConnector.wait_for_action():147
Waiting for action completed: 10

N2VC 2020-05-26 18:11:01.753711 (+13.283) coro-7fe44c1c40f8 instantiate_N2VC()


,→ n2vc_juju_conn.py::N2VCJujuConnector._juju_execute_action():1041
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1276 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae
,→ member_vnf_index=2 instantiated at VCA
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1747 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deploy KDUs:
,→ Done
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1747 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deploy at VIM:
,→ Done
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1747 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deploy VCA 1.:
,→ Done
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1747 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Deploy VCA 2.:
,→ Done
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1814 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae End of
,→ instantiation: True
2020-05-26T18:11:01 DEBUG lcm lcm.py:261 Task kafka_read receives ns instantiated: {’
,→ nslcmop_id’: ’8dde5240-59e2-488a-aeeb-83244c3b60ae’, ’nsr_id’: ’7a5869db-6a14
,→ -45cf-8320-dab77758412e’, ’operationState’: ’COMPLETED’}
2020-05-26T18:11:01 DEBUG lcm.ns ns.py:1827 Task ns=7a5869db-6a14-45cf-8320-
,→ dab77758412e instantiate=8dde5240-59e2-488a-aeeb-83244c3b60ae Exit
Appendix

D
Research Paper

111
VPN-as-a-Service for 5G Networks:
Proof-of-Concept
Simen Haga, Katina Kralevska, Ali Esmaily and Danilo Gligoroski
Dep. of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU)
Email: {simehag, katinak, ali.esmaeily, danilog}@ntnu.no

Abstract—The fifth-generation (5G) aims to host different types of virtual resources, such as virtual machines (VMs),
of services on the same physical infrastructure. Network slicing storage, networking, and connectivity.
is considered the key enabler to achieve this goal. Although there • The VNFs, network services, and network slices block
are signs of progress in applying and implementing network slic-
ing in the context of 5G, the security and performance of multiple includes the collection of VNFs, and the composition of
network slices still demand a high level of attention. In this paper, VNFs into network services to form network slices.
we implement WireGuard secure network tunneling protocol in • The third section of the NFV partitioning is the NFV
a 5G network to provide a VPN-as-a-Service functionality for Management and Orchestration (NFV-MANO) compo-
virtualized network functions. We demonstrate that WireGuard- nent, which controls the lifecycle of the VNFs, network
enabled services possess considerable enhancement in terms of
network throughput and latency. services, and network slices. It also covers the general
Keywords: 5G, VPNaaS, Slice isolation, Private networks, resource orchestration and service lifecycle of the service
WireGuard, Orchestration, OSM, PoC. it manages.
In 5G networks, there will be a portfolio of isolation tech-
I. I NTRODUCTION nologies available rather than a single technology like virtual
private network (VPN). This means that it will be necessary
As the 5G architecture is still evolving, the security threat to integrate and manage a variety of isolation mechanisms on
landscape of 5G has grown enormously due to the unprece- different levels. Moreover, there are different levels of slice
dented increase in types of services and in the number isolation: isolation of traffic, isolation of bandwidth, isolation
of devices. 5G is expected to deliver simultaneous services of processing and isolation of storage. Papers such as [3], [4]
with different latency, throughput, connectivity, and security outline only the challenges of providing slice isolation without
requirements. These services should be provided on the same proposing solutions. To be more precise, [3] lists several
infrastructure with the help of network slicing. The key potential slice isolation technologies such as tag-based slices
feature of network slicing is the capability of virtualizing isolation, VLAN-based network slices isolation, VPN-based
the underlying infrastructure and creating independent logical slices isolation with IPSec, Secure Socket Layer/Transport
networks. It follows Software-Defined Networking (SDN) [1] Layer Security (SSL/TLS), Secure Socket Tunneling Protocol
and Network Function Virtualization (NFV) [2] principles by (SSTP), SSH (Secure Shell) and SDN-based isolation.
decoupling the network functions, such as firewalls, load- We focus on a VPN solution for isolation of traffic, meaning
balancers, proxy servers, intrusion detectors and others, from that data flow of one slice cannot move to another slice and
proprietary hardware appliances and running them as software be accessed from another slice although they are using the
in virtual machines (VMs). The different functions are called same network resources. We even further extend the concept
virtual network functions (VNFs). SDN and NFV provide of traffic isolation to security isolation, i.e. the property that
programmability of the network and enable flexible placement an unauthorized party outside the slice, for instance, another
and configuration of virtualized functions. user of the same infrastructure but belonging to another slice,
Service function chaining is a technique for selecting and cannot modify or even eavesdrop the traffic flow of the slice
steering data traffic flows through various network functions, [5]. This also provides confidentiality and integrity protec-
i.e it interconnects the VNFs in a specific order via virtual tion of the tenant’s traffic even against the Mobile Network
links to provide a complete end-to-end service as shown in Operator (MNO). For instance, a tenant that is in need of a
Figure 1. The VNFs, network services and network slices run network fit for a certain use case, rents a slice of the network
on top of the NFV Infrastructure (NFVI). ETSI has done a from the infrastructure provider. The network slice is then used
significant amount of work for monitoring and orchestration by the tenant to provide a network service to the end users.
of VNFs. The high-level NFV management framework given There is no way to guarantee confidentiality unless the data is
by ETSI has three functional blocks: encrypted.
• The Infrastructure block, composed of the NFVI and Encrypting the data between chained VNFs can lead to
Virtualized Infrastructure Manager (VIM), is responsi- telecommunication operators being more open towards multi-
ble for providing the virtualization environment for the vendor NFV solutions [6]. There are two main reasons for
VNFs. The VIM performs the lifecycle management telecommunication operators being hesitant to multi-vendor
NFV solutions. First, the NFV implementations are not steered and throughput compared to existing VPN technologies. That
by operators, but rather by the vendors and integrators that is the reason for using it in our VPN as a Service (VPNaaS)
perform the original root cause analysis of the problems. Here proof of concept. OSM is used to manage and orchestrate
the risk lies in the fact that most vendors’ business rely on components and their interconnections, which constitutes a
proprietary boxes, which could be negatively impacted by more extensive network infrastructure. We present our OSM-
moving to a pure NFV platform. Second, operators are used to WireGuard framework and its lifecycle. The north bound
managing contracts with a handful of large vendors that pro- interface of OSM in combination with juju proxy charms
vide the design, engineering and network deployment. Thus, allow us to manage easily the WireGuard infrastructure with
the implementation of a multi-vendor NFV solution brings our API. The performance results show that WireGuard is a
unwanted complexity for operators with regards to whom to promising solution for providing traffic isolation in slices with
blame for security issues. For this reason, most operators are strict latency and throughput requirements. This is the first
adopting a vertical NFV solution, where a vendor delivers application and implementation of WireGuard in a modern
the full NFV deployment and retains the responsibility. A network topology where OSM facilitates the configuration
horizontal NFV model is the most preferred solution in the and key distribution, thus, it opens new potentials for both
telecommunication community, as it will significantly improve technologies.
automation and contribute to higher flexibility. In this type of
deployment, multiple NFV vendor products are managed and II. BACKGROUND
orchestrated on a single commodity NFVI by a single NFV A. Technical background
MANO entity. This architecture is the telecom equivalent to
cloud computing. OpenStack is a cloud operating system used to provide
Infrastructure as a Service (IaaS) orchestration and fault and
service management to operators. In our framework, Open-
Stack as a VIM hosts the base images on which the VNFs
are instantiated. OSM handles the rest of the orchestration,
configuration, and management of this tool to set up the
network services.
Open Source MANO (OSM) is a tool developed by ETSI
for management and orchestration of NFV. OSM makes use
of Virtual Network Function Descriptors (VNFDs), which
adhere to the unified VNF catalogue, to describe all VNFs
and network services in a standard way. VNFs are packed
into VNF packages, which include the VNFD, configuration
Fig. 1. A network slice is the composition of multiple network services,
composed of multiple VNFs. scripts, and other artifacts. Operators are then able to combine
these packages to describe their own network service, in a
Network Service Descriptor (NSDs), and then again combine
NSDs to provide a network slice instance.
The process of producing a VNF package and making the
package functional, by satisfying its lifecycle stages, is known
as the VNF onboarding process. The process consists of three
stages which are named Day-0, Day-1, and Day-2. They all
revolve around the full lifecycle of a single VNF package.
• Day-0: Day-0: Determines all necessary elements of a
VNF package for it to be instantiated successfully, and
set up its management, so it is possible to configure
the VNF later. The main requirements in this stage are:
describing different involved components of the VNF
(the essential Virtualization Deployment Units (VDUs)
for hosting VNFs), designating NFVI requirements, in-
Fig. 2. The ETSI-NFV architectural framework as illustrated in [7]. dicating topology and management mechanism for the
VNF, stipulating the certain Linux images and cloud-init
Our contribution: With the aid of Open Source MANO files, and identifying the instantiation parameters.
(OSM), which is one of the dominant service orchestrators, • Day-1: During Day-1 the VNF is instantiated and con-
and efficient VPN technologies such as WireGuard, it is figured to enable the continuous delivery of its services.
possible to create an overlay network that can ensure confiden- It grants the capability to instantiate and initialize the
tiality of data passing between VNFs in a 5G network slice. necessary parameters to configure the VNF in order to
WireGuard shows superior performance in terms of latency provide the expected network service. The essential step
in this stage are: classifying dependencies between the public key gN65...z6EA in Figure 4 has configured the cryp-
involved VNF components, determining the fundamental tographic key routing table to contain an endpoint for the peer
configuration for service initialization. HIgo...8ykw, and changed the IP and port of the WireGuard
• Day-2: Provides all necessary elements for the VNF interface to 192.95.5.64:21841. The host, gN65...z6EA, sends
package to be fully operational. It provides the possibility an encrypted packet to HIgo...8ykw at 192.95.5.69:41414.
to re-configure VNFs’ operations to modify them during When HIgo...8ykw receives the packet, it learns that the
the runtime process. Day-2 operations include reconfigu- endpoint for sending a reply has changed. Thus, it updates
ration, monitoring of Key Performance Indicators (KPIs) the table in Figure 5 and ensures that the reply reaches the
and automatic scaling based on the status of KPIs. new endpoint.
OSM will be used to deploy VNFs, network services, and
network slices, and to manage and orchestrate WireGuard.
Juju is one of the main VNF Managers (VNFMs) for
OSM. It works by interacting with charms that act as a
structured VNFM for VNFs on the NFVI. A charm holds all
the necessary hooks to manage the lifecycle of a VNF, such as
deployment, configuration, and exposing services to external Fig. 4. The cryptographic key routing table of a host 2b [9].
processes.
Wireguard is a new protocol for secure network tunnel
proposed by Donenfeld in 2017 [8]. It aims to replace IPsec
and TLS-based solutions for most use cases. WireGuard
demonstrates that it is possible to implement a secure network
tunnel, using state of the art cryptography in less than 4000
lines of code. Reported results show that WireGuard outshines
both IPsec and OpenVPN in terms of throughtput and ping Fig. 5. The cryptographic key routing table of host 1b [9].
time.
WireGuard’s approach to key distribution is agnostic, and
relies on out-of-band mechanisms. It operates solely on Layer B. Related Work
3 of the network stack and works by associating public keys The use of WireGuard for virtual private networks for future
with IP addresses, and is identifying peers strictly by their industrial systems has been investigated in [10]. The authors
public key. We explain it with a simple example, parahprased compared various VPN technologies, including WireGuard.
from [8]. The reported results confirm that WireGuard outperforms
When the sender transmits a packet out the wg0 interface, IPsec in terms of throughput and latency on different hardware
it consults the cryptographic key routing table, shown in platforms. References [11] discusses the use of WireGuard as
Figure 3, to determine which public key to use for encryption. an open source solution for ensuring secure communications
For instance, a packet destined for 10.192.122.4 would be on the transport layer in terms of confidentiality, integrity
encrypted with the TrMv...WXX0 public key. This also works and authenticity for future decentralised industrial IoT infras-
for receiving packets; if the interface wg0 receives a packet, tructures. The automatic establishment of secure authenticated
after decryption and authentication, it will only accept the connections, harder DOS attacks, less resource consuming
packet if the IP resolves in the table to the public key used in cryptographic primitives, better throughput performance, sim-
the secure session for decrypting it. This process dramatically ple setup and code of less than 4000 lines that also allows
simplifies firewall rules from an administrative point of view, formal verification make WireGuard an appealing solution for
because any packet arriving on the WireGuard interface will embedded IoT appliances. These works highlight WireGuard’s
have a reliably authentic source IP. relevance in a 5G environment where the low latency impact
and high throughput make it a strong contender for future
5G networks. These are the key objectives of WireGuard that
motivated us to develop a PoC of using WireGuard for VPN
soluton in 5G. It is certain that private 5G networks tailored
for a specific enterprise will foster Industry 4.0 [5].
Reference [12] identifies the need for per-flow encryption
in between chained NFVs and in this way to isolate the end-
Fig. 3. The cryptographic key routing table of a host 1a [9]. user traffic between VNFs. It presents the Software Defined
Security Associations (SD-SA) component as an alternative to
To implement a secure network tunnel in a mobile network Internet Security Key Exchange protocol (IPsec-IKE). IPsec-
environment, where mobility is a priority the following exam- IKE is not applicable inside a VNF due the lack of bidirec-
ple applies. Given the two cryptographic key routing tables tional data plane communication channels between chained
shown in Figures 4 and 5, where the peer identified by the VNFs. SD-SA automates the establishment of dynamic VPNs
in a NFV domain by introducing VNF manager and SDN of WireGuard on each VNF might be possible. In the proposed
controller where the last is omitted from the implementation. OSM-WireGuard framework, illustrated in Figure 6, the OSM-
In our work, we use the latest technologies for encryption and NBI in combination with Juju proxy charms manages the
orchestration. WireGuard infrastructure through OSM actions. The use of
Another direction of security in 5G is optimizing the place- proxy charms with the OSM-NBI let us script configurations
ment of network functions while taking into consideration the on the VNFs. The charms make it easy to expose functions
security deployment constraints. The authors of [13] focus
on the efficient placement of virtual security function when
deployed using SFCs, to make sure that they are used most
effectively and least expensively. They first transform the high
level security needs by the tenants into a set of security
patterns, then introduce the security deployment constraints
and take them into account together with the deployment costs
for security implementation.
In [14], Pattarantakul et al. present a threat analysis in
the context of NFV based network services and a conceptual
design framework for NFV based security management and
service orchestration, based on the ETSI-NFV MANO frame-
work. They argue that the tenants’ lack of visibility and control
over the network resources might lead to security threats and
vulnerabilities. The tenant only has access to the prepared Fig. 6. Our proposed OSM-WireGuard framework where the NBI of OSM
VNF and not to the image before build time, therefore, it makes use of the extensive information model, to minimize integration efforts
of network operators. Proxy charms are used to configure VNFs deployed on
is hard for the tenant to know how data is accessed. Thus, the NFVI [7].
the NFV ecosystem relies on VNF providers that follow best
practices and allow as much transparency as possible into
their VNF - to avoid security incidents. They conclude that to the administrator to define tasks such as key generation,
even though there are several ETSI-NFV MANO implemen- adding and deleting peers, starting and stopping the WireGuard
tations, few of them focus on NFV security management service, and gathering performance metrics. In our case, the
and orchestration. They propose their conceptual framework, administrator uses the OSM-NBI to configure peering rela-
secMANO, which expands the ETSI NFV MANO framework tionships between gateways, which allows the RBAC feature
by including security orchestration and policy components. to control configuration access of the gateways. Figure 7
secMANO provides security functions as a service, such as shows the relationship between the major components of our
access control to NFV resources. Recently, much has happened software stack explained in Section II-A. The installed VIM
in the ETSI-NFV MANO ecosystem. OSM introduced Role- is Microstack, which is a minimal version of OpenStack, with
Based Access Control (RBAC) which restricts access to OSM only the core components installed. To run OSM and the proxy
API calls and ensures that only authorized tenants are able charms we use Microk8s. The technical specifications of the
to interact with the NFV resources, which motivates our testbed are listed in Table I.
choice for an orchestrator and the relevance of our proposed
framework.
A group of NFV vendors showcased how to built a vir-
tualized multi-vendor LTE Packet Core on top of OpenStack
and commercial-of-the-shelf servers orchestrated by OSM [6].
They realize site-to-site VPN. The difference is that we aim
to provide a point-to-point solution for VPN. Since VNFs are
instantiated in multiple domains of the network architecture,
a tenant might want the confidentiality of the traffic between
the VNFs secured in a true end-to-end fashion.
III. OSM-W IRE G UARD F RAMEWORK
Here we present the PoC where OSM manages and or-
chestrates services in a network that provides a VPN tunnel
between endpoints in a virtualized 5G environment. OSM
works by utilizing its extensive information model, aligned
with ETSI-NFV which makes it possible to model and auto-
mate the lifecycle of VNFs, network services, and network Fig. 7. The software stack used during the development of the VPN-as-a-
slices. OSM has multiple angles from which the configuration Service PoC.
Host OS Ubuntu 18.04 Bionic
Memory 32 GB
process is also depicted in Figure 6. The extra configurations
CPU Intel i7-4790 included with –config is used to specify instantiation parame-
NIC NetXtreme BCM5722 ters, which makes it simpler to effectively launch the service
TABLE I and customize the network service’s attributes according to the
H ARDWARE AND OS USED FOR THE M ICROSTACK DURING THE SECOND network service provider’s needs.
DESIGN CYCLE .

Day-2
When the network service is instantiated, and all services
are running on the VDU of each WireGuard VNF, the network
service is ready to be operated at runtime. By this point, the
administrator can use the OSM-NBI and the actions add-peer
and del-peer to manage peering relationships between new
WireGuard VNFs while running Day-2 operations. During this
time, the OSM KPI monitoring framework can also be used
to monitor the performance of the WireGuard gateways, and
if there is a need for more performance, OSM can scale the
gateways accordingly.
To further connect the WireGuard-enabled network service
with other network services, forming a network slice, the
administrator will use the Connection Points and Virtual Link
Descriptors, which are part of the Network Slice Template.
Through the Connection Points, modularity is achieved, and
the WireGuard-enabled network service can easily be inte-
Fig. 8. Lifecycle of the WireGuard VPN-as-a-Service solution with OSM. grated with the existing infrastructure to provide confidential-
ity where needed.

A. OSM-WireGuard lifecycle B. Building a network service


In regard to the three days lifecycle of OSM, Day-1 and Here we list the steps of building the VNF and network
Day-2 impact directly the configuration of WireGuard in service packages.
the network service. Adding VNF configuration with OSM 1) Use a minimal Ubuntu Server 18.04 image to create a
requires that we first define our configuration primitives in the baseline for the VDU, and install WireGuard through
VNFD before we specify them in detail in our proxy charm. cloud-init during the instantiation of the VNF.
Adding primitives to the VNFD simplifies the process and 2) Create the VNF package, configure the yaml files and
oversight of the generated API to verify and manage, as it set path variables in order to create the juju charm.
provides us with a detailed list that can be read as documen- 3) Define the list of actions used by the juju charm frame-
tation of the configuration primitives and their instantiation work.
parameters. OSM’s NBI allows the administrator to configure 4) Build the charm, and export it to the VNF package and
these primitives when creating the network service through verify that the VNFD is valid.
additional YAML files which automate the process. 5) Create the NS package which includes the required
A simplified version of the lifecycle is given in Figure VNFs.
8, which portrays the interaction between components in the 6) Update the yaml file with the NSD and validate the NSD.
framework for each day of the VNF lifecycle. 7) Compress the packages and upload them to OSM.
8) When OSM creates an instance of the network service,
Day-0 the resource orchestrator component of OSM instructs
The administrator must first retrieve the VNF and network the connected VIM to spin up the described NFVI.
service packages, make sure the hardware requirements are The outcome of the process is the network topology
met, upload the packages to the OSM platform through the displayed in Figure 9.
NBI, and confirm that a compatible Ubuntu image exists on 9) Connect the two WireGuard gateways by adding the
the connected VIMs. The administrator then utilizes the OSM- peer’s public key to the configuration file for the wg0
NBI to launch an instance of the network service and make interface, along with the peer’s IP on the VPN, the
the VNFs manageable from OSM itself. subnet we want to connect to, and finally the public
endpoint connected top the tunnel-network created by
Day-1 the VIM.
During this process, as illustrated by the Day-1 arrow in 8, 10) Instantiate the network service.
OSM employs juju charms to instantiate the network service, As we can see there are only a few additional steps about
according to the parameters given at instantiation time. The configuring the yaml files and the juju charm framework
Fig. 9. Openstack’s network topology display shows the two WireGuard
gateways connected to the tunnel network, and the test VMs.

compared to the steps for only onboaring a network service


in OSM as described in [15]. The simplicity and efficiency of
WireGuard became apparent during the first design cycle as
Fig. 10. The process of instantiating the WireGuard network service through
the simple command-line interface made it easy to understand the use of the OSM-NBI.
how most of the process of launching a tunnel network with
Wireguard was going to proceed. One thing to note is that
with Wireguard’s simplicity, it also becomes a necessity to primitives from the initial-config-primitive section to instan-
script or use external tools to solve cases with increased tiate the network service. After observing that the VNFs are
complexity. However, the simplicity is an intended feature to configured, the operator extracts the public key from each
avoid cases such as IPsec, which has myriads of configurations VNF and uses it while calling the ns-action add-peer, which
the administrator can choose from - and therefore increase the initiates the process of adding a peer to the VNF’s WireGuard
probability of introducing vulnerabilities through misconfigu- configuration. At this point the network service instance is
rations. operational and ready to serve connected network services
involved in the network slice.
C. Key distribution
Since Wireguard relies on an out-of-band process to perform IV. P ERFORMANCE A NALYSIS
key distribution we chose to use juju proxy charms to handle
creation of peering relationships. During Day-1 the public and One of the goals of 5G-PPP is reducing the average service
private keys of each VNF is generated, while the keys are creation time cycle from 90 hours to 90 minutes. We use
retrieved through SSH, and distributed with add-peer during these definitions defined by Yilma et al. [16] for the KPIs
Day-2. reported in this paper. On-boarding process delay (OPD) is the
time it takes to boot-up a virtualized network function image.
D. Summary Deployment Process Delay (DPD) is the time-delay introduced
The summary of the process of instantiating the WireGuard by deploying and instantiating the VNF on the VM, to produce
network service is shown in Figure 10. The figure also includes an operational network service. A summary of the measured
some of the key performance indicators (KPIs), On-boarding OSM-WireGuard framework’s KPIs can be found in Table II.
process delay (OPD) and Deployment Process Delay (DPD), The service’s OPD is 159 second, while the DPD ended at
that we present in the next Section. The process begins when 107 seconds. This means that the total time from the point we
the operator uses the OSM-NBI to call ns-create. OSM then initiated our service until it was ready to provide the required
logs the instantiation parameters in its database, and the service was 266 seconds. During the on-boarding process, the
internal resource orchestrator (RO) unit utilizes the Openstack installation of WireGuard took 102 seconds, making it the
API to deploy the necessary NFVI. When the deployment largest contribution to the OPD. The DPD includes the action
of the NFVI is complete, the resource orchestrator updates add-peer, but excludes the time it takes for the admin to ssh
the status on OSM’s shared message bus, which initiates the onto the gateways and extract the public keys - which means
configuration of the VNFs. The Juju Controller, which acts that the configuration due to the initial-config-primitives took
as the VNF config and abstraction unit, now executes the 47 seconds.
TABLE II
S ERVICE CREATION TIME .

NVF-MANO KPI Time


On-boarding Process Delay 159 s
Deployment Process Delay 107 s

TABLE III
N ETWORK SERVICE ACTION PRIMITIVES

Action Time
add-peer 60 s
del-peer 51 s

We also compare the throughput and latency of WireGuard Fig. 12. Average latency, after 1000 ICMP requests between a gateway and
and OpenVPN with default settings in order to determine the a peering subnet’s interface on the gateway.
benefits of using WireGuard instead of OpenVPN. To test the
throughput of our network service, we used iperf between
the gateway nodes and averaged the results over 30 minutes. ing. Alternatively, the cloud-init API can be used to create the
The tests were first performed while WireGuard was active on public and private keys on each VNF as well. Which would
the two gateways. Then, we stopped the WireGuard service, be faster than juju charms additional delay due to its atomic
installed an OpenVPN server on one gateway, and used the operations. However, as a design decision it makes more sense
other as a client, before we ran iperf between the nodes again. to utilize the juju charms for the generation of keys, as it allows
The tests, show that with our configurations WireGuard has a network services that require regeneration of keys to move
5.3 times higher throughput than OpenVPN. the primitive to the standard config primitives, which makes
it simple to use the primitive during Day-2 operations. The
Measuring latency was done by using the ping tool to send
throughput and latency results of WireGuard and OpenVPN
1000 ICMP Requests, and record the average latency between
resemble to those reported in [9].
the gateway and a subnetwork on the opposite side of the VPN
tunnel. As Figure 12 illustrates, packets traveling through the
V. C ONCLUSIONS
WireGuard VPN had an average latency 41 % lower than those
from OpenVPN. We presented the first proof of concept of providing VPN
The performance analysis of the operational KPIs made it as a Service in 5G with the use of WireGuard technology.
clear that OSM combined with WireGuard can be used to WireGuard’s performance shows that it is still more applicable
provide VPNaaS while still being in line with the requirement and promising compared to both IPSec and OpenVPN in par-
of 90min. The total time that OSM uses to instantiate the ticular for network slices with low latency and high throughput
network service with two VNFs was 266 seconds, and it can requirements. Thus, WireGuard should be considered a prime
be reduced by about 102 seconds if the operator prepares candidate for network slices which require isolation by the
images used in the VM with a pre-installed and up to date means of VPN.
version of WireGuard prior to the on-boarding process. By As a future work we plan to investigate a scenario where the
pre-configuring the images it is also possible to omit actions network service or VNF protected by the WireGuard network
performed by the juju charms such as those enabling forward- service migrates to a new location. In this case we should make
use of the OpenStack migration features and WireGuard’s
ability to handle roaming.

R EFERENCES
[1] M. Jammal, T. Singh, A. Shami, R. Asal, and Y. Li, “Software-defined
networking: State of the art and research challenges,” 2014.
[2] R. Mijumbi, J. Serrat, J. Gorricho, N. Bouten, F. De Turck, and
R. Boutaba, “Network function virtualization: State-of-the-art and re-
search challenges,” IEEE Communications Surveys Tutorials, vol. 18,
no. 1, pp. 236–262, 2016.
[3] Z. Kotulski, T. Nowak, M. Sepczuk, M. Tunia, R. Artych, K. Bocianiak,
T. Osko, and J. Wary, “On end-to-end approach for slice isolation in 5g
networks. fundamental challenges,” in 2017 Federated Conference on
Computer Science and Information Systems (FedCSIS), 2017, pp. 783–
792.
[4] Z. Kotulski, T. W. Nowak, M. Sepczuk, and M. A. Tunia, “5g
networks: Types of isolation and their parameters in ran and cn slices,”
Fig. 11. Throughput of WireGuard and OpenVPN between the gateway nodes, Computer Networks, vol. 171, p. 107135, 2020. [Online]. Available:
averaged over 30 minutes with iperf. http://www.sciencedirect.com/science/article/pii/S1389128619304797
[5] P. Schneider, C. Mannweiler, and S. Kerboeuf, “Providing strong 5g
mobile network slice isolation for highly sensitive third-party services,”
in 2018 IEEE Wireless Communications and Networking Conference
(WCNC), 2018, pp. 1–6.
[6] Whitestack, “Open multi-vendor nfv showcase,” ETSI, Tech. Rep.
1st Edition, 2019. [Online]. Available: https://whitestack.com/posts/
results-multivendor-nfv-showcase/
[7] E. Community, “Osm scope, functionality, operation and
integration guidelines,” European Telecommunications Standards
Institute, Sohia Antipolis, FR, Standard Issue 1, 2019.
[Online]. Available: https://osm.etsi.org/images/OSM EUAG White
Paper OSM Scope and Functionality.pdf
[8] J. Donenfeld, “Wireguard: Next generation kernel network tunnel,” 01
2017.
[9] J. A. Donenfeld, “Wireguard: Next generation kernel network
tunnel,” Tech. Rep. revision 416d63b, 2018. [Online]. Available:
https://www.wireguard.com/papers/wireguard.pdf
[10] T. Lackorzynski, S. Köpsell, and T. Strufe, “A comparative study on
virtual private networks for future industrial communication systems,”
in 2019 15th IEEE International Workshop on Factory Communication
Systems (WFCS), 2019, pp. 1–8.
[11] S. Plaga, N. Wiedermann, S. D. Anton, S. Tatschner, H. Schotten, and
T. Newe, “Securing future decentralised industrial iot infrastructures:
Challenges and free open source solutions,” Future Generation
Computer Systems, vol. 93, pp. 596 – 608, 2019. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S0167739X18314043
[12] H. Gunleifsen, T. Kemmerich, and V. Gkioulos, “Dynamic setup of
ipsec vpns in service function chaining,” Computer Networks, vol. 160,
pp. 77 – 91, 2019. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/S1389128619300969
[13] A. Shameli-Sendi, Y. Jarraya, M. Pourzandi, and M. Cheriet, “Efficient
provisioning of security service function chaining using network security
defense patterns,” IEEE Transactions on Services Computing, vol. 12,
no. 4, pp. 534–549, 2019.
[14] M. Pattaranantakul, R. He, A. Meddahi, and Z. Zhang, “Secmano: To-
wards network functions virtualization (nfv) based security management
and orchestration,” in 2016 IEEE Trustcom/BigDataSE/ISPA, 2016, pp.
598–605.
[15] A. Esmaeily, K. Kralevska, and D. Gligoroski, “A cloud-based sdn/nfv
testbed for end-to-end network slicing in 4g/5g,” arXiv preprint
arXiv:2004.10455, 2020.
[16] G. M. Yilma, F. Z. Yousaf, V. Sciancalepore, and X. P. Costa, “On the
challenges and kpis for benchmarking open-source nfv mano systems:
Osm vs onap,” ArXiv, vol. abs/1904.10697, 2019.
Simen Haga Towards 5G network slice isolation with WireGuard and Open Source MANO

You might also like