An Analysis of Container-based Platforms for NFV

Sriram Natarajan, Deutsche Telekom Inc.

Ramki Krishnan, Dell Inc.
Anoop Ghanwani, Dell Inc.
Dilip Krishnaswamy, IBM Research
Peter Willis, BT Plc
Ashay Chaudhary, Verizon

1
 Virtual Machine vs. Container Stack
KVM Container-stack
VNF

Libraries VNF

Guest-OS Libraries
Hypervisor Container Engine
Host-OS Host-OS

• Lightweight footprint: Very small

• Deployment time:
images with API-based control to
Rapidly deploy
automate the management of services Pod
Container B (container
applications with minimal
Container A
(Application (Application group) A run-time requirements
• Resource Overhead: Lower use of + + (Application
system resources (CPU, memory, etc.) Libraries) Libraries) +
Libraries)
• Updates: Depending on
by eliminating hypervisor & guest OS
requirements, updates,
overhead
Container Engine failures or scaling apps
can be achieved by
scaling containers
Kernel Functions and Modules: up/down
Namespaces, cgroups, capabilities, chroot, SELinux
Host-OS
2
VM based Network Functions
Key Challenges

3
 Service Agility/Performance
• Provisioning time: VNF VNF VNF
– Hypervisor configuration Libraries Libraries Libraries
– Spin-up guest OS
Guest-OS Guest-OS Guest-OS
– Align dependencies between Guest-OS
& VNFs Hypervisor

Host-OS

• Runtime performance overhead:

– Performance proportional to resource allocated to individual VMs (throughput,
line rate, concurrent sessions, etc.)
– Overhead stems from components other than VNF process (e.g. guest OS)
– Need for inter-VM networking solution
– Meeting SLAs requires dynamic fine tuning or instantiation of additive features,
which is complex in a VM environment

4
 Portability/ Elasticity/Scalability
• Porting VNFs require:
– Identifying suitable nodes for new VNF
instances (or re-locating existing
instances). For example, resource types, VNF VNF
available capacity, guest OS images,
Libraries Libraries
hypervisor configs, HW/SW accelerators,
etc.) Guest-OS
Same
Guest-OS
– Allocating required resources for new
Hypervisor
instances Hypervisor
Re-config
– Provisioning configs to components in the Host-OS (vCPU, RAM,
guest OS, libraries and VNF Host-OS
SSL accelerator)

• Elastic scalability needs are driven by

workloads on the VNF instances, and
stateful VNFs increase the latency to
spin up new instances to fully
working state.
5
 Security/Isolation

VNF
VNF VNF VNF ✗ If VNF is compromised
Securely recover
with minimal or no Libraries Libraries Libraries (misconfiguration,
downtime etc.), how to securely
Guest-OS Guest-OS Guest-OS
(reschedule VNF) quarantine the VNF,
Hypervisor
but ensure continuity
of other VNFs?
Host-OS

Guarantee complete isolation across
resource entities (hardware units,
hypervisor, protection of shared
resource, isolation of virtual networks,
L3 cache, QPI, etc.)
Resource hungry VNF can starve the
shared resources (noisy neighbor
effect) that are allocated to other VNFs;
Need to monitor and cut-off hungry
VNF usage

6
 Containerized Network Functions
Key Benefits, Challenges and Potential Solutions

7
Service Agility/Performance/Isolation (1)

Key Benefits:
VNF VNF VNF
C B A - Containers can provide better
service agility (e.g. dynamically
Container Engine provision VNFs for offering on-
demand services), and performance
Host-OS as it allows us to run the VNF process
directly in the host environment
Cluster
Management VNF
- Inter-VNF communication latency
VNF
E depends on inter-process
Tool D
communication option (when hosted
Scheduler
Container Engine in the same host)
Host-OS

8
Service Agility/Performance/Isolation (2)
Key Challenges:
- Isolation: Containers create a slice of
VNF VNF VNF the underlying host using techniques
C B A like namespaces, cgroups, chroot etc.;
Container Engine
several other kernel features that are
not completely isolated.
Host-OS - Resource Mgmt: Containers do not
provide a mechanism to quota manage
the resources and hence susceptible to
Cluster the “noisy neighbor” challenge.
Management VNF VNF
Tool D E Potential Solutions:
Container Engine - Kernel Security Modules: SElinux,
Scheduler AppArmor
Host-OS - Resource Mgmt: Kubernetes
- Platform Awareness: ClearLinux

9
 Elasticity & Resilience
Key Benefits:
VNF VNF VNF
Pod Pod Pod
- Auto-scaling VNFs or achieving
Container Engine service elasticity in runtime can be
simplified by the use of container
Replication Host-OS
based VNFs due to the lightweight
Controller
resource usage of containers (e.g.
Cluster Mesosphere/Kubernetes)
Management
Tool
VNF
Pod
VNF
Pod
VNF
Pod - Container management solutions
(e.g. Kubernetes) provide self-healing
Scheduler
Container Engine features such as auto-placement,
restart, and replacement by using
Host-OS service discovery and continuous
monitoring

10
 Operations & Management

VNF VNF VNF

Service
Pod Pod Pod Key Challenges:
Discovery Container Engine
- Containers are supported in
selective operating systems such as
Replication Host-OS Linux, Windows and Solaris
Controller - In the current range of VNFs, many
Cluster don’t support Linux OS or other OSes
such as Windows and Solaris
Management
VNF VNF VNF
Tool Pod Pod Pod
Potential Solutions:
Scheduler Container Engine - Hybrid deployment with VMs and
containers can be envisioned, e.g.
Host-OS leverage ideas from Aptible
Security technology currently used for
applications

11
Conclusion and Future Work

12
 Conclusion and Future Work
• Use of containers for VNFs appears to have significant
advantages compared to using VMs and hypervisors,
especially for efficiency and performance
– “Virtual Customer CPE Container Performance White Paper,”
http://info.ixiacom.com/rs/098-FRB-840/images/Calsoft-Labs-CaseStudy2015.pdf
• Test Setup:
– COTS server with Intel Xeon E5-2680 v2 processor
– Virtual CPE VNFs (Firewall etc.) fast path optimized using Intel DPDK
– Measured L2-L3 TCP traffic throughput per core
• VM (KVM) environment with SRIOV -- 5.8Gbps
• Containers (LXC) environment -- 7.2Gbps
– ~25% PERFROMANCE IMPROVEMENT OVER VMs
• Opportunistic areas for future work
– Distributed micro-service network functions
– VNF Controller discovery/management/etc. standardization
– etc.

13
 Call for Action
• Address aforementioned challenges
• Further research to identify currently unknown challenges
• Vendors to consider developing container based solutions –
especially to support proof of concepts and field trials
• Reach consensus on a common framework for use of
containers for NFV
• Field trial container-based VNFs

14