End to End MTU

Designing For 5G
Network
CCXX, PCG, PCC, FABRICS, NFVI, SWITCHES AND ROUTERS

Ehsan Ghasemi
ERICSSON | PRINCIPLE END-TO-END SOLUTION ARCHITECT
MTU and MSS are key and crucial values not only in the IP networks but also in the
telecommunication environments which I might highlight 5G core since the
purpose of introducing 5G system is the eye-catching speed boosting.
These values directly impact the user experience and almost all user plane KPIs.
In the local testing and with the local destination servers, the results are fascinating
however when we commercialize the network, the speed and bandwidth is not
satisfactory. When companies reach to 2 Gbps, they accept it however it is not the
expected bandwidth in 5G SA networks.
It is so much possible to reach 20 Gbps in the ideal environment and 4 Gbps for the
commercial subscribers.
Initially, lets demonstrate an end-to-end view of the 5G network with respect to
MTU-MSS. Obviously we have far more nodes and even different slices however
the below figure is a summery of that big picture in order to simplify the MTU
assignment.

Figure 1 5G on Hypervisor
 Figure 2 BareMetal 5G core

Now, we must determine the meaning of MTU and its kinds. MTU (maximum
transmission unit) gives the maximum size of IP packet that can be transmitted
without fragmentation. The MTU should be designed and configured carefully in
the transport network, infrastructure, and applications to avoid fragmentation,
packet drop, jitter and delay.
In the transport network, when it is possible, the MTU should be uniformly
configured to support 9000 bytes for on all relevant interfaces of routers and
switches to support Jumbo frames.
MTU in different router models may include different layers. Interface MTU in
some router models only includes payload while some router models also contain
L2 headers. Please note, the different software versions may also have different
definition for interface MTU for the same router model.
IP MTU in different routers have the same meanings which only includes the IP
packets. It is important to align the interface MTU settings in the transport
network. If the interface MTU includes both L2 header and payload, 14 bytes
should be added additional to support 9000 bytes for payload.
Take an example, to support 9000 bytes for payload packets, the ASR9K should
configured 9014 bytes in the main interface. The dot1q sub-interface will inherits
the MTU from the main interface and add 4 bytes automatically for dot1q header.

Interface MTU Interface type Comments

BE6 9014 Main interface Configure manually

towards 5G Core

BE6.802 9018 Dot1q sub- Inherited from BE6, add 4 bytes

interface for dot1q header.

Routers on the transport network should be also configured to fragment those

packets with the DF bit set. Since the path MTU discovery is not seen feasible
(some transport providers would configure their routers to discard ICMP
message), the MTU mismatching issue needs to be manually solved on per
application level.
It is recommended to set the MTU to support 9000 bytes on the interfaces of all
passed switches/routers in the transport network to support Jumbo frames.
These interfaces might be the data fabric layers including Leaf, spine and boarder
leaf.
The DPDK vNIC IP MTU is set to 2090 bytes if not specific. The SR-IOV PF IP MTU is
set to 2140 bytes. The Calico is set to 2090 bytes by default. So, The MTU setting
on application interface which in the primary networking should not exceed 1500
bytes to avoid packet fragmentate by Calico. For the application interface in the
secondary networking, the MTU setting should not exceed the MTU size in the
worker node. For the application interface using SR-IOV VF, the MTU setting
should not exceed the MTU size of SR-IOV PF.
It is recommended to set the Application interface MTU not exceed 1500 bytes
which in the primary networking. It is recommended to set the Application
interface MTU which in the secondary networking not exceed worker node
interface MTU.
CCRC should
be 1500
-- DPDK
vNI C I P MTU
2090 bytes

primary network should not 9018

Support Jumbo
exceed 1500 bytes frames
secondary net work should Be
2090
9216
PCG should 9018
be 2090 Support Jumbo
SR-I OV PF frames 9018
I P 2090 Support Jumbo
frames

k8e MPLS-Transport
Control Fabric
9216
9018
PCC should Support Jumbo
PCG be 1500 frames
-- DPDK
vNI C I P MTU
DCGW IP backbone
2090 bytes
9018
Support Jumbo
Data Fabric frames

Internet Transport
PCC

Firewall

Figure 3 MTU size allocation

CNF MTU size

The application should configure the MTU on the application and application
interface manually to avoid the MTU mismatch along the transport path, or the
usage of Path MTU discovery. The MTU setting in the application should consider
the MTU within transport network and the NFVI/Data Center.
For the user plane application, there are two MTU configurations in the PCG,
which is GTP MTU and interface MTU. The GTP MTU is used to measure the non-
encapsulated payload received on N3 or N6. The interface MTU represents the
MTU of the link that an interface is connected to. If the incoming packet is bigger
than the MTU, it will be dropped or fragmented which decide by PCG. The max
MTU supported by GTP MTU is 2900, and the max MTU supported by interface
MTU is 3022.
The MTU setting for the interface MTU should not exceed the minimum MTU size
supported by the transport network and NFVI/Data center. The IPv4 MTU setting
for the GTP MTU should use interface MTU – 20 bytes (IPv4 header) – 8 (UDP
header) – 16 (GTP header).
The TCP Maximum Segment Size (MSS) for the UE is depended on the Minimum
MTU supported by the transport network and NFVI/Data center. The IPv4 TCP
MSS should no bigger than the Minimum MTU – 44 (IPv4 header + UDP header +
GTP header) – 40 (IPv4 + L4 header) – IPsec header (variable).
PCG Interface MTU

PCG GTP MTU

L1 L2 IP Header UDP Header GTP Header IP Header L4 Header User Data

variable 20 btyes 8 btyes 16 btyes 20 btyes 20 btyes TCP MSS

Payload leave UE

Payload arrive PCG