VXLAN Part V: Flood and Learn

In this chapter, I am going to show how the VXLAN Flood & Learn mac learning process
works. I am going to ping from Host-1 to Host-2 and then walk through the Flood and
Learn process starting from ARP request. I am using the same Lab that was used in
VXLAN Part-IV. Configurations can be found from the VXLAN Part-1 and Part IV.

Figure 1: VXLAN Flood & Learn topology

Step 1: Host-1 starts pinging Host-2. Based on Host-2 IP address, Host-1 knows that
they are in same Layer 2 Broadcast domain. To be able to send a Ping request to target
in same Layer 2 domain, Host-1 has to resolve the Host-2 Mac address first and it sends
an ARP request, where destination mac is broadcast address (ff:ff:ff:ff:ff:ff) address and
source is Host-1 own Mac address (fa:16:3e:2d:98:c5).  When VTEP-101 receives the
frame, it will learn Host-1 Mac address from source Mac field. This is traditional Mac
learning process.

Step 2: Since received destination mac address of Ethernet frame is a broadcast

address, VTEP-101 first check for the VLAN/VNI (VXLAN Network Id = VXLAN Segment)
association and then sends the broadcast frame towards RP of Multicast groups attach to
the VNI. In our example, VLAN 10 belongs VNI1000, which uses Multicast group
238.0.0.10 for BUM traffic.

Through the traditional mac learning process and VLAN to VNI mapping VTEP-101
creates a Layer 2 entry [mac fa:16:3e:2d:98.c5 <=> VNI 10000 <=> E1/3]

VTEP-101 Configurations related to this is shown below.

Vlan 10
  vn-segment 10000
!
interface nve1
  source-interface loopback100
  member vni 10000
     mcast-group 238.0.0.10

Step-3: From the Capture 1, we can see how VTEP-101 handles the broadcast frame.
First, it adds a VXLAN header with VNI 10000 in front of the original Ethernet frame.
Then it adds a UDP header (VXLAN is MAC over IP-UDP) where the destination port is
well-known port 4789. Then it adds an outer IP header (tunnel header) with the
destination IP 238.0.0.10 and source IP 192.168.100.101 (Loopback 100 used by NVE 1)
and a new Ethernet header. Based on the Multicast group 238.0.0.10 OIL (Outgoing
Interface List), VTEP-101 sends encapsulated ARP request out of the Ethernet 1/1.

Leaf-101 MRIB
(*, 238.0.0.10/32), bidir, uptime: 00:25:52, nve ip pim
  Incoming interface: Ethernet1/1, RPF nbr: 192.168.0.11
  Outgoing interface list: (count: 2)
    Ethernet1/1, uptime: 00:07:51, pim, (RPF)
    nve1, uptime: 00:25:52, nve

Figure 2: ARP from Host-1 to Host

Capture 1: ARP request from Host-1

When Spine-11 receives the Multicast Packet from VTEP—101, it routes the packet based
on the group 238.0.0.10 OIL (Outgoing Interface List) towards VTEP-102.

Spine-11 MRIB
(*, 238.0.0.10/32), bidir, uptime: 00:54:51, pim ip
  Incoming interface: loopback238, RPF nbr: 192.168.238.1
  Outgoing interface list: (count: 3)
    Ethernet1/2, uptime: 00:08:10, pim
    Ethernet1/1, uptime: 00:08:10, pim
    loopback238, uptime: 00:08:11, pim, (RPF)

VTEP-102 receives the packet (Figure 3) and processes it since it has been joined to
Multicast Group 238.0.0.10 (It also has group 238.0.0.10 assigned to VN10000). It first
removes the outer IP header and based on the UDP destination port number 4789, it
knows that next header is VXLAN header. Then, based on VNI field in the VXLAN header,
it notices that the packet belongs to VLAN 10 (VLAN 10 is assigned to VNI 10000). As a
last two steps, it first removes the VXLAN header and sends the original Ethernet frame
out of all ports that belong to VLAN 10. It also learns, that source mac
fa:16:3e:2d:98.c5 (Host-1) belongs to VNI 10000 and is located behind the IP
192.168.100.101 (NVE 1 IP of VTEP-101).

Now VTEP-102 has a L2 entry [mac fa:16:3e:2d:98.c5 <=> VNI 10000 <=>
192.168.100.101].
Figure 3: VTEP-102 operation

Host 2 receives the ARP request and sends an ARP reply as a Layer 2 unicast to mac
address fa:16:3e:2d:98.c5 learned from ARP request source field. When VTEP-102
receives this frame, it sends it as an unicast to VTEP-101 based Layer 2 entry [mac
fa:16:3e:2d:98.c5 <=> VNI 10000 <=> 192.168.100.101]. It also learns the Mac
address of Host 2, and updates the Layer 2 forwarding table. Now it has two entries:

[mac fa:16:3e:2d:98.c5 <=> VNI 10000 <=> 192.168.100.101]

[mac fa:16:3e:0c:3d.b2 <=> VNI 10000 <=> E1/3]

From the Capture 2, we can see that the ARP reply is encapsulated with VXLAN header
(VNI 10000) and that the destination IP address is now VTEP-101 NVE 1 address
192.168.100.101 instead of Multicast Address 238.0.0.10. When the Spine-11 receives
the packet, it just routes it based on outer IP headers destination IP address to VTEP-
101. VTEP-101 receives the packet and notices that the destination address is its own IP
address, so it removes the outer IP header first. Then, based on the UDP destination
port number 4789, it knows that the next header is a VXLAN header. Based on VNI field
in VXLAN header it also knows that the inner frame belongs to its local VLAN 10 (VLAN to
VNI mapping). It removes VXLAN header switches frame based on mac table out from
E1/3 [mac fa:16:3e:2d:98.c5 <=> VNI 10000 <=> E1/3]. VTEP-101 check the original
frames mac address and creates a new entry to its mac table.

[mac fa:16:3e:2d:98.c5 <=> VNI 10000 <=> E1/3]

[mac fa:16:3e:0c:3d.b2 <=> VNI 10000 <=> 192.168.100.102]
Figure4

Capture 2: ARP reply from Host 2

From the last two captured frames, we can see that ICMP request and reply are sent
successfully over the VXLAN Fabric. Captures are taken from the link between VTEP-101
and Spine-102. 
Capture 4: ICMP echo request from Host 1 (connected to VTEP-101).

Capture 5: ICMP echo reply from Host 2 (connected to VTEP-101).