VLAN Trunk and Flood for Physical

Network Traffic Monitoring

DISCLAIMER:
Nutanix has tested a workaround to enable SPAN support of traffic from a single physical NIC to
a single VM running on AHV. The following requirements and limitations must be accepted
before proceeding.

● A dedicated physical NIC must be available on the AHV server where packets will be
received.
● A dedicated bridge must be configured by CLI on the AHV server.
● An OVS rule must be configured by CLI on the AHV server each time the AHV server
boots.
● When support for an official AHV SPAN configuration is available, the temporary
workaround may not be migrated.
● Most network traffic will be forwarded to the monitoring VM, but L2 link local traffic such
as STP and LLDP will not be forwarded.

OVERVIEW:
Customers desire a way to send traffic received on an AHV physical network interface to a
monitoring VM for security tools and packet captures. Customers will configure a SPAN session
on a physical top-of-rack switch that sends all interesting network traffic out a SPAN port. AHV
hosts will be connected to this output switch port as shown below. More details can be found in
PM-562.

This document outlines the trunk and flood workaround for sending SPAN traffic from a physical
NIC to a monitor VM.
CONFIGURATION
The trunk and flood configuration relies on a dedicated physical adapter and a new bridge that
must be added to the AHV host where the monitoring VM runs. By default all Nutanix AHV hosts
are installed with a single bridge, br0. The trunk and flood configuration adds a bridge, br1. If
br1 already exists on the AHV server, add a new bridge with a unique name that is 6 characters
or less.

Step 1 - Add Bridge - One Time

Add a dedicated bridge to the AHV host where the monitoring VM runs. This configuration must
be repeated on each AHV host where monitoring is desired. Follow the AHV Networking BPG
for more information.

ssh nutanix@CVM-IP
nutanix@CVM$ manage_ovs --bridge_name br1 create_single_bridge

Step 2 - Add Interface - One Time

Add a single interface to the newly created bridge. This is the physical interface on the AHV
host that is connected to the physical switch with a SPAN or mirror configuration enabled.
Replace ethX with the desired AHV host interface name.

nutanix@CVM$ manage_ovs --bridge_name br1 --interfaces ethX update_uplinks

Step 3 - Add Network - One Time

Configure a network that uses the new bridge, br1 Replace the network name, bridge name,
and VLAN number with the desired values for your network.

nutanix@CVM$ acli
<acropolis> net.create mirror_net vswitch_name=br1 vlan=100

Step 4 - Verify Network - One Time

Verify that the newly created network exists.

<acropolis> net.get mirror_net

mirror_net {
identifier: 0
logical_timestamp: 5
name: "mirror_net"
type: "kBridged"
uuid: "f59eaf42-b5e4-4839-80f4-addb40d22b60"
vswitch_name: "br1"
}
Step 5 - Add Monitoring NIC - One Time
Configure an additional trunked NIC on the monitoring VM that is directly attached to bridge br1
and the monitoring network. This bypasses the bridge chain. This also allows all VLANs to be
passed to the monitoring VM in addition to the previously specified VLAN.

In most cases this will be a NIC that does not have an IP address and is not used to send any
traffic. The monitoring VM should use another NIC connected to br0 (not shown here) for
management traffic.

<acropolis> vm.nic_create monitor-vm-trunk network=mirror_net type=kDirectNic

vlan_mode=kTrunked

Step 6 - Confirm Monitoring NIC - One Time

Confirm the monitoring VM has a NIC that is attached to the monitor network directly. Use the
name of your monitoring VM.

<acropolis> vm.get monitor-vm-trunk

monitor-vm-trunk {
...
nic_list {
ip_address: "10.21.238.172"
mac_addr: "50:6b:8d:96:ac:35"
network_name: "2381"
network_uuid: "6fbb6a17-a068-47c0-8e15-236b5ef15a8b"
type: "kNormalNic"
uuid: "9f0911a3-efb3-4ed7-be29-ef76c05d3c0d"
vlan_mode: "kAccess"
}
nic_list {
mac_addr: "50:6b:8d:ea:11:d8"
network_name: "mirror_net"
network_uuid: "f59eaf42-b5e4-4839-80f4-addb40d22b60"
type: "kDirectNic"
uuid: "ed8c360b-2845-46a7-a735-3e988d6e2320"
vlan_mode: "kTrunked"
...

Step 7 - Enable Flooding - Must be performed each time AHV host reboots
Enable flooding of inbound traffic by disabling MAC address learning on the physical AHV
interface connected to the monitoring bridge. This guarantees that all inbound traffic on the
monitoring bridge is flooded to all VMs on the monitoring bridge.

This step MUST be performed every time the AHV host reboots since the interface index may
change and the flood rule is removed when the host boots.
There are two parts to this step. We need to find the index of the physical interface in AHV, then
enable flooding in OVS for that interface.

7a. Find the interface index with the following:

nutanix@CVM$ ssh root@192.168.5.1
root@AHV# ovs-ofctl show br1

Look for the index number of the desired ethX interface.The value will be in the following format.

(index)ethX

Note this index for the next step.

7b. Enable flooding in the ethX interface index, replacing <index> with the integer value:

root@AHV# ovs-ofctl add-flow br1 table=0,priority=100,in_port=<index>,actions=FLOOD

Confirm the actions=FLOOD rule exists in the desired bridge:

root@AHV# ovs-ofctl dump-flows br1 | grep FLOOD