Quick Start Guide

Facebook Network Appliance (FNA) – Cisco Switch SKUs

Hardware Installation
Before beginning, make sure you have received all
items per your SKU (see SKU Summary). You should
be experienced with racking and installing servers and
switches. Consult the manufacturer’s documentation for
assistance as needed. For a list of additional FNA literature
see Related Documentation.

1. Rack the Servers

a. Install the servers (4 are illustrated) into a compatible
rack (installing servers in a 2-post rack requires
appropriate conversion hardware) .
b. Connect power cables from each server to the power
distribution units (PDUs) . AC power supplies are
illustrated. For DC power installations, please consult
an electrician.

2. Rack the Switch

a. Install the switch into the rack with the airflow as port-
side exhaust (switch ports must face the rear of the
rack, as shown) . 4. Install Optics into Ports
a. Install SFP+ optical transceivers into ports 1, 2, 3
3. Connect the Network Cables from the Servers and 4 .
to the Switch
b. Note that QSFP Port 1 is internally multiplexed with
a. Connect the network cable (green sticker facing up) to
the (4) SFP+ uplink ports for both the Cisco Nexus
the NIC port (green sticker) . 3132Q and 3132Q-X. The (4) SFP+ ports and QSFP
b. Connect a server at the highest position in the rack Port 1 cannot be utilized at the same time .
(fna001) to QSFP+ port 32 on the switch. Utilize the
illustration as a reference for port connectivity for the
remaining servers (4 are shown).
c. For clusters over 4 servers, follow the port
assignments as shown .
d. When upgrading a 12, 16, or 20 server FNA cluster, a
QSFP to SFP+ adapter needs to be installed for each
additional group of four servers .
5. Connect the Switch to the Router
a. Connect fiber patch cables from the switch to your
router .

- 1-
Facebook Network Appliance Quick Start Guide (Cisco switch) Ver 2.2
6. Power On the FNA
a. Press the power buttons on all four servers .
b. Connect two power cables to the network switch (the switch
powers on when connected to a power source) .
c. The switch’s system status LED displays as amber while
powering up, then steady green for normal. Blinking amber
indicates that the switch has overheated.
d. If USB install is needed, refer to the FNA USB Boot Disk
Creation and IP Reconfiguration Guide.

7. Verify Server Operation

Verify server operation by checking the behavior of LEDs as illustrated
below on each server.

8. Check Optical Light Levels

Check the optical light levels to ensure that the cluster has adequate signal strength. Light levels need to be between -2 dB and
-7 dB. For deployments utilizing a Cisco Nexus switch, refer to Measuring the Optical Level through IOS, Document ID: 16150.

Network Configuration
1. Configure Point-to-Point Link
The point-to-point connection is configured on the Link-Aggregation Control Protocol (LACP) interface. Make sure the following
parameters are configured as follows:
●● Untag the Virtual LAN (VLAN) link-aggregation interface.
●● Set a static route to the allocated subnets via the link-aggregation interface.
●● Set the appropriate maximum transmission unit (MTU) for your router software:
○○ For Cisco IOS XR router software, set the MTU to 1514. Cisco IOS XR software includes both the L2 and L3
overhead in the interface MTU command.
○○ For all other router software, set the MTU to 1500.

- 2-
Facebook Network Appliance Quick Start Guide (Cisco switch) Ver 2.2

The FNA must have internet connectivity before BGP peering can be established. The BGP peering session does not need to be
routing before the FNA has internet connectivity.
The FNA kit is preconfigured with the IP addresses that were specified in the order. If IP addresses need to be changed at any
time, refer to the FNA USB Boot Disk Creation and IP Reconfiguration Guide.

2. Configure Border Gateway Protocol (BGP)

●● Use the IPv4 and IPv6 addresses originally indicated in the FNA order form.
●● (Optional) Enable Graceful Restart (GR). Enabling GR is not required, but it is recommended.
●● Enable External BGP (EBGP) Multihop.
●● Use the peering Autonomous System Number (ASN): 63293
●● Configure your router’s BGP settings.
●● FNA’s prefix should be announced to all Facebook peering connections.
●● All servable Facebook netblocks should be announced for network traffic optimization (completed by the latency based
routing sampling).1
1
Routes advertised to the FNA must be present in the global routing table with the same destination Autonomous System (AS). FNA applies inbound filtering to
reject the following address spaces: default route, bogons, RFC 1918, Facebook’s IP, CGNAT, and prefixes smaller than /24 for IPv4 and /48 for IPv6.

After the BGP settings are saved, the FNA cluster will begin downloading application data. Within about two business days,
BGP peering will be established:
●● The connection will be coming from the 16th IP address in each allocated subnet
(for example, 192.168.1.16, 2001:db8::10).
●● FNA will not advertise any routes / prefixes.

3. Contact FNA Operations

When installation is complete contact fna-ops@fb.com.

- 3-
Facebook Network Appliance Quick Start Guide (Cisco switch) Ver 2.2

SKU Summary
Item Qty Dimensions
Cisco Nexus 3132Q / 3132Q-X Switch2 1 1RU
HP ProLiant DL380p Gen8 / Gen9 / G9v2 Server
2
4 2RU
HP ProLiant DL380 Gen8 / Gen9 / G9v2 Server user manual 1 N/A
Cisco Nexus 3132Q / 3132Q-X Switch user manual 1 N/A
4-Post rail kit 4 N/A
C13 - C14 AC Power Cord (DC Cord not provided) 8 2 Meters
Network Cable 4 3 Meters
Cisco SFP+ 10G Optical Transceiver 2 N/A
Cisco QSFP 100G Adapter Note3 N/A
2
Model numbers vary per SKU.
3
Quantity as needed for upgrading server cluster (see Step 3d on page 1).

Related Documentation
To obtain the following documentation, visit the Network Partner Portal at https://partners.facebook.com/network.
Title Description
FNA Operations Troubleshooting Describes basic troubleshooting for the components of the Facebook Network
Appliance (FNA), and how to obtain support.
FNA USB-Based Software Installation Describes how to create a USB boot disk and use it to modify the Facebook Network
Appliance (FNA) IP configuration.
FNA White Paper Overview of Facebook Network Appliance (FNA), Facebook’s content caching
program.

- 4-