NSX-T 3.1 LB Encyclopedia-V1.3

NSX-T Load Balancer
Encyclopedia
(Updated for NSX-T 3.1)
August 2022
Dimitri Desmidt - Senior TPM NSX
ddesmidt@vmware.com
1
Agenda
1 LB Deployment Modes
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
7 LB Rules (HTTP or HTTPS)
8 Troubleshooting
2
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
3
Agenda 1. LB Deployment Modes
• Load Balancing Supported Topologies
• LB InLine Deployment
• LB OneArm Deployment
• IPv4 LB and IPv6 LB
• Configuration Steps
• Specific step for LB OneArm Deployment
• InLine and LB OneArm Deployment
• Load Balancer Deployment

• Configuration
• Scale Design
• Miscellaneous
• Service Chaining (LB + NAT + FW)
• LB High-Availability 4
Load Balancing Supported Topologies (1/2)
LB InLine Deployment
LB InLine Deployment 2 Enable LB on an existing Tier-1 GW

C
Note: LB not available on Tier-0 GW
LB-SNAT can be required depending on

traffic flows.
Tier-0
LB-SNAT required:
Overlay
• Clients and Servers are connected to
or VLAN same T1-Dowlink (Overlay) 1
Tier-1+LB Tier-1+LB 2 LB-SNAT not required:

C
• Other use cases 2
Overlay Overlay VLAN VLAN

1 1 1 1 2 2
S S C C C S S C C
Note: VIP can be placed in any subnet:
C
• Linked-segment (Downlink) or Service
Server Pool Server Pool Interface (CSP)
• A new dedicated network as a loopback
interface
• T0 uplink subnet 5
Load Balancing Supported Topologies (2/2)
LB OneArm Deployments
LB OneArm Deployments Deploy dedicated One-Arm Tier-1 GW for

Load Balancer
LB OneArm using T1 LB OneArm using T1 Note: LB not available on Tier-0 GW
Service Interface Uplink Interface
Can be deployed on Overlay or VLAN
Tier-0 Tier-0
LB-SNAT always required
Physical T1+LB
Tier-1 Tier-1
Router Note: VIP can be placed in any subnet:
Overlay
or VLAN VLAN • Service Interface (CSP)
• A new dedicated network as a loopback
S
interface (require manual routing
S S S S S
advertisement)
Server Pool Server Pool Server Pool
T1+LB T1+LB
6
Configuration Steps (1/3)
Specific step for LB OneArm Deployment using T1 Service Interface
For LB OneArm using T1 Service Interface

Important Point:
• Under Networking – Tier-1 Gateways Even if no overlay is used (everything is
VLAN), Overlay must still be on
– Add Service Interface to New Tier-1 configured on the Edge Nodes (see notes
for more information).
Select Overlay or VLAN
Segment
– Configure its Default Gateway

Tier-1 or
Physical Router IP@
Default Route
7
Specific step for LB OneArm Deployment using T1 Uplink Interface
For LB OneArm using T1 Uplink Interface

• Under Networking – Tier-1 Gateways
– Attach Tier-1 to Tier-0
Select T0 to attach it to
8
InLine and LB OneArm Deployment
Configure
• Load Balancer (see section below "Load Balancer Deployment")
• Monitor (see section "Monitors and Server Pools")
• Pool (see section "Monitors and Server Pools")
• Virtual Servers (see section " Layer4 VIP, " Layer7 HTTP VIP", and " Layer7
HTTP VIP")
9
IPv4 LB and IPv6 LB
Dual Stack Load Balancing
IPv4 Load Balancing support

LB IPv4
Clients LB Server
IPv4 IPv4
S Pool
Clients VIP Pool Members

IPv4 IPv4 IPv4
IPv6 Load Balancing support

LB IPv6
Clients LB Server
IPv6 IPv6
S Pool
Clients VIP Pool Members

IPv6 IPv6 IPv6
10
Load Balancer Deployment
Configuration
Create Load Balancer and attach it to existing Tier-1

• Under Networking – Load Balancing – Load Balancers
– Add Load Balancer
Select Tier-1
Size of LB (Small/Medium/Large/XLarge).
See NSX-T LB Admin Guide for LB scale of each LB size (# of VIP, Pools, etc)
11
Scale Design (1/2)
https://configmax.vmware.com/ The Edge Node hosts LB service (active/standby) based on its Tier-1
(active/standby) with LB attached.
Both LB active and standby are consuming resources in the Edge Node.
So for instance in "1 Edge VM - Large", in NSX-T 2.4 you can have up to:
"40 LB-Active", or "40 LB-Standby", or "20 LB-Active + 20 LB-
Standby", etc
LB Large have a factor of 40, LB Medium have a factor of 10, LB Small have a
factor of 1.
So 1 Edge VM – Large can host for instance: "40 LB Small" or "2 LB Medium
+ 20 LB Small" but not "3 LB Medium and 11 LB Small".
Load Balancer scale/provisioning is NOT affected / impacted by other

services hosted on Edge Nodes (i.e. Tier0, VPN, etc...)
Max # of rules per VIP = 512.

Note: This value is the same whatever the Edge Node or LB form factor.
12
Scale Design (2/2)
# of LB and #VIP per LB

• Recommendation
Dev Prod
None of them
Performance High LB config churn
(no Perf nor churn)
Either design is fine:
. Dedicated LB on dedicated Edge . Multiple LBs per Edge Node . Many VIPs on limited LBs per
. Many LBs per Edge Node
Node . and Limited # of VIPs per LB Edge Node
. and/or many VIPs per LB
13
Miscellaneous
Service Chaining (LB + NAT + FW)
Tier-1 offers different centralized

Server Pool services:
S • NAT
• Firewall
S
• Load Balancing
S
Tier-1
1 2 3 Service Chaining order is:
• Ingress:
• DNAT – FW – LB
1: DNAT: Client-IP@ => L3-DNAT-IP@ (translated to L3-Internal-VIP-IP@) Note: If DNAT is configured with
NAT_Pass, FW is skipped but not LB.
2: FW: Client-IP@ => L3-Internal-VIP-IP@ (allowed)
3: LB: Client-IP@ => Pool-Member-IP@ • Egress
• LB – FW – SNAT
14
Miscellaneous
LB High-Availability (1/3)
Active / Hot-Standby per LB
LB HA heartbeat per LB done by Edge

Node
Active Hot-Standby
• Failover after 3 successive failures
– 3 secs failover on EN-VM
– 0.9 sec on EN-BM
Hot-Standby Active
Very limited data plane impact thanks
Edge Node 1 Edge Node 2 to synch of LB State
LB HA messages • Healthcheck State
per LB
• Source-IP Persistence State
Edge Cluster • L4 Flow State
15
Miscellaneous
Active / Hot-Standby per LB
(0.9 sec later on EN-BM)

(3sec later on EN-VM) LB HA heartbeat per LB done by Edge
Active
Node
Active Hot-Standby
• Failover after 3 successive failures
– 3 secs failover on EN-VM
– 0.9 sec on EN-BM
Hot-Standby Active
Very limited data plane impact thanks
Edge Node 1 LB HA messages Edge Node 2 to synch of LB State
per LB • Healthcheck State
(every 0.3 sec on EN-BM)
(every 1 sec on EN-VM) • Source-IP Persistence State
Edge Cluster • L4 Flow State
16
Miscellaneous
UI
Information on Edge Node running LB
Select Tier-1 hosting LB and click on "Auto Allocated" service:
• UI/API
• CLI
CLI When synch is needed, the steps are:

1. "HEALTH CHECK" (sync of Monitors)
lab1-edge1> get load-balancer cc81d6b1-9c2c-431f-b726-85846c6c5cde high-availability-state 2. "PERSISTENCE" (sync of source-ip persistence table)
LB HA Sync State: ACTIVE, FULL_SYNC - NOT STARTED 3. "COMPLETE" (sync finished – this state is only for a brief time)
4. "NOT STARTED" (sync idle)
Note: Sync of L4 sessions are automatically done by Edge engine (datapathd) with
Status: Information about the Edge FW flow sync.
Active or Standby Sync of LB State 17
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
18
Agenda 2. Monitors
• Type of Monitors
• Active Monitor
• Passive Monitor
• Active Monitor Configuration

• ICMP
• TCP
• UDP
• HTTP
• HTTP
• Passive Monitor Configuration
19
Type of Monitors (1/3)
Monitoring server health status
Active Monitor Passive Monitor
LB periodically sends a health monitor message to LB passively observes server responses to detect
pool members failures
Supported health monitor types Failure detection methods

• ICMP, TCP/UDP, HTTP, HTTPS • TCP connection errors
• ICMP unreachable messages
• SSL connection errors
20
Active Monitor - Detect Pool Member status (based on LB probe)
LB Monitor Server Pool 1. LB generates a Monitor request to

1
Probe S
each Pool Member
S
2. LB validates the Pool Member
2 Pool Member
Response S response
T1+LB
LB (T1) generates Monitor Probe with:
To T0 To T0 • LB InLine:
LB InLine LB OneArm
T1-uplink IP@ (100.64.x.x)
100.64.x.x T1-NoLB T1 Service • LB OneArm:
Interface
T1-Service Interface IP@
10.1.1.0/24 10.1.1.1
LB Note: In case of Pool configured with LB-
Monitor 10.1.1.7
Probe 10.1.1.1 LB InLine SNAT IP_Pool, then the LB Probe uses
LB
S S
Monitor
the IP_Pool IP@.
10.1.1.0/24 Probe
Server Pool LB OneArm
S S Server Pool
Multiple Active Monitors can be associated to a
Pool (all active health monitors must pass
successfully for the member to be considered up)
21
Passive Monitor - Detect Pool Member status (based on Client data plane traffic)
LB monitors data plane traffic.

Server Pool
S
SYN
If a pool member fails to connections
SYN (default = 5),
S
then pool member is marked down for a
RST S period (default = 5 sec).
VIP
L4 or L7
In case of heavy traffic with thousands of connections per second, detection can be very fast
(milliseconds).
22
Active Monitor Configuration
ICMP
Create ICMP Monitor

• Under Networking – Load Balancing – Monitors
– Add Active Monitor – ICMP LB Monitor Interval How long does LB Monitor waits
for the Pool Member response
How many consecutive failures before How many consecutive successes

LB Monitor considers the Pool Member before LB Monitor considers the Pool
down Member up
Size of ICMP packet
23
TCP
Create TCP Monitor

How long does LB Monitor waits
– Add Active Monitor – TCP LB Monitor Interval
Optional:
TCP Pool Member port tested.
If not specified, Pool member port is used by Monitor.
How many consecutive failures before LB

Monitor considers the Pool Member down
How many consecutive successes
before LB Monitor considers the Pool
Member up
Optional: TCP Payload packet Optional: Payload LB expects from

sent from LB Pool Member response
24
All below pool member responses will get the
UDP pool member in DOWN state:
. No response
. icmp not reachable
. any other response than that “UDP Data
Expected”
Create UDP Monitor
– Add Active Monitor – UDP LB Monitor Interval
Optional:
UDP Pool Member port tested.


Member up
Payload LB expects from Pool

UDP Payload packet sent from LB Member response
25
HTTP (1/2)
Create HTTP Monitor

– Add Active Monitor – HTTP (1/2) LB Monitor Interval
Optional:

Member up
See next slide
26
HTTP (2/2)
– Add Active Monitor – HTTP (2/2)
Optional: Expected Response code.

By default: 2xx and 3xx are considered valid responses.
HTTP Method
(Get, Head, Options, Post, Put)
HTTP URL
HTTP Version (1.0 and 1.1 are supported.

If HTTP1.1, Host header is automatically added.
. For Pool Members using TCP port 80 or 443:
Host = " PoolMember-IP@"
. For Pool Members using other TCP ports: Optional: Validate specific string in Pool Member HTTP
Host = "PoolMember-IP@:Port" Response.
String can be at any position in the Response Body (no
limit of response body size).
Optional: Insert HTTP headers
In this example, HTTP Host with FQDN is added.
Optional: HTTP Body

For HTTP Method PUT/ or POST.
27
HTTPS (1/3)
Create HTTPS Monitor

LB Monitor Interval How long does LB Monitor waits
– Add Active Monitor – HTTPS (1/3) for the Pool Member response
Optional:

Member up
See next slides
28
HTTPS (2/3)
– Add Active Monitor – HTTPS (2/3)
Optional: Expected Response code.

By default: 200 and 3xx are considered valid responses.
HTTP Method
HTTP URL
HTTP Version (1.0 and 1.1 are supported.

If HTTP1.1, Host header is automatically added.
. For Pool Members using TCP port 80 or 443:
Host = " PoolMember-IP@"
. For Pool Members using other TCP ports: Optional: Validate specific string in Pool Member HTTP
Host = "PoolMember-IP@:Port" Response
Optional: Insert HTTP headers

In this example, HTTP Host with FQDN is added.
Optional: HTTP Body

For HTTP Method PUT/ or POST.
29
HTTPS (3/3)
– Add Active Monitor – HTTPS (3/3)
Enable HTTPS to send

HTTPS Monitor to Pool Members
Optional: Only if Pool Members HTTPS

server require a Client Certificate Optional: If specified, LB Monitor accepts only
Pool Member HTTPS certificates signed by this CA.
If disabled, LB Monitor accepts any Pool Member
HTTPS certificate (even self-signed).
What SSL (ciphers + protocols) are
presented by LB to Pool Members
30
Passive Monitor Configuration
Create Passive Monitor

– Add Passive Monitor
How many consecutive connection failure Once the Pool Member is marked down by
before LB Monitor considers the Pool Passive Monitor, how long does it remain
Member down unused by LB engine.
Note: If Active Monitor marked it down too, it
will not be used by LB engine even after that
timeout period.
31
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
32
Agenda 3. Server Pools
• Server Pool Packet Flow
• Transparent (No SNAT)
• LB-SNAT (Automap or IP Pool)
• Server Pool Configuration

• Minimum Configuration
• LB-SNAT: Transparent (No SNAT) or LB-SNAT (Automap or IP Pool)
• Static Membership
• Group Membership
• Other configuration
• Backup Servers
• Pool Member Max Concurrent Connections
• Acceleration with TCP Multiplexing
• Server Pool Monitoring/Statistics
33
Server Pool Packet Flow (1/2)
Transparent (No SNAT)
LB InLine Client IP is preserved.
1 2
Server Pool
Client-IP:5000 Client-IP:5000
 VIP:80  Pool-IP:80 S
S
4 3
LB
S
InLine
VIP:80 Pool-IP:80
 Client-IP:5000  Client-IP:5000
34
Server Pool Packet Flow (2/2)
LB-SNAT (Automap or IP Pool)
LB InLine LB OneArm To T0 Client IP replaced by LB-SNAT:

To T0
• LB-SNAT Automap:
T1-NoLB
100.64.x.x
T1 Service • LB InLine (LB-SNAT IP =
Interface
10.1.1.1 T1-uplink IP@ (100.64.x.x))
10.1.1.0/24
• LB OneArm (LB-SNAT IP =
LB
10.1.1.7 T1-Service Interface IP@)
InLine
C C S S
4 • LB-SNAT IP Pool:
3
1 2 LB Server Pool • LB-SNAT = IP Pool IP@
OneArm
1 2
C C S S 4 3 Automap:
For LB-Pools with small/medium load
Server Pool
(below 1k new connections per minute).
IP Pool:
1 Client-IP:5000 2 LB-SNAT:6000 3 Pool-IP:80 4 VIP:80 For LB-Pools with large load.
 VIP:80  Pool-IP:80  LB-SNAT:6000  Client-IP:5000
35
Server Pool Configuration
Minimum Configuration (1/4)
Create Server Pool

• Under Networking – Load Balancing – Server Pools
See next slides
– Add Server Pool "Static Membership"
or "Group Membership"
LB algorithm
How Pool Member is selected
See next slide "LB-SNAT"
Optional:
Active Monitor used on Pool Members
Optional:
Passive Monitor used on Pool
Members
36
• LB-SNAT
– Transparent (No SNAT) or LB-SNAT (Automap or IP Pool)
Algorithms:
Round Robin, Weighted Round Robin, Least Connection, Weighted
Least Connection, IP Hash.
Comment about Slow Start for Least Connection, Weighted Least
Connection algorithms in the Notes.
LB-SNAT configuration.
. Disabled: No LB-SNAT
. Automap: T1-uplink IP@ (100.64.x.x) or T1-ServiceInteface IP@
. IP Pool: List of IP@ used for SNAT (VIP IP@ can be used too)
Comment about scale in the Notes.
37
• Pool Members
– Static Membership Static Members
Optional: Port.
If not configured:
Pool Member Name IP@ . Pool must use an Active Monitor with Monitoring Port specified
Weight (only used in case of Weighted

LB algorithm in the Virtual Server
using the Pool). Pool Member State:
Bigger the weight, bigger the load on . Enabled: Pool Member used in the LB algorithm
the pool member . Graceful Disabled: Existing connections are still load balanced, and in case of persistence
(Source-IP or Cookie) new connections from old clients are still sent to that Pool Member
. Disabled: Existing connections are still load balanced, and no new connections are sent to that
Pool Member (even in case of persistence)
38
• Pool Members
– Group Membership Group Members
This is not the LB Pool status.

This is the Group Status (Group is correctly populated and can be used)
Select Group
Max IP used from the Group to populate the Pool Members list.
If not specified, all members are used up to max Pool Member capacity of LB.
Optional: What Port for Pool Members. If not configured, "Default Pool Member Ports" must be configured in Virtual Servers using
that Pool.
39
Other configuration - Backup Members (1/3)
If the number of Non-Backup Pool

Members go below a "Pool Min Active
Members" (default=1), then only
Backup Pool Members are used.
Connections to VIP
S Min Active
Members= Note: In case of VIP with Persistence
2 (Source-IP or Cookie), former clients with
S persistence on Backup Pool Members will
get their new connections to Backup Servers
VIP S – even when Non-Backup Pool Members are
back UP.
S
backup
S
backup
Server
Pool
40
• Backup Servers
– Configure Pool Members backup
41
• Backup Servers
– Configure Minimum Active Members
If the number of Non-Backup Pool Members go

below that threshold, then only Backup Pool
Members will be used.
42
Other configuration - Pool Member Max Concurrent Connections (1/2)
Load Balancer will stop load balancing

new connections to a specific Pool
Connection to VIP Member if this one reached its "Max
Concurrent Connections".
S Pool Member
Max Concurrent Note about Pool Backup Member option:
Connections = Once all the Non-Backup Members reached their
S N Max Concurrent Connections, if the pool is
configured with Backup Members, those will start
VIP being used.
S
Note about VIP Persistence option:
Server In case of Persistence (Source IP or Cookie), a
former client that was load balanced to a server
Pool
that reached its Max Concurrent Connections will
still have its new connections load balanced to that
same server.
In that case, the server will have more concurrent
connections than its configured Max Concurrent
Connections.
43
Other configuration - Pool Member
Max Concurrent Connections (2/2)
• Pool Member Max Concurrent Connections
– Configure Pool Members Max Concurrent Connections
Pool Member will not have more than this number of

concurrent connections.
Note: Only exception is in case of Persistence (Source-
IP or Cookie) when a former client has connections to
that server and sends new connections.
44
Other configuration – Acceleration with TCP Multiplexing (1/2)
All Clients TCP connections Persistent TCP connections TCP Multiplexing is activated only if
are terminated on LB between LB and Pool Members Pool is used in a L7 VIP.
Server Pool
S With TCP Multiplexing, LB keeps TCP
connections persistent to Pool
S
Members.
LB S
L7 VIP only Different clients requests will be
forwarded to the same persistent Pool
Client1:2000 => VIP1:80 LB-SNAT:6000 => S1:80
Member connection
GET /page1.php GET /page1.php
Response Response Servers have less concurrent

connections, as well as less
Client2:3000 => VIP1:80 Same LB-SNAT:6000 => S1:80 opening/closing connections.
GET /page1.php GET /page1.php Thus they are faster.
Response Response
45
Other configuration – Acceleration
with TCP Multiplexing (2/2)
• TCP Multiplexing
– Configure TCP Multiplexing
Highly recommended
See Note.
Enable TCP Multiplexing
Maximum Persistent Concurrent connections

for the whole Pool.
IMPORTANT NOTE: From NSX-T 3.0, we suggest
keeping the default “Max Multiplexing
Connections” at ”6”.
See Notes for more explanation.
46
Server Pool Monitoring/Statistics (1/2)
Monitoring
Monitor of Server Pool

• Under Advanced Networking & Security – Load Balancing – Server Pools
– Select Pool Deeper Status information available on
Advanced UI.
Reason of the Pool

Member Status Down
For LB to run Active Monitor on Pool Members, make sure you have
47
the Monitor associated to the Pool + Pool attached to a VIP + VIP attached to LB + LB attached to T1
Server Pool Monitoring/Statistics (2/2)
Statistics
Statistics of Server Pool

• Under Networking – Load Balancing – Server Pools
– Expend Pool and Click on "View Statistics"
• Pools attached to L4-VIP work at layer4 => don’t

have HTTP (layer7) statistics For LB to run Active Monitor on Pool Members, make sure you have
• Rate statistics are last second rate statistics the Monitor associated to the Pool + Pool attached to a VIP + VIP attached to LB + LB attached to T1 48
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
49
Agenda 4. Layer4 VIP
• Layer4 VIP Packet Flow
• Layer4 VIP Configuration • LB NSX Groups auto-created

• Minimum Configuration
• Fast-TCP Profile
• Layer4 VIP Monitoring/Statistics
• Virtual Server L4 TCP
• Persistence
• Source-IP Persistence
• Port Range
• Sorry Server Pool
• Connection Throttling/Rating
• Access Log
50
Layer4 VIP Packet Flow (1/2)
LB InLine Layer4 Load Balancing is done at the

first Client packet (SYN)
1 2
Client-IP:5000 Client-IP:5000
Server Pool
 VIP:80  Pool-IP:80
(SYN) (SYN) S Note: Transparent Mode (for Client IP
preservation) is a Server Pool setting.
S
4 3
LB
S
InLine
VIP:80 Pool-IP:80
 Client-IP:5000  Client-IP:5000
(SYN-ACK) (SYN-ACK)
51
Layer4 VIP Packet Flow (2/2)
LB InLine LB OneArm To T0 Layer4 Load Balancing is done at the first

To T0
Client packet (SYN)
T1-NoLB
T1 Service
100.64.x.x Interface
10.1.1.1
10.1.1.0/24 Note: LB-SNAT Mode is a Server Pool setting.
LB
10.1.1.7
InLine
C C S S
4
3
1 2 LB Server Pool
OneArm
1 2
C C S S 4 3
Server Pool
Client-IP:5000 LB-SNAT:6000 Pool-IP:80 VIP:80

1  VIP:80 2  Pool-IP:80 3  LB-SNAT:6000 4  Client-IP:5000
(SYN) (SYN) (SYN-ACK) (SYN-ACK)
52
Layer4 VIP Configuration
Different connections from the same

client can be load balanced to
different Pool Members.
Connection1 Server
to VIP:80 S Pool
S
Connection2
VIP L4 S
to VIP:80
TCP:80
53
Use the default Fast-TCP profile, or Create a new one

• Under Networking – Load Balancing – Profiles Are the load balanced Clients connections synchronized
How long Clients idle with LB Standby.
– Add Application Profile – Fast TCP connections remain in Load In case of LB failover, this allows transparent failover
Balancer table. with no clients connections lost.
How long Clients connections after first

FIN or RST remain in the Load Balancer
table.
54
Create Virtual Server

• Under Networking – Load Balancing – Virtual Servers
– Add Virtual Server – L4 TCP
Load Balancer
VIP IP address VIP TCP Port hosting the VIP Pool load balanced
L4 TCP Profile
Optional:
Only required if Pool Members are NOT configured with Port information
(for instance using Pool with Group Membership)
AND if Pool Members TCP Ports is different than VIP Port. 55
Persistence - Source-IP Persistence (1/3)
In case of Source-IP Persistence

configuration, Load Balancer creates a
persistence table with Client IP@ and
Connection1
Selected Pool Member information.
Server Pool
to VIP:80 S
This table is used by the Load Balancer to
S send the same Client IP@ always to the
same Pool Member.
Connection2
VIP L4 S
to VIP:80
TCP:80 Note about Pool Member Down:
Let's say Client IP1 has a Source-IP Persistence to
Pool Member 1.
Then later that Pool Member1 is detected down
by LB Monitor.
New connections from Client IP1 will be load
balanced to another Pool Member and its
persistence entry will be updated with that new
Pool Member. So all future Client IP1 connections
will use that use Pool Member (even if old Pool
Member comes back up)
56
Use the default Source IP Persistence profile, or Create a new one

• Under Networking – Load Balancing – Profiles
– Add Persistence Profile – Source IP
Same Source IP Persistence Table is shared
among different VIP
. With this option enabled:

If Source IP Persistence Table is full,
then oldest entry is replaced by new
entry
. With this option disabled:
new client connection will be rejected How long an idle client entry remains in the
Source IP Persistence Table.
Size of the Source-IP Persistence table
in the Notes.
Synchronize Source IP Persistence Table with LB Standby.
So even in case of LB failover, Client IP persistence remains.
57
Same configuration as "L4 VIP Minimum Configuration", with the extra configuration
Source IP Persistence
Source IP Persistence Profile
58
Other configuration - Port Range (1/2)
Allow a simple configuration of a

single VIP + Pool for the use case of
application running on multiple ports
Connection1 Server Pool
to VIP:80 to Pool1:80
S TCP 80-90
S
Connection2 to Pool2:81
VIP L4 S
To VIP:81
TCP:Port-Range
(for instance 80-90)
59
Other configuration - Port Range (2/2)
VIP Port Range
Optional: Pool Member Port Range

If not specified, same VIP Port Range is used
for Pool Members.
60
Other configuration - Sorry Server Pool (1/2)
If Server Pool is dead, then Load Balancer

will load balance clients to a (Sorry) Server
Pool.
Connections
to VIP:80 Server Pool1
S Note: In case of L4-VIP with Persistence (Source-
IP), persistence is not offered on Sorry Server Pool.
S When Main Server Pool is Down, Clients will be
load balanced without persistence to Sorry Server
VIP L4 S Pool.
TCP:80 Then when Main Server Pool becomes UP again,
immediately Clients are load balanced to Main
Sorry Server Server Pool with Persistence.
S Pool Pool2
61
Same configuration as "L4 VIP Minimum

Configuration", with the extra configuration
Sorry Server Pool
62
Other configuration - Connection Throttling/Rating (1/2)
Load Balancer can protect against excessive

load.
Connections to VIP:80
Load Balancer will stop load balancing new
Server Pool
connections if VIP reached its:
S
• "Max Concurrent Connections"
S • or "Max New Connection Rate".
VIP L4 S
TCP:80 Note: VIP L4 Persistence (Source IP) does not
override this option.
If a former client that is load balanced to a Pool
Member sends a new connection when the VIP L4
reached its limit, this former client new
connection will be discarded.
63
Other configuration –
Connection Throttling/Rating (2/2)
Maximum Concurrent Connections

rate on the VIP
Maximum Concurrent
Connections on the VIP
64
Other configuration - Access Log (1/2)
Load Balancer generates an Access Log L4

Connection1 Server Pool for each connection load balanced.
to VIP:80 S
This Audit Log can be sent also to an
S
external syslog server if the Edge Node is
VIP L4 configured with a syslog server.
S
TCP:80
To limit the number of logs, only sessions

rejected can be logged (See next slide
Notes “Log Significant Event Only”)
Access Log
lab2edge1> get load-balancer a339fa40-8f30-4ff7-b66a-2090b5542ee3 virtual-server
e71b88ea-aa3b-4b77-9af8-7aa27daa81e1 access-log follow
Operation.Category: 'LbAccessLog', Operation.Type: 'TCP', Lb.UUID: 'a339fa40-8f30-
4ff7-b66a-2090b5542ee3', Lb.Name: 'LB1', Vs.UUID: 'e71b88ea-aa3b-4b77-9af8-
7aa27daa81e1', Vs.Name: 'VIP1', Vs.Ip: '30.30.30.6', Vs.Port: '80', Pool.UUID:
'210a9e2b-d495-4d3e-ad47-f42475b4ca95', Pool.Name: 'Pool1', PoolMember.IP:
'10.1.1.11', PoolMember.Port: '80', Client.Ip: '10.114.218.184', Client.Port:
'45172', Snat.Ip: '30.30.30.6', Snat.Port: '4184', Error.Reason: 'No error’
65
Same configuration as "L4 VIP Minimum

Access Log
Only error logs

(see Notes)
66
LB NSX Groups auto-created (1/2)
Explanation
Server Pool
S NSX-T Manager creates automatically
Groups for:
S • Server Pool
VIP L4 • VIP
S
TCP:80
This simplifies the creation of security:

NSX-T Groups created: • T1 Gateway Firewall
• LB Pool Group (NLB.PoolLB.[Pool_Name][LB_Name]) Allow traffic from Clients to "LB VIP" (=
Group Member IP Addresses: NLB.VIP.[VIP_Name])
• If Pool configured with no LB-SNAT (Transparent): 0.0.0.0/0
• DFW rules
• If Pool configured with LB-SNAT Automap: T1-Uplink IP 100.64.x.y
On Pool Members, allow traffic from "LB" (=
+ T1-ServiceInterface IP
NLB.PoolLB.[Pool_Name]
• If Pool configured with LB-SNAT IP-Pool: LB-SNAT IP-Pool
[LB_Name])
• VIP Group (NLB.VIP.[VIP_Name])
Group Member IP Addresses: Note: [Pool/LB/VIP names] with more than 12
• VIP IP@ characters will be truncated in UI (Group Display
Name). The Group ID is NOT truncated.
67
Visualization
Look at the NSX Groups auto-created by LB

• Under Inventory – Groups - Groups
[Pool-Bugzilla] is truncated with [Pool…zilla] in the Group Display Name.

However the Group ID of that object is still NLB.PoolLB.[Pool-Bugzilla][LB1]
68
Layer4 VIP Monitoring/Statistics (1/3)
Monitoring
Monitor of Layer4 VIP

• Under Advanced Networking & Security – Load Balancing – Virtual Servers Deeper Status information
available on Advanced UI.
– Select Virtual Server
For VIP to run, make sure you have

the Pool attached to a VIP + VIP attached to LB + LB attached to T1 69
Statistics
Statistics of Layer4 VIP

• L4-VIP work at layer4 => don’t have HTTP (layer7)

statistics
• Bytes Rate statistics are not supported on L4-VIP

Real Time Statistics
Real Time Statistics of Layer4 VIP

• Under Advanced Networking & Security – Load Balancing – Virtual Servers
– Select Virtual Server Real Time Statistics available
on Advanced UI.

the Pool attached to a VIP + VIP
attached to LB + LB attached to T1 71
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
72
Agenda 5. Layer7 HTTP VIP
• Layer7 HTTP VIP Packet Flow
• Layer7 HTTP VIP Configuration • Layer7 HTTP VIP Configuration

• Minimum Configuration • Other configuration
• HTTP Profile • Sorry Server Pool
• Virtual Server L7 HTTP • Connection Throttling/Rating
• Persistence • Increase Response Header Size
• Source-IP Persistence • Response Buffering
• Cookie Persistence • Access Log
• Generic Persistence
• LB NSX Groups auto-created
• X-Forwarded-For header (Client IP@ injected in
http header)
• Redirect Clients to HTTPS • Layer7 HTTP VIP Monitoring/Statistics
• NTLM Authentication / Server Keep-Alive 73
Layer7 HTTP VIP Packet Flow (1/2)
LB InLine Layer7 HTTP Load Balancing is done

1 Client-IP:5000 2 Client-IP:5000
 VIP:80  Pool-IP:80 after Client TCP handshake + HTTP
(TCP handshake) (TCP handshake) request.
Server Pool
(HTTP Request) (HTTP Request)
S By default on the server side, each TCP
connection is closed by LB after each
S request.
4 3
LB
S
InLine
VIP:80 Pool-IP:80 Note: Transparent Mode (for Client IP
 Client-IP:5000  Client-IP:5000 preservation) is a Server Pool setting.
(HTTP Response) (HTTP Response)
74
Layer7 HTTP VIP Packet Flow (2/2)
LB InLine LB OneArm To T0 Layer7 HTTP Load Balancing is done after

To T0
Client TCP handshake + HTTP request.
T1-NoLB
T1 Service
100.64.x.x Interface By default on the server side, each TCP
10.1.1.1
10.1.1.0/24 connection is closed by LB after each
LB request.
10.1.1.7
InLine
C C S S
4
3
1 2 LB Server Pool
OneArm Note: LB-SNAT Mode is a Server Pool setting.
1 2
C C S S 4 3
Server Pool
Client-IP:5000 LB-SNAT:6000 Pool-IP:80 VIP:80

1  VIP:80 2  Pool-IP:80 3  LB-SNAT:6000 4  Client-IP:5000
(TCP handshake (TCP handshake (HTTP Response) (HTTP Response)
+ HTTP Request) + HTTP Request)
75
Layer7 HTTP VIP Configuration
Server Pool Load Balancing is done at the Request

S level.
S Different requests from the same

client / same TCP connection are load
VIP L7 S balanced to different Pool Members.
HTTP:80
HTTP Request1 HTTP Request1

1 S1

2 S3
76
Use the default HTTP profile, or Create a new one

• Under Networking – Load Balancing – Profiles How long LB waits for Server HTTP Response before LB
How long Clients idle connections gives up and:
– Add Application Profile – HTTP remain before LB closes them . Close the connection to the pool member
(FIN). . Retry the request to another server
77

– Add Virtual Server – L7 HTTP
Load Balancer
L7 HTTP Profile
Optional:
AND if Pool Members TCP Ports is different than VIP Port.
78
Server Pool In case of Source-IP Persistence

S configuration, Load Balancer creates a
persistence table with Client IP@ and
S Selected Pool Member information.
VIP L7 S This table is used by the Load Balancer to

HTTP:80 send the same Client IP@ always to the
HTTP Request1 HTTP Request1 same Pool Member.
1
Note about Pool Member Down:
HTTP Response1 HTTP Response1 S1 Let's say Client IP1 has a Source-IP Persistence to
Pool Member 1.
HTTP Request2 by LB Monitor.
HTTP Request2
2 S1
balanced to another Pool Member and its
persistence entry will be updated with that new
Pool Member. So all future Client IP1 connections
will use that use Pool Member (even if old Pool
Member comes back up)
79
Use the default Source-IP Persistence profile, or Create a new one

– Add Persistence Profile – Source IP
Same Source IP Persistence Table is shared
among different VIP
. With this option enabled:

then oldest entry is replaced by new
entry
. With this option disabled:
new client connection will be rejected How long an idle client entry remains in the
Size of the Source-IP Persistence table
in the Notes.
80
Same configuration as "L7 HTTP VIP Minimum Configuration", with the extra configuration
Source IP Persistence
Source IP Persistence Profile
81
Persistence - Cookie Persistence (1/3)
Server Pool In case of Cookie Persistence configuration,

S Load Balancer inserts in its responses a
Cookie with Pool Member information.
S
Follow-up Clients requests have the Cookie
VIP L7 S and Load Balancer uses that information to
HTTP:80 send the Request to the same Pool
HTTP Request1 HTTP Request1 Member.
1
HTTP Response1 S1 Let's say Client IP1 has a Cookie Persistence for
+ Set-Cookie NSX-Cookie=S1 HTTP Response1 Pool Member 1.
by LB Monitor.
HTTP Request2
HTTP Request2 balanced to another Pool Member and Cookie
+ Cookie NSX-Cookie=S1
S1 persistence will be with that new Pool Member. So
2
all future Client IP1 connections will use that use
Pool Member (even if old Pool Member comes
back up)
82
Use the default Cookie Persistence profile,

or Create a new one
– Add Persistence Profile – Cookie Cookie Mode:
. Insert: Adds a unique cookie to identify the session
. Prefix: Appends to the Pool Member HTTP cookie information
Cookie Fallback Enabled: . Rewrite: Rewrites the Pool Member HTTP cookie information
Client HTTP Request with Cookie Same Source IP Persistence Table is shared
pointing to a Pool Member in DISABLED among different L7 VIP
or DOWN state is load balanced to new Cookie settings (for Cookie Insert mode only)
Pool Member UP and cookie is updated. . Domain: Cookie Domain
Option Disabled: . Path: Cookie Path
Client HTTP Request with Cookie . Type: Session Cookie (not stored and lost when
pointing to a Pool Member in DISABLED browser is closed) or Persistence Cookie (stored by
or DOWN state is rejected (502 HTTP browser and not lost when browser is closed)
Error) . Secure Flag: When enabled, it prevents browsers
from sending cookie over http adding Cookie
Cookie Garbling Enabled: option “secure” (only available in Cookie Mode
Pool Member IP information is encoded. Insert).
Option Disabled: . Max Idle Time: How long in seconds idle clients
Pool Member IP information is in clear keep persistence. If empty for Session Cookie,
text. persistence is always honored as long as browser is
New NSX-T 3.1 not closed.
. Max Cookie Age: Only for Session Cookie, how
HttpOnly Flag Enabled:
long in seconds clients keep persistence overall
Prevents a script running in the
from their very first request. If empty, persistence
browser from accessing the cookie
is always honored as long as browser is not closed.
adding Cookie option “httponly” (only
See Notes for more information. 83
available in Cookie Mode Insert)..
Same configuration as "L7 HTTP VIP Minimum

Cookie Persistence
Cookie Persistence Profile
84
Persistence - Generic Persistence (1/4)
Server Pool Generic Persistence is offered for

S customers who want persistence on other
than source IP or NSX Cookie.
S Generic Persistence can be done on any
specific Client Request field, such as Cookie
VIP L7 S JSESSION or XFF header.
HTTP:80
HTTP Request1
HTTP Request1 Load Balancer creates a persistence table
1
with that request or response field (like
HTTP Response1 HTTP Response1
+ Set-Cookie JSESSIONID = xxx S1 Application Set-Cookie JSESSIONID).
+ Set-Cookie JSESSIONID = xxx
This table is used by the Load Balancer to

HTTP Request2 always send the follow-up requests with
HTTP Request2
+ Cookie JSESSIONID = xxx the same field to the same Pool Member.
2 S1
Same behavior as Source-IP and Cookie
Persistence
85
Server Pool
S
Layer7 HTTP VIP Configuration S
Persistence - Generic Persistence on JSESSIONID (2/4) VIP L7

HTTP:80
S

1
+ Set-Cookie JSESSIONID = xxx S1
Use the default Generic Persistence profile, or Create a new one + Set-Cookie JSESSIONID = xxx
HTTP Request2
• Under Networking – Load Balancing – Profiles + Cookie JSESSIONID = xxx HTTP Request2
S1
2 is shared
Same Source IP Persistence Table
– Add Persistence Profile – Generic among different L7 VIP How long an idle client entry remains in the

86
Server Pool
S

HTTP:80
S

1
Use LB rules to configure Generic Persistence + Set-Cookie JSESSIONID = xxx
HTTP Request2
• Under Networking – Load Balancing – Virtual Servers + Cookie JSESSIONID = xxx HTTP Request2
S1
2
– Add Response Rewrite Phase
Example for Generic Persistence based
on Cookie JSESSION
Create Generic Persistence when “Set-

Cookie JSESSIONID” is present “Crete the persistence entry based on the “Set-Cookie
JSESSIONID” value.
87
Server Pool
S

HTTP:80
S

1
Use LB rules to configure Generic Persistence + Set-Cookie JSESSIONID = xxx
HTTP Request2
• Under Networking – Load Balancing – Virtual Servers + Cookie JSESSIONID = xxx HTTP Request2
S1
2
– Add Request Forwarding Phase
on Cookie JSESSION
Enable Generic Persistence based on the Cookie

JSESSIONID (built-in variable “_cookie_JSESSIONID”)
Enforce Generic Persistence only if

Cookie JSESSIONID is present
“Hash Variable” has to be used if the Variable is very

long, as NSX-T Generic Persistence stores only its first
88
64 characters.
Server Pool
S
Persistence - Generic Persistence on XFF (other) VIP L7

HTTP:80
S
HTTP Request1
+ X-Forwarded-For = xxx HTTP Request1
1
Use LB rules to configure Generic Persistence HTTP Response1 HTTP Response1

S1
• Under Networking – Load Balancing – Virtual Servers HTTP Request2

HTTP Request2
+ X-Forwarded-For = xxx
– Add Request Forwarding Phase 2 S1

on X-Forwarded-For header
Enable Generic Persistence based on the Header X-

Enforce Generic Persistence only if Forwarded-For (variable “xff”)
Header X-Forwarded-For.
And create a variable “xff” with the
value of the X-Forwarded-For.
“Hash Variable” has to be used if the Variable is very

long, as NSX-T Generic Persistence stores only its first
64 characters. 89
Other configuration - X-Forwarded-For header (1/2)
Client request is load balanced to Pool

Member with extra HTTP Headers:
• X-Forwarded-For
Server Pool
S • Value= Client IP@
• X-Forwarded-Proto
S • Value= http | https (based VIP listening on
http or https)
• X-Forwarded-Port
VIP L7
S
• Value= VIP port
HTTP:80
1 2
Note:
HTTP Request HTTP Request Web servers may need information, such as client IP@, for
+ X-Forwarded-For: Client-IP@ logging or other purposes.
+ X-Forwarded-Proto: http In case of LB-NAT configuration on the Pool, Web servers
+ X-Forwarded-Port: 80 will see the LB-NAT IP address in the source IP. However
configuring the web servers to look at those HTTP headers,
they can still retrieve the real Client IP information.
90
Server
S Pool
Other configuration - X-Forwarded-For header (2/2) VIP L7 S

HTTP:80
Create new HTTP Profile HTTP Request HTTP Request

+ X-Forwarded-For: Client-IP@
+ X-Forwarded-Proto: http
and use it un the L7 HTTP Virtual Server + X-Forwarded-Port: 80
• Under Networking – Load Balancing – Profiles . Insert: If the XFF HTTP header is not present in the incoming request, then the LB inserts a new XFF
header with the client IP@. If XFF HTTP header is present in the incoming request, then the LB
– Add Application Profile – HTTP appends the SFF header with the client IP@.
. Replace: If the XFF HTTP header is already present in the incoming request then the LB replaces the
header
91
Other configuration - Redirect Clients to HTTPS (1/3)
Client access application on HTTP.

No
Load balancer asks the client to come
Pool
back on HTTPS (via HTTP Redirect).
2 Options:
VIP L7
• HTTP Redirect 301
HTTP:80
Redirect all clients to the same page.
GET http://www.xyz.com/page1.html 1
• HTTP to HTTPS Redirect 301
Redirect all HTTP Clients request to
the same site/page on HTTPS
Redirect https://maintenance.xyz.com/sorry.html 1 2
Redirect https://www.xyz.com/page1.html 2
92
Other configuration - Redirect Clients to HTTPS (2/3) VIP L7
HTTP:80
GET http://www.xyz.com/page1.html
Create new HTTP Profile and use it Redirect https://maintenance.xyz.com/sorry.html 1

in the L7 HTTP Virtual Server
– Add Application Profile – HTTP
Redirection:
In this mode specify the “HTTP
Redirect:
93
No
Layer7 HTTP VIP Configuration Pool
Other configuration - Redirect Clients to HTTPS (3/3) VIP L7

HTTP:80
Create new HTTP Profile and use it Redirect https://www.xyz.com/page1.html 2

Redirection:
In this mode specify the “HTTP to
HTTPS Redirect:
94
Other configuration – NTLM Authentication / Server Keep-Alive (1/2)
Server Pool
Client access application on HTTP to Web
S Servers configured with NTLM
authentication.
S NTLM Authentication authenticate the TCP
connection. And once authenticated, all
S HTTP requests is that TCP connection are
VIP L7 replied.
HTTP:80
1 TCP1: GET 1.html + NTLM Negotiate TCP1: GET 1.html + NTLM Negotiate
TCP1: Status Code: 401 + NTLM Challenge TCP1: Status Code: 401 + NTLM Challenge 2 When NTLM configuration / Server Keep-
TCP1: GET 1.html + NTLM Authenticate TCP1: GET 1.html + NTLM Authenticate
Alive is enabled, as soon a Client request
3 contains NTLM authentication (NTLM
TCP1: Status Code: 200 TCP1: Status Code: 200 4 Negotiate) this Client TCP connection is stick
to one dedicated Server TCP connection.
5 TCP1: GET 2.html TCP1: GET 2.html
Note: “NTLM + Sce-IP Persistence”:
TCP1: Status Code: 200 TCP1: Status Code: 200 6 If “Client/Server traffic NOT always uses NTLM
Authentication”, then NSX configuration requires
“Pool-SNAT”.
95
Server Pool
S
S
Other configuration – VIP L7
HTTP:80
NTLM Authentication / Server Keep-Alive (2/2) 1 TCP1: GET 1.html + NTLM Negotiate TCP1: GET 1.html + NTLM Negotiate
TCP1: Status Code: 401 + NTLM Challenge TCP1: Status Code: 401 + NTLM Challenge 2
Create new HTTP Profile and use it 3 TCP1: GET 1.html + NTLM Authenticate
TCP1: Status Code: 200

TCP1: GET 1.html + NTLM Authenticate
TCP1: Status Code: 200 4

5 TCP1: GET 2.html TCP1: GET 2.html
• Under Networking – Load Balancing – Profiles TCP1: Status Code: 200 TCP1: Status Code: 200 6
Enable Server Keep-Alive

(previously called “NTLM Authentication”) New NSX-T 3.1
96
If Server Pool is dead, then Load Balancer

will load balance clients to a (Sorry) Server
Pool.
Connections to VIP:80
HTTP Requests Server Pool1 Note: In case of L7-VIP with Persistence (Source-IP
S or Cookie), persistence is not offered on Sorry
Server Pool.
S When Main Server Pool is Down, Clients will be
load balanced without persistence to Sorry Server
VIP L7 S Pool.
HTTP:80 Then when Main Server Pool becomes UP again,
immediately Clients are load balanced to Main
Sorry Server Server Pool with Persistence.
S Pool Pool2
97

Sorry Server Pool
98
Other configuration - Connection Throttling/Rating (1/2)
Load Balancer can protect against

Connections to VIP:80 excessive load.
HTTP Requests
Load Balancer will stop load balancing
Server Pool
new connections if VIP reached its:
S
• "Max Concurrent Connections"
S • or "Max New Connection Rate".
VIP L7 S Note: VIP L7 Persistence (Source IP or

HTTP:80 Cookie) does not override this option.
If a former client that is load balanced to a
Pool Member sends a new connection when
the VIP L4 reached its limit, this former
client new connection will be discarded.
99
Connection Throttling/Rating (2/2)
Maximum Concurrent Connections

rate on the VIP
Maximum Concurrent
Connections on the VIP
100
Other configuration - Increase Response Header Size (1/2)
Server Pool By default Layer7 HTTP Profiles

S (HTTP/HTTPS) allow only pool member
response with HTTP headers < 4k bytes.
S 1
VIP L7 S There is an option of the LB Profile to allow

HTTP:80 HTTP response headers up to 64k bytes.
HTTP Request HTTP Request
2
HTTP Response
with http headers > 4k
1
HTTP Response HTTP Response With LB Profile

with http headers > 4k with http headers > 4k configuration to allow
2
response headers > 4k
101
Increase Response Header Size (2/2)
Update LB Profile "response_header_size" (API only)
Response Header Size (default

= 4096
Max = 65536)
102
Other configuration - Response Buffering (1/2)
Server Pool From NSX-T 3.0:

S Ability to choose between
1 • No Response buffering
S • LB VIP forwards pool member
response immediately
VIP L7 S 2 • Response buffering
HTTP:80 • LB VIP forwards pool member
HTTP Request HTTP Request response once the whole object is
received
“Response Buffering = Disabled” 1

Response starts Large HTTP Response
in multiple packets
Default Response Buffering:
when object fully
received by LB • NSX-T 3.0 fresh install or upgrade
• No Response Buffering
Large HTTP Response • Prior NSX-T 3.0
“Response Buffering = Enabled” 2 • Response Buffering (no option to
in multiple packets
Response starts change this setting)
when object fully
received by LB
103
Server
S Pool
Other configuration – VIP L7

HTTP:80
S
HTTP Request
Response Buffering (2/2) HTTP Request
“Response Buffering = Disabled” 1

Update LB Profile ”Response Buffering Response starts Large HTTP Response
in multiple packets
when object fully
received by LB
“Response Buffering = Enabled” Large HTTP Response

2
in multiple packets
Response starts
when object fully
received by LB
Response Buffering
104
Load Balancer generates an Access Log L7

Connection to VIP:80 Server Pool HTTP for each HTTP request load balanced.
HTTP Request S
This Audit Log can be sent also to an
S
external syslog server if the Edge Node is
VIP L7 configured with a syslog server.
S
HTTP:80
To limit the number of logs, only requests

Access Log with HTTP Response Errors can be logged
lab2edge1> get load-balancer a339fa40-8f30-4ff7-b66a-2090b5542ee3 virtual-server (HTTP Response >=400)
e71b88ea-aa3b-4b77-9af8-7aa27daa81e1 access-log follow
Operation.Category: 'LbAccessLog', Operation.Type: 'Http', Lb.UUID: '877bf9f8-
0c83-4f59-a689-e712f091c949', Lb.Name: 'LB1', Vs.UUID: '15538df4-7095-4a32-8ec5-
f2479fe73f9d', Vs.Name: 'New VIP HTTP', Vs.Ip: '30.30.30.6', Vs.Port: '80',
Pool.UUID: '28364f91-f6c0-42f8-b893-07f0fa322580', Pool.Name: 'Pool1',
PoolMember.Ip: '10.1.1.12', PoolMember.Port: '80', Client.Ip: '10.114.218.184',
Client.Port: '58910', Snat.Ip: '30.30.30.6', Snat.Port: '4097',
HttpRequest.Method: 'GET', HttpRequest.UserAgent: 'curl/7.47.0', HttpRequest.X-
Fwd-For: '-', HttpRequest.Uri: '/test.php', HttpRequest.Host: '30.30.30.6',
HttpResponse.Status: '200', HttpResponse.StatusCategory: '2xx', HttpResponse.Size:
'62', HttpResponse.ServerTime: '0.031', HttpResponse.TotalTime: '0.031',
Error.Reason: '-' 105

Access Log
Only error logs

106
(HTTP Response >=400)
Explanation
Server Pool
S NSX-T Manager creates automatically
Groups for:
S • Server Pool
VIP L7 • VIP
S
HTTP:80
This simplifies the creation of security:

NSX-T Groups created: • T1 Gateway Firewall
• Server Pool Group (NLB.PoolLB.[Pool_Name][LB_Name]) Allow traffic from Clients to "LB VIP" (=
Group Member IP Addresses: NLB.VIP.[VIP_Name])
• If Pool configured with no LB-SNAT (Transparent): 0.0.0.0/0
• DFW rules
• If Pool configured with LB-SNAT Automap: T1-Uplink IP 100.64.x.y
On Pool Members, allow traffic from "LB" (=
+ T1-ServiceInterface IP
NLB.PoolLB.[Pool_Name]
• If Pool configured with LB-SNAT IP-Pool: LB-SNAT IP-Pool
[LB_Name])
• VIP Group (NLB.VIP.[VIP_Name])
Group Member IP Addresses: Note: [Pool/LB/VIP names] with more than 12
• VIP IP@ characters will be truncated in UI (Group Display
Name). The Group ID is NOT truncated.
107
Visualization
Look at the NSX Groups auto-created by LB

• Under Inventory – Groups - Groups
[Pool-Bugzilla] is truncated with [Pool…zilla] in the Group Display Name.

However the Group ID of that object is still NLB.PoolLB.[Pool-Bugzilla][LB1]
108
Layer7 HTTP VIP
Monitoring/Statistics (1/3)
Monitoring
Monitor of Layer7 HTTP VIP

Deeper Status information
• Under Advanced Networking & Security – Load Balancing – Server Pools available on Advanced UI.
– Select Pool

Layer7 HTTP VIP
Statistics
Statistics of Layer7 HTTP VIP

• Rate statistics are last second rate statistics

• Packets In/Out statistics are not supported on L7-
VIP

Layer7 HTTP VIP
Real Time Statistics of Layer7 HTTP VIP

on Advanced UI.

Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
112
Agenda 6. Layer7 HTTPS VIP
• Layer7 HTTPS Modes
• HTTPS Off-Load
• HTTPS End-to-End
• SSL Passthrough
• Layer7 HTTPS VIP Configuration

• HTTPS Off-Load Configuration • SSL Passthrough Configuration
• HTTPS End-to-End Configuration
• Other Configuration (for HTTPS Off-Load and HTTPS End-to-
End) • Layer7 HTTPS VIP Monitoring/Statistics
• HTTPS Selection of Protocol and Ciphers
• HTTPS Client Authentication
• Auto Certificate Selection based on SNI
Note: The Use Cases described in section "Layer7 HTTP VIP” also applies to Layer7 HTTPS VIP.
Refer to that section to do such configuration. 113
Layer7 HTTPS Modes
3 modes (1/2)
HTTPS Off-Load LB decrypts

and forwards in clear
Layer7 HTTPS VIP offers 3 modes:
Server Pool
S • HTTPS Off-Load
HTTPS HTTP Best balance between security, performance,
S and LB flexibility.
S • Security:
VIP L7 Traffic is fully encrypted from the Client up to
HTTPS:443 the LB.
• Performance:
Traffic is decrypted / encrypted only once.
HTTPS End-to-End SSL LB decrypts
and re-encrypts before
forwarding Server Pool
S • HTTPS End-to-End SSL
Best security, and LB flexibility.
HTTPS HTTPS
S
• Security:
Traffic end to end encrypted.
S • Performance:
VIP L7
This mode has lower performance with traffic
HTTPS:443
decrypted/encrypted twice.
114
Layer7 HTTPS Modes
3 modes (2/2)
Layer7 HTTPS VIP offers 3 modes:
SSL Passthrough LB does not decrypt • SSL Passthrough

and SSL connection is terminated on Best security and performance, but limited
Pool Members Server Pool
LB flexibility.
S
HTTPS • Security:
S End-to-end encryption.
• Performance:
S Highest performance because LB does not
VIP L7 terminate SSL traffic.
HTTPS:443
115
Layer7 HTTPS VIP Configuration
HTTPS Off-Load Configuration (1/5)
LB decrypts Server Pool

and forwards in clear
Layer7 HTTPS Load Balancing is done after
S Client TCP handshake + SSL Handshake +
HTTPS HTTP HTTP request.
S
In HTTPS Off-Load mode, LB forwards
S
VIP L7 Clients' requests in HTTP (clear text).
HTTPS:443
TCP handshake
+ SSL handshake
Note: This diagram does not represent
1
Transparent Mode or LB-SNAT modes. They are
HTTP Request TCP handshake both supported.
2
HTTP Request
HTTP Response HTTP Response

4 3
116
Import Server Certificate or Create a Self-Signed Certificate

• Under System – Certificates
– Import Certificate
Screenshot shows RSA certificate.

Server Certificate
Wildcard and ECC certificates are (See Notes of example)
also supported.
Server Key
(See Notes of example)
Must be enabled to use that

certificate in LB L7 HTTPS VIP
117

How long Clients idle connections How long LB waits for Server HTTP Response
– Add Application Profile – HTTP remain before LB closes them before LB gives up and sends an Error to Client
(FIN). and Server (FIN).
118

Load Balancer
SSL Configuration
(See next slide) L7 HTTP Profile
Optional:

– Configure SSL
Enable HTTPS on the Client side

Disable HTTPS on the Server
Select Certificate LB will present to
side
Clients (see Notes)
Select SSL Ciphers and Protocols LB

accepts on Client side
(Optional) In case the same VIP is used

for multiple web sites (app1.xyz.com and
app2.xyz.com), select here all certificates
in addition to the “Default Certificate”.
120
HTTPS End-to-End Configuration (1/5)
LB decrypts
and re-encrypts before Server Pool Layer7 HTTPS Load Balancing is done after
forwarding S Client TCP handshake + SSL Handshake +
HTTPS HTTPS HTTP request.
S
In HTTPS End-to-End mode, LB forwards
S
VIP L7 Clients' requests in new HTTPS connections
HTTPS:443 (encrypted).
TCP handshake
+ SSL handshake
1
TCP handshake
HTTP Request + SSL handshake Note: This diagram does not represent
2 Transparent Mode or LB-SNAT modes. They are
HTTP Request both supported.
HTTP Response HTTP Response

4 3
121
Import Server Certificate or Create a Self-Signed Certificate

Server Certificate
Screenshot shows RSA certificate. (See Notes of example)
ECC certificates are also supported.
Server Key

122

How long Clients idle connections How long LB waits for Server HTTP Response
– Add Application Profile – HTTP remain before LB closes them before LB gives up and sends an Error to Client
(FIN). and Server (FIN).
123

Load Balancer
SSL Configuration
(See next slide) L7 HTTP Profile
Optional:

– Configure SSL
Enable HTTPS on the Client side Enable HTTPS on the Server side
Select Certificate LB will present to

Clients (see Notes) Optional:
Only if Server require SSL Client
certificate (not popular option)
Select SSL Ciphers and Protocols LB
accepts on Client side
Select SSL Ciphers and Protocols
LB present to Servers
Optional Advanced Properties:

Only if LB must validate
Certificate presented by Servers
(not popular option)
125
Other Configuration (for HTTPS Off-Load and HTTPS End-to-End) -
- HTTPS Selection of Protocol and Ciphers (1/2)
Server Pool Layer7 HTTPS Load Balancing offers

S control on SSL Ciphers and Protocols
allowed.
HTTPS HTTP or HTTPS S
This control can be applied on the
S
VIP L7 Client Side (as described on the
HTTPS:443 diagram) or the Pool side.
Client SSL Hello

1
(with SSL Ciphers + Protocol supported) NSX comes with built-in list:
• Balanced (recommended)
LB selects one of the Client proposed SSL Ciphers + Protocol Best balance between Performance /
2 which is part of its supported Security / Variety of Client support
• High Compatibility
Best variety of Client support
• High Security
Highest Secured SSL Ciphers +
Protocols
And can use custom list.
126
HTTPS Selection of Protocol and Ciphers (2/2)
Create SSL Profile and use it in the L7 HTTP Virtual Server
– Add SSL Profile – Client SSL Profile
Select built-in list of SSL Ciphers Performance optimization when Clients open
and Protocol, or create a Custom multiple TCP connections with SSL re-use
one option
List of SSL Ciphers allowed
Prefer LB SSL ciphers and

List of SSL Protocols allowed Protocols (or Client)
If SSL Session Cache enabled,

how long does LB cache each
SSL Session key
127
HTTPS Client Authentication (1/3)
Server Pool Layer7 HTTPS Load Balancing offers the

S option to require a Client Certificate.
HTTPS HTTP or HTTPS 2
S Once Client Certificate is validated, then LB
load balances the request to the Pool
S
VIP L7 Members.
Client HTTPS:443 3
Certificate
Client SSL Hello
1 (with SSL Ciphers +
Protocol supported)
LB selects one of the Client proposed SSL

Ciphers + Protocol AND request for the Client
Certificate
2
Client sends its Certificate
to LB
3 Request to Server
128
Import Domain CA Certificate
– Import CA Certificate
Server Certificate

129
– And Configure SSL
Select the Certificate Authority (CA) who

signed the Client Certificate
Mandate Client Certificate
130
Auto Certificate Selection based on SNI (1/3)
Server Pool Layer7 HTTPS Load Balancing offers

S the option to load multiple Certificates
under the same Virtual Server.
HTTPS HTTP or HTTPS S
Then based on the Client SNI request,
S
VIP L7 LB presents the right appropriate
HTTPS:443 certificate. 1
2
https://blog.xyz.com
Certificate blog.xyz.com Site Certificate "blog"
https://www.xyz.com
Certificate www.xyz.com Site Certificate "www"
131
Import Server Certificate
Server Certificate
Server Key

132
– Configure SSL
Add the list of other Certificates LB can

present to Clients
133
SSL Passthrough Configuration (1/5)
LB does not decrypt

and SSL connection is terminated on Layer7 HTTPS Load Balancing is done at the
Pool Members Server Pool
SSL Client Hello.
S
HTTPS In SSL Passthrough mode, LB checks the SNI
S
information in the SSL Client Hello.
S If matching the SSL Passthrough rule, LB
VIP L7 does not terminate Client SSH handshake
TCP handshake HTTPS:443
but forwards it in a new TCP connection to
a Pool Member.
1 SSL Client Hello
SNI: app1.xyz.com TCP handshake
Since LB does not terminate SSL, LB can not
2
SSL Client Hello look at HTTP clear traffic and modify it (like
SNI: app1.xyz.com insert XFF header).
SSL Server Hello
3
Note: This diagram does not represent
End of SSL handshake + HTTPS Requests Transparent Mode or LB-SNAT modes. They are
4
both supported.
134
Server
S Pool
Layer7 HTTPS VIP Configuration HTTPS

S
SSL Passthrough Configuration (2/5) VIP L7

S
HTTPS:443

How long Clients idle connections
– Add Application Profile – HTTP remain before LB closes them Other fields are irrelevant for the VIP doing SSL
(FIN). Passthrough
135
Server
S Pool

S

S
HTTPS:443

Load Balancer
SSL Configuration
(See next slide +
Note for more information) L7 HTTP Profile
LB Rule SSL Passthrough
(See next slide)
Optional:
SSL Passthrough Configuration (4/5)
Create Virtual Server That configuration is required because of the

“config sanity check”, but is not used in cased of
VIP using SSL Passthrough for all SNI
– Configure SSL
137
Server
S Pool

S

S
HTTPS:443

– Create an LB Rule “Transport”
Action SSL Passthrough
Match all Clients SNI (option) Select a Specific Pool.

If not, it’s using the Pool configured under the VIP
138
Layer7 HTTPS VIP
Monitoring
Monitor of Layer7 HTTP VIP

• Under Advanced Networking & Security – Load Balancing – Virtual Servers Deeper Status information
available on Advanced UI.
– Select Virtual Server

Layer7 HTTPS VIP
Statistics
Statistics of Layer7 HTTPS VIP

• Rate statistics are last second rate statistics

• Packets In/Out statistics are not supported on L7-
VIP

Layer7 HTTPS VIP
Real Time Statistics of Layer7 HTTPS VIP

on Advanced UI.

Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
142
Agenda 7. LB Rules (HTTP or HTTPS)
• LB Rules Packet Flow
• LB Rules examples
• Block Specific URLs
• Redirect Specific URL/Host to Specific Pools (based on url, query, host)
• Redirect Specific Host requests to HTTPS
• Redirect Non-Authenticated Clients
• Replace Response Header ”Server”
• Rewrite Server Redirect
• Inject header with character ' (single quote)
• Advanced Rewrite of Host + URL
• Inject Client Certificate in HTTP header (for “HTTPS VIP + Client Authentication”)
• Access control based on claims in JWT (+ selection of specific Pool)
143
LB Rules Packet Flow
Modifying or acting upon HTTP request or response
Modify or Act upon HTTP phases

LB
HTTP or HTTPS HTTP or HTTPS
Clients 1 Transport Server 1. Transport Phase
Request 2 HTTP Access Request S Pool • SSL mode + Pool selection based on
3 Request Rewrite Client HTTPS request
S
4 Request Forward
Response Response 2. HTTP Access
5 Response Rewrite
S • JSON Web Token validation
3. Request Rewrite Phase

Rule Match Conditions Match Strategy Actions • Request header, path rewriting
1 If host header is www.xyz.com All Rewrite header to app1.xyz.com
If uri is "/index.html" Rewrite uri to "/default.php" 4. Request Forwarding Phase
2 If host header is "blog.xyz.com" Or Select Pool "Pool2"
• Pool selection
If host header is "new.xyz.com" • HTTP Redirect
• Reject / drop request
3 If Response header All Rewrite Response header
"Server = Microsoft-IIS/7.5" "Server = Apache/2.4.18 (Ubuntu)
5. Response Rewrite Phase
• Response header rewriting / deletion
144
LB Rules examples
Block Specific URLs (1/2)
Server Use Case:

S Pool
Block specific requests starting with
"/private" or "/finance"
S
VIP L7 S
HTTP or HTTPS
http://www.xyz.com/public/page1.htlm http://www.xyz.com/public/page1.htlm
http://www.xyz.com/finance/page1.htlm
http://www.xyz.com/private/page1.htlm
145
LB Rules examples
Block Specific URLs (2/2)
Add LB Rule configuration to Virtual Server

146
LB Rules examples
Redirect Specific URL/Host to Specific Pools (1/4)
Server Pool Use Case:

S Default
Redirect requests starting with
"/private" or "/finance" to specific
S
pools
VIP L7 S
HTTP or HTTPS
Server Pool
S Finance and
http://www.xyz.com/public/page1.htlm Private
S
http://www.xyz.com/finance/page1.htlm
http://www.xyz.com/private/page1.htlm
147
LB Rules examples

148
LB Rules examples

S Default
Redirect requests for specific Host to
specific pools
S
VIP L7 S
HTTP or HTTPS
Server Pool
S Finance and
http://www.xyz.com/page1.htlm Private
S
http:/finance.xyz.com/page1.htlm
http://private.xyz.com/page1.htlm
149
LB Rules examples

150
LB Rules examples
Redirect Specific Host requests to HTTPS (1/3)
Server Use Case:

S Pool
Redirect specific Host requests
(finance.xyz.com) to HTTPS
S
VIP L7 S
HTTP or HTTPS
GET http://www.xyz.com/page1.html GET http://www.xyz.com/page1.html
GET http://finance.xyz.com/page1.html
Redirect https://finance.xyz.com/page1.html
151
LB Rules examples
Create L7-HTTP VIP

• See "Layer7 HTTP VIP section"
152
LB Rules examples

finance.xyz.com
https://$_host$_request_uri
153
LB Rules examples
Redirect Non-Authenticated Clients (1/2)
Server Use Case:

S Pool
Redirect the client requests to
"/authent.php" if they don't have a
S
cookie
VIP L7 S
HTTP or HTTPS
No Cookie Authent
Redirect http://www.xyz.com/authent.php
GET http://www.xyz.com/authent.php GET http://www.xyz.com/page1.html

No Cookie Authent No Cookie Authent
154
LB Rules examples
Redirect Non-Authenticated Clients (2/2)

https://$_host/authent.php
.*
155
LB Rules examples
Replace Response Header ”Server” (1/2)
Server Use Case:

S Pool
Hide real web server type information
S
VIP L7 S
HTTP or HTTPS
200 OK 200 OK
Server: Apache/2.4.18 (Ubuntu) Server: Microsoft-IIS/7.5
<body Response> <body Response>
156
LB Rules examples
Replace Response Header ”Server” (2/2)

.*
Apache/2.4.18(Ubuntu)
157
LB Rules examples
Rewrite Server Redirect (1/2)
Server Use Case:

S Pool
Rewrite Server Response Header
Location with VIP TCP Port
S
VIP L7 S
HTTP or HTTPS Note:
This use case also requires another LB Rule to
rewrite the request:
GET http://www.xyz.com:8080/page1.html GET http://www.xyz.com/page1.html
GET http://www.xyz.com:8080/page1.html
to
301 Redirect 301 Redirect
This Request Rewrite LB Rule is not documented on
Location: http://www.xyz.com:8080/login.php Location: http://www.xyz.com/login.php
next slide.
158
LB Rules examples
Rewrite Server Redirect (2/2)

http://(?<hostname>.[^/]*)/(?<redirect>.*) http://$hostname:8080/$redirect
159
LB Rules examples
Inject header with character ' (single quote) (1/2)
Server Use Case:

S Pool
Redirect specific Host requests
(finance.xyz.com) to HTTPS
S
VIP L7 S
HTTP or HTTPS
200 OK 200 OK
Server: Microsoft-IIS/7.5 Server: Microsoft-IIS/7.5
Content-Security-Policy: blob: 'unsafe-inline' <body Response>
<body Response>
160
Server
S Pool
LB Rules examples S
Inject header with character ' (single quote) (2/2) VIP L7 S
HTTP or HTTPS

• Under Networking – Load Balancing – Virtual Servers 200 OK
Server: Microsoft-IIS/7.5
200 OK
Server: Microsoft-IIS/7.5
Content-Security-Policy: blob: 'unsafe-inline' <body Response>
<body Response>
blob: \39unsafe-inline\39
Note: ' (single quote)

character is not allowed to be
used in a LB rule.
The workaround is to use the
\unicode instead (\39 for ')
161
LB Rules examples
Advanced Rewrite of Host + URL (1/2)
Server Use Case:

S Pool
Rewrite the URL based on:
• Original URL
S
• Client IP@
VIP L7 S
HTTP or HTTPS
http://www.xyz.com/news/news1.html http://www.xyz.com/news.py
?story=news1.html&client=10.16.116.47
162
LB Rules examples
Advanced Rewrite of Host + URL (2/2)

/news.py
/news/(?<article>.*\..*)
story=$article&client=$_remote_addr
163
LB Rules examples
Inject Client Certificate in HTTP header (for “HTTPS VIP + Client Authentication”) (1/2)

S LB-VIP is configured to ask Client its Client
Certificate
HTTPS HTTP or HTTPS S (see “Layer7 HTTPS VIP Configuration - Other Configuration
- HTTPS Client Authentication”).
S
VIP L7
Client HTTPS:443 Server Pool needs that Client Certificate
Certificate information too.
1 Client SSL Hello
LB-VIP LB Rule will inject that received Client

LB request for the Client Certificate
Certificate in a HTTP header.
2
Client sends its Certificate
to LB
3 Request to Server
with client certificate in
HTTP header
164
Server
S Pool
LB Rules examples HTTPS HTTP or HTTPS S
Inject Client Certificate in HTTP header VIP L7

S
HTTPS:443
(for “HTTPS VIP + Client Authentication”) (2/2) Client
Certificate
1 Client SSL Hello
Add LB Rule configuration to Virtual Server 2

LB request for the Client Certificate
Client sends its
• Under Networking – Load Balancing – Virtual Servers Certificate to LB
3 Request to Server
with client certificate
in HTTP header
Client-cert
/
(to catch all requests)
$_ssl_client_escaped_cert
165
LB Rules examples
Access control based on claims in JWT (+ selection of specific Pool) (1/4)

S Default
JSON Web Token validation (+ selection of
specific Pool)
HTTP or HTTPS HTTP or HTTPS S
1 Client sends a request with JWT
JWT Token VIP L7 Server Pool
HTTPS:443 S for User Dimi 2 LB-VIP validates it can decode the JWT
Client Request “Starts with “/jwt” with
1
JWT Token S LB-VIP processes the request
3
If LB-VIP can’t decode token, (optionally) LM-VIP selects a specific Pool
2
LB-VIP reject Request (401 Response)
4 based on JWT
If LB-VIP can decode token,
3 LB-VIP processes the request
(and will go through other LB Rules if any)
(optionally) LB-VIP Forward

4 request to Specific Pool
166
Server
S Pool
LB Rules examples HTTP or HTTPS HTTP or HTTPS S

Default
Access control based on claims in JWT JWT Token

VIP L7 Server
HTTPS:443 S Pool for
(+ selection of specific Pool) (2/4) 1 Client Request “Starts with “/ jwt” User Dimi
with JWT Token S
Create the JWT 2 If LB-VIP can’t decode token,

LB-VIP reject Request (401Response)

• On portal https://jwt.io 3 LB-VIP processes the request
(optionally) LB-VIP
4 Forward request to
Specific Pool
(optional) Added JWT header ”app”:

“App1”
”Name” in the payload = “Dimi”
(optional)
Enter ”Secret””
Do not select “encoding of the secret”

167
Server
S Pool

Default

VIP L7 Server
with JWT Token S
Access Control Based (LB Rule) 2 If LB-VIP can’t decode token,


• Under Networking – Load Balancing – Virtual Servers 3 LB-VIP processes the request
(optionally) LB-VIP
Specific Pool
(optional)
Specify the specific URL
path you want with JWT
(optional)
access control: “/jwt” (optional)
For use case of JWT is not in
Specify a Realm for Clients invalid
Authorization header.
requests (this information will be
See Notes for more info.
sent in “WWW-Authenticate”
Response Header”)
(optional)
Enter JWT secret
(optional)
See Notes for more information + examples
Forward the JWT to the Pool Member 168
Server
S Pool

Default

VIP L7 Server
with JWT Token S
(Optional) Selection of specific Pool (LB Rule) 2 If LB-VIP can’t decode token,

• Under Networking – Load Balancing – Virtual Servers 3 LB-VIP processes the request
(optionally) LB-VIP
Specific Pool
(optional)
Select specific JWT
header field
(_jwt_header_<header-
name>) Select the LB Pool
(optional)
Select specific JWT
header field
(_jwt_claim_<payload-
name>) (optional)
This LB rule is optional.
That’s only if you want to send specific JWT Users to special LB Pool 169
Agenda
2 Monitors
3 Server Pools
4 Layer4 VIP
5 Layer7 HTTP VIP
6 Layer7 HTTPS VIP
8 Troubleshooting
170
Agenda 7. Troubleshooting
• Why Pool Member down
• Why Clients receive a “502 Bad Gateway” response
• Show LB Session Table
• Advanced Statistics
• Packet Capture
• LB Syslog Messages
• Edge Node and LB Monitoring (Capacity & Performance)
• LB Diagnosis
• Error Messages
• Miscellaneous
• Cascade of VIP
171
Troubleshooting
Why Pool Member down
• Deep information on Pool Member status is available

• CLI
lab1-edge1> get load-balancer d81cfcf8-063f-451c-a61a-774286c3d3d9 pool 66a321cf-62f2-4e2d-a0f5-c06716b68c47 status
Pool
UUID : 66a321cf-62f2-4e2d-a0f5-c06716b68c47
Display-Name : Pool1-HTTPS
Status : partially_up
Total-Members : 2
Primary Up : 1
Primary Down : 1
Primary Disabled : 0
Primary Graceful Disabled : 0
Backup Up : 0
Backup Down : 0
Backup Graceful Disabled : 0
Backup Disabled : 0
Member
Display-Name : S2
Type : primary
IP : 10.1.1.12
Port : 443 Member
Status : up Display-Name : S1
Last-Check-Time : 2019-04-08 20:03:58 Type : primary Reason of the Pool Status Down
Last-State-Change-Time : 2019-04-08 18:52:00 IP : 10.1.1.11
Port : 443
Status : down
Last-Check-Time : 2019-04-08 20:04:00
Last-State-Change-Time : 2019-04-08 18:55:23
Failure-Reason : Connect to Peer Failure 172
Troubleshooting
Why Clients receive a “502 Bad Gateway” response
VIP L7
• When all the Pool Members are down, NSX-T L7-VIP can not forward the clients request to a pool member
• In that case NSX-T L7-VIP does reply a “502 Bad Gateway” response to clients
173
Troubleshooting
Show LB Session Table (1/2)
• Layer 4 UDP VIP

• Client IP@: 10.114.218.199 / VIP: 21.21.21.6:22 / Pool Member: 10.1.1.12:22
lab1-edge1> get load-balancer d81cfcf8-063f-451c-a61a-774286c3d3d9 session-tables
Session-Tables
TABLE ID PROTO CADDR CPORT VADDR VPORT SADDR SPORT DADDR DPORT
l4lb-0 0000000000000006 udp 10.114.218.199 59843 21.21.21.6 53 10.114.218.199 59843 10.1.1.11 53
The Pool is with SNAT Disabled
• Layer 4 TCP/UDP VIP

Session-Tables
l4lb-0 0000000000000000 tcp 10.114.218.199 58288 21.21.21.6 22 21.21.21.6 4096 10.1.1.12 22
The Pool is with SNAT-IPLIST=VIP
174
Troubleshooting
Show LB Session Table (2/2)
• Layer 7 HTTP VIP

Session-Tables
l7lb-0 0000000000000004 http 10.114.218.199 53230 21.21.21.6 80 100.64.144.1 4097 10.1.1.12 80
l7lb-0 000000000000000d http 10.114.218.199 53262 21.21.21.6 80 - 0 - 0
The Pool is with SNAT-Automap
A session with no SADDR/DADDR (no Pool Member) is possible.

Most likely it's with the use case:
• The client established the TCP connection to VIP + sent the request
• LB established the TCP connection to the Pool Member + sent the request
• The Pool Member replied the object to the LB and closed the connection
• And
• LB is still replying the object to the client
• Or LB replied the object to the client, but client is HTTP1.1 with keep-alive. So
Client does not close immediately the TCP connection.
175
Troubleshooting
Advanced statistics
• See nginx config

nsx-edgebm3> set debug
nsx-edgebm3> get load-balancer <LB-UUID> engine-config
• See advanced stats

nsx-edgebm3> set debug
nsx-edgebm3> get load-balancer <LB-UUID> stats verbose
• Clear stats
nsx-edgebm3> clear load-balancer <lb-uuid> virtual-server <vs-uuid> stats
nsx-edgebm3> clear load-balancer <lb-uuid> virtual-servers stats
nsx-edgebm3> clear load-balancer <lb-uuid> pool <pool-uuid> stats
176
Troubleshooting
Packet Capture (1/4)
• Validate you're on the Edge Node Active for the LB

lab1-edge1> get load-balancer d81cfcf8-063f-451c-a61a-774286c3d3d9 status
Load Balancer
UUID : d81cfcf8-063f-451c-a61a-774286c3d3d9
Display-Name : LB1
Enabled : True
LB-State : ready
LR-HA-State : active
Virtual Servers : 2
Up Virtual Servers: 2
Pools : 2
Up Pools : 2
177
Troubleshooting
Packet Capture (2/4) Go to the section "Logical Router" where you have
the name of your router prepended with "SR-"
• Find Interface of the Tier-1 hosting LB

lab1-edge1> get logical-router bebe68ff-5b3e-46a2-8ef6-20fda85c1a5e interfaces
<snip>
Logical Router
UUID VRF LR-ID Name Type
bebe68ff-5b3e-46a2-8ef6-20fda85c1a5e 8 1092 SR-T1-LR1 SERVICE_ROUTER_TIER1
<snip>
Interface : 23a296e3-8488-44bf-ae32-e5c628c0c7a7
Ifuid : 297 That's the T1 downlink interface (facing to T1-DR)
Name : bp-sr0-port
Mode : lif
IP/Mask : 169.254.0.2/28;fe80::50:56ff:fe56:5300/64
<snip>
Interface : d4f8222a-0669-448d-915c-f09728b2f50a
Ifuid : 320 FYI you can see your LB-VIP on the loopback
Mode : loopback
IP/Mask : 21.21.21.6/32;127.0.0.1/8;::1/128
<snip>
Interface : f6ab8bad-a4b2-4bef-9502-994124b50e18
Ifuid : 307 That's the T1 "uplink" interface (facing the T0)
Name : T0-T1-LR1-t1_lrp
Mode : lif
IP/Mask : 100.64.144.1/31;fe80::50:56ff:fe56:4455/64;fca0:479a:b5e5:9000::2/64
<snip>
178
Client IP@: 10.114.218.199
Troubleshooting VIP: 21.21.21.6:80
Packet Capture (3/4) Possibillity to filter the packet capture using

tcpdump syntax
• Start packet capture on Edge Node T1 "uplink" interface (facing the T0)
lab1-edge1> start capture interface f6ab8bad-a4b2-4bef-9502-994124b50e18 [expression port 80]
16:16:26.392694 02:50:56:56:44:52 > 02:50:56:56:44:55, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0,
ethertype IPv4, 10.114.218.199.53192 > 21.21.21.6.80: Flags [S], seq 3483355859, win 29200, options [mss
1460,sackOK,TS val 494725924 ecr 0,nop,wscale 7], length 0
<base64>AlBWVkRVAlBWVkRSgQAAAAgARQAAPAWGQAA+BifiCnLaxxUVFQbPyABQz5/K0wAAAACgAnIQU24AAAIEBbQEAggKHXzrJAAAAAABAwM
H</base64>
16:16:26.393147 02:50:56:56:44:55 > 02:50:56:56:44:52, ethertype IPv4 (0x0800), length 74: 21.21.21.6.80 >
10.114.218.199.53192: Flags [S.], seq 4261070536, ack 3483355860, win 28960, options [mss 1460,sackOK,TS val
2391628344 ecr 494725924,nop,wscale 8], length 0
<base64>AlBWVkRSAlBWVkRVCABFAAA8AABAAEAGK2gVFRUGCnLaxwBQz8j9+sbIz5/K1KAScSCmwgAAAgQFtAQCCAqOjVo4HXzrJAEDAwg=</b
ase64>
ethertype IPv4, 10.114.218.199.53192 > 21.21.21.6.80: Flags [.], ack 1, win 229, options [nop,nop,TS val
494725926 ecr 2391628344], length 0
<base64>AlBWVkRVAlBWVkRSgQAAAAgARQAANAWHQAA+BifpCnLaxxUVFQbPyABQz5/K1P36xsmAEADlRckAAAEBCAodfOsmjo1aOA==</base6
4>
ethertype IPv4, 10.114.218.199.53192 > 21.21.21.6.80: Flags [P.], seq 1:83, ack 1, win 229, options [nop,nop,TS
val 494725926 ecr 2391628344], length 82: HTTP: GET /test.php HTTP/1.1
<base64>AlBWVkRVAlBWVkRSgQAAAAgARQAAhgWIQAA+BieWCnLaxxUVFQbPyABQz5/K1P36xsmAGADlf/AAAAEBCAodfOsmjo1aOEdFVCAvdGV
zdC5waHAgSFRUUC8xLjENCkhvc3Q6IDIxLjIxLjIxLjYNClVzZXItQWdlbnQ6IGN1cmwvNy41OC4wDQpBY2NlcHQ6ICovKg0KDQo=</base64>
179
Pool-Member- IP@: 10.1.1.12:80
Troubleshooting LB-SNAT-Automap: 100.64.144.1
Packet Capture (4/4) Possibility to filter the packet capture using

tcpdump syntax
• Start packet capture on Edge Node T1 downlink interface (facing to T1-DR)

lab1-edge1> start capture interface 23a296e3-8488-44bf-ae32-e5c628c0c7a7 [expression port 80]
100.64.144.1.4134: Flags [S.], seq 3105287591, ack 3951362276, win 28960, options [mss 1460,sackOK,TS val
405916912 ecr 1630120989,nop,wscale 7], length 0
<base64>AlBWVlMAAlBWVkRSCABFAAA8AABAAD8GPG4KAQEMZECQAQBQECa5Fu2n64UA5KAScSBBegAAAgQFtAQCCAoYMczwYSmsHQEDAwc=</b
ase64>
100.64.144.1.4134: Flags [.], ack 102, win 227, options [nop,nop,TS val 405916912 ecr 1630120992], length 0
<base64>AlBWVlMAAlBWVkRSCABFAAA0IVhAAD8GGx4KAQEMZECQAQBQECa5Fu2o64UBSYAQAOPgGwAAAQEIChgxzPBhKawg</base64>
Attention: On the T1-SR_downlink interface, you do not capture the dataplane

traffic from "LB to Pool-Member".
You capture only the response traffic from "Pool-Member to LB"
180
Troubleshooting
LB Syslog Messages (1/5)
Each Edge Node running LB service generates LB syslog messages:

• Every minute for summary of the LB/VIP/Pool
• When LB / VIP / Pool status change
• When LB generates an error
181
Troubleshooting
LB Syslog Message – Summary of LB/VIP/Pool (2/5)
Important:
Summary syslog of LB/VIP/Pool You need to run the nsx-exporter service at the debug level:
lab1-edge1> set service nsx-exporter logging-level debug
• LB information
To check the nsx-exporter logging-level (on Edge as root user):
– CPU, Memory root@lab1-edge1:~# cat /etc/vmware/nsx-edge-exporter/config.json
LOGGING_LEVEL_MAP =
– Enabled, Status, HA_State {
– # of Pools, # Pools up "off":"0",
"fatal":"1",
– # of Virtual Servers, # Virtual Servers Up "error":"2",
"warn":"3",
• Virtual Server(s) information "info":"4",
"debug":"5",
– Name, IP, Port "trace":"6"
}
– Status
• Pool(s) information
– Name,
– Individual Members IP, Port, Status
– Pool Backup usage
– Status
Example in the Notes

182
Troubleshooting
LB Syslog Messages – When LB / VIP / Pool status change (3/5)
Syslog message when LB / VIP / Pool status change

• Pool Member status change (for instance 1 Pool Member from UP to DOWN)
– 1 syslog message for Pool Member Down
2019-07-31T20:02:11.193020+00:00 2ec2e890c7c4 NSX 152 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="WARN"] [4857fbb2-5ffd-4391-
afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'PoolMember', Obj.Ip: '10.1.1.12', Obj.Port: '80', Pool.UUID:
'd7d702ac-6801-4968-99c6-6bf31a85c64b', Pool.Name: 'Pool1-Paris', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris', Vs.UUID:
'4f421b51-917f-4e95-a5d8-f00974774db2', Vs.Name: 'VIP2-Paris_HTTPS', Status.NewStatus: 'Down', Status.Msg: 'Receive Timeout'
– 1 syslog message for Pool Partially UP

afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'Pool', Obj.UUID: 'd7d702ac-6801-4968-99c6-6bf31a85c64b',
Obj.Name: 'Pool1-Paris', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris', Vs.UUID: '52f47036-d1b5-401e-914d-5d21c1a1c178',
Vs.Name: 'VIP1-Paris', Status.NewStatus: 'PartiallyUp', Status.Msg: 'pool members are partially up
– 1 syslog message for VIP Partially UP

afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'VirtualServer', Obj.UUID: '4f421b51-917f-4e95-a5d8-
f00974774db2', Obj.Name: 'VIP2-Paris_HTTPS', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris', Status.NewStatus: 'PartiallyUp',
Status.Msg: 'pool members are partially up'
183
Troubleshooting
LB Syslog Messages – When LB / VIP / Pool status change (4/5)
Syslog message when LB / VIP / Pool status change

• Pool/VIP status change (for instance all Pool Members from UP to DOWN)
– 1 syslog message for Pool Member Down
afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'PoolMember', Obj.Ip: '10.1.1.11', Obj.Port: '80', Pool.UUID:
'd7d702ac-6801-4968-99c6-6bf31a85c64b', Pool.Name: 'Pool1-Paris', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris', Vs.UUID:
'4f421b51-917f-4e95-a5d8-f00974774db2', Vs.Name: 'VIP2-Paris_HTTPS', Status.NewStatus: 'Down', Status.Msg: 'Receive Timeout'
– 1 syslog message for Pool DOWN
2019-07-31T20:15:46.443003+00:00 2ec2e890c7c4 NSX 152 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="ERROR"
errorCode="EDG1200000"] [4857fbb2-5ffd-4391-afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'Pool',
Obj.UUID: 'd7d702ac-6801-4968-99c6-6bf31a85c64b', Obj.Name: 'Pool1-Paris', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris',
Vs.UUID: '4f421b51-917f-4e95-a5d8-f00974774db2', Vs.Name: 'VIP2-Paris_HTTPS', Status.NewStatus: 'Down', Status.Msg: 'all pool members are down'
– 1 syslog message for VIP DOWN

2019-07-31T20:15:46.444179+00:00 2ec2e890c7c4 NSX 152 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="ERROR"
errorCode="EDG1200001"] [4857fbb2-5ffd-4391-afda-e447227d291a] Operation.Category: 'LbEvent', Operation.Type: 'StatusChange', Obj.Type: 'VirtualServer',
Obj.UUID: '4f421b51-917f-4e95-a5d8-f00974774db2', Obj.Name: 'VIP2-Paris_HTTPS', Lb.UUID: '4857fbb2-5ffd-4391-afda-e447227d291a', Lb.Name: 'LB-Paris',
Status.NewStatus: 'Down', Status.Msg: 'all pool members are down'
184
Troubleshooting
LB Syslog Messages – When LB generates an error (5/5)
Syslog message when LB generates an error

• Example with Pool Response with HTTP Header too big
2019-08-13T23:21:05.522954+00:00 nsx-edgebm3 NSX 7007 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb_log" level="ERROR"
errorCode="EDG9999999"] [5d68753b-9325-4119-9d21-98cf6b805c85] [error] 7007#0: *2868860 upstream sent too big header while reading response
header from upstream, client: 10.114.213.15, server: , request: "GET /big.php HTTP/1.1", upstream: "http://10.114.213.123:80/big.php", host:
"10.114.213.133"
Note: By default LB Profile allows up to 4k bytes HTTP response headers.

It's possible to increase it up to 64k bytes.
See slide " Layer7 HTTP VIP Configuration / Other configuration - Increase Response Header Size"
185
Troubleshooting
Edge Node and LB Monitoring – Capacity (1/4)
https://configmax.vmware.com/ The Edge Node hosts LB service (active/standby) based on its Tier-1
(active/standby) with LB attached.
Both LB active and standby are consuming resources in the Edge Node.
So for instance in "1 Edge VM - Large", in NSX-T 2.4 you can have up to:
"40 LB-Active", or "40 LB-Standby", or "20 LB-Active + 20 LB-
Standby", etc
LB Large have a factor of 40, LB Medium have a factor of 10, LB Small have a
factor of 1.
So 1 Edge VM – Large can host for instance: "40 LB Small" or "2 LB Medium
+ 20 LB Small" but not "3 LB Medium and 11 LB Small".
Load Balancer scale/provisioning is NOT affected / impacted by other

services hosted on Edge Nodes (i.e. Tier0, VPN, etc...)
186
Troubleshooting
Edge Node and LB Monitoring – Capacity of Edge Nodes (2/4)
• Based on Edge Node form factor, a fixed number of LB instances can be configured
• Monitor All Edge Nodes LB usage
https://<NSX-Mgr>/policy/api/v1/infra/lb-node-usage-summary
"results": [
{
"current_load_balancer_credits": 8,
"load_balancer_credit_capacity": 20, LB_Small = 1 credit
"current_pool_member_count": 20, LB_Medium = 10 credits
"pool_member_capacity": 4000, LB_Large = 40 credits
"usage_percentage": 40.0,
"severity": "GREEN", # of LB credits used in NX-T Platform
"node_counts": [ # of LB credits remaining in NX-T Platform (based on the # of Edge Nodes)
{
"severity": "GREEN",
"node_count": 2
# of Edge Nodes with },
. Plenty of LB capacity (Green) {
. Minor LB capacity (Orange) "severity": "ORANGE",
. No more LB capacity (Red) "node_count": 0
},
{
"severity": "RED",
"node_count": 0
}
],
"enforcement_point_path": "/infra/sites/default/enforcement-points/default"
}
], 187
"intent_path": "/infra/lb-node-usage-summary"
Troubleshooting
Edge Node and LB Monitoring – Capacity of Specific Edge Node (3/4)
• Monitor a Specific Edge Node LB usage
https://<NSX-Mgr>/policy/api/v1/infra/lb-node-usage?node_path=/infra/sites/default/enforcement-
points/default/edge-clusters/<edgecluster-uuid>/edge-nodes/<edgenode-uuid>
{
"form_factor": "MEDIUM_VIRTUAL_MACHINE",
"edge_cluster_path": "/infra/sites/default/enforcement-points/default/edge-clusters/f62c83e6-7adc-4704-81bb-42d03c9e4f46",
"current_load_balancer_credits": 4,
"load_balancer_credit_capacity": 10,
"usage_percentage": 40.0,
"severity": "GREEN", LB_Small = 1 credit
"current_pool_member_count": 10, LB_Medium = 10 credits
"current_virtual_server_count": 7, LB_Large = 40 credits
"current_pool_count": 6,
"pool_member_capacity": 2000, # of LB credits used in NX-T Platform
"current_small_load_balancer_count": 4, # of LB credits remaining in NX-T Platform (based on the # of Edge Nodes)
"current_medium_load_balancer_count": 0,
"current_large_load_balancer_count": 0,
"current_xlarge_load_balancer_count": 0,
"remaining_small_load_balancer_count": 6,
"remaining_medium_load_balancer_count": 0,
Specific LB capacity
"remaining_large_load_balancer_count": 0,
"remaining_xlarge_load_balancer_count": 0,
"resource_type": "LBEdgeNodeUsage",
"node_path": "/infra/sites/default/enforcement-points/default/edge-clusters/f62c83e6-7adc-4704-81bb-42d03c9e4f46/edge-
nodes/2ea190d2-b6df-11e9-9296-0050568463ea"
}
188
Troubleshooting
Edge Node and LB Monitoring – Capacity of LB Services (4/4)
• Monitor All LB Service usage
https://<NSX-Mgr>/policy/api/v1/infra/lb-service-usage-summary
{
"pool_usage_percentage": 2.5,
"pool_severity": "GREEN",
"pool_capacity": 240,
"current_pool_count": 6,
"virtual_server_usage_percentage": 8.75,
"virtual_server_severity": "GREEN", Pool + Pool Member + Virtual
"virtual_server_capacity": 80, Server usage
"current_virtual_server_count": 7,
"pool_member_usage_percentage": 0.83,
"pool_member_severity": "GREEN",
"pool_member_capacity": 1200,
"current_pool_member_count": 10,
"service_counts": [
{
"severity": "RED",
# of LB Service with "service_count": 0
. Plenty of LB Service capacity (Green) },
. Minor LB Service capacity (Orange) {
. No more LB Service capacity (Red) "severity": "ORANGE",
"service_count": 0
}, LB Service usage Information for a specific LB Service is also available:
{ GET /policy/api/v1/infra/lb-services/<lb-service-id>/service-usage
"severity": "GREEN",
"service_count": 4 189
}
Troubleshooting
Edge Node and LB Monitoring – Performance of Edge Node (1/3)
Performance has to be monitored on:

• Edge Node
• Load Balancer
LB CPU Edge CPU Action Required from Admin

Low Low No action required
Increase the size of Edge node
Low High Or find out an Edge node with low utilization and move the LB
Or add a new Edge node and move the LB into
High Low Increase the LB instance size
Increase the size of Edge node
High High Or find out an Edge node with low utilization and move the LB
Or add a new Edge node and move the LB
190
Troubleshooting
Edge Node and LB Monitoring – Performance of Edge Node (2/3)
• LB Performance is related to the Edge Node CPU.

• By default 1/2 of the Edge Node vCPU are used by DPDK
(Edge Nodes BM: 1/2 of its Cores or 1/4 if hyperthreading enabled)
– For instance on Edge Large (8 vCPU), DPDK uses 4 vCPU, and so remaining 4 vCPU are available for other services such as LB.
• Monitor the Edge Node CPU via SSH (as root)

• "top" + "1”
top - 00:40:49 up 8:20, 1 user, load average: 1.87, 1.74, 1.21
Tasks: 222 total, 1 running, 136 sleeping, 0 stopped, 1 zombie
%Cpu0 : 5.2 us, 2.4 sy, 0.0 ni, 91.8 id, 0.7 wa, 0.0 hi, 0.0 si, 0.0 st
• Check if any of those is constantly using high CPU

• If that's the case, you reach the Edge Node performance limit
More information in the Notes.
191
Troubleshooting
Edge Node and LB Monitoring – Performance of Load Balancer (3/3)
• Based on Load Balancer form factor, one or more Edge Node CPU will be consumed
Note: LB oversubscription is allowed by NSX-T platform.
• Monitor LB load via API

https://<NSX-Mgr>/policy/api/v1/infra/lb-services/LB1-GW1/detailed-status
{
"results": [
{
<snip>
"virtual_servers": [
<snip>
"pools": [
<snip>
"memory_usage": 2, Usage in %
"cpu_usage": 1,
"active_transport_nodes": [
"72a83fae-b763-11e9-93d9-005056844f6f"
],
"standby_transport_nodes": [
"2ea190d2-b6df-11e9-9296-0050568463ea"
],
<snip>
• Check if any of those is constantly close to 100%

• If that's the case, you reach the NSX-T LB performance limit
192
Troubleshooting New NSX-T 3.1
LB Diagnosis (1/1)
• Edge Node CLI command to get a quick diagnosis of all LB components
lab1-edge1> get load-balancer 3554fa76-2b3b-4690-ba65-3675706155dc diagnosis

Fri Dec 18 2020 UTC 03:07:16.017
Checking
Action : checking system
Result : passed
Action : checking crash

Result : passed
Action : checking daemon status

Result : passed
Action : checking configuration

Result : passed
Action : checking runtime

Result : passed
Action : checking stats

Result : passed
193
Troubleshooting
Error Messages (1/1) This LB is attached to a Tier-1, and that Tier-1 is active/standby in 2 Edge nodes.
Edge Nodes can host a specific number of LB (See "LB Service Scale").
The Edge Nodes hosting the Tier-1 active or Tier-1 standby don't have resources to add that LB.
• There is no available capacity on edge node …
194
Troubleshooting
Miscellaneous - Cascade of VIP
Pool Members of VIP1 are VIP2.
This topology is supported as long as

VIP1 and VIP2 are on different NSX-T
S
LB Services.
S
VIP1 VIP2 S
L4 or L7 L4 or L7
VIP1 VIP2
Server Server
Pool Pool
VIP1 and VIP2 must be attached to different LB Services.
195

NSX-T 3.1 LB Encyclopedia-V1.3

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

NSX-T 3.1 LB Encyclopedia-V1.3

Uploaded by

Copyright:

Available Formats

NSX-T Load Balancer

5 Layer7 HTTP VIP

6 Layer7 HTTPS VIP

7 LB Rules (HTTP or HTTPS)

5 Layer7 HTTP VIP

6 Layer7 HTTPS VIP

7 LB Rules (HTTP or HTTPS)

• Load Balancer Deployment

LB InLine Deployment 2 Enable LB on an existing Tier-1 GW

LB-SNAT can be required depending on

Tier-1+LB Tier-1+LB 2 LB-SNAT not required:

Overlay Overlay VLAN VLAN

LB OneArm Deployments Deploy dedicated One-Arm Tier-1 GW for

Can be deployed on Overlay or VLAN

For LB OneArm using T1 Service Interface

– Configure its Default Gateway

For LB OneArm using T1 Uplink Interface

• Monitor (see section "Monitors and Server Pools")

• Pool (see section "Monitors and Server Pools")

IPv4 Load Balancing support

Clients VIP Pool Members

IPv6 Load Balancing support

Clients VIP Pool Members

Create Load Balancer and attach it to existing Tier-1

Load Balancer scale/provisioning is NOT affected / impacted by other

Max # of rules per VIP = 512.

# of LB and #VIP per LB

Tier-1 offers different centralized

Active / Hot-Standby per LB

LB HA heartbeat per LB done by Edge

Active / Hot-Standby per LB

(0.9 sec later on EN-BM)

CLI When synch is needed, the steps are:

5 Layer7 HTTP VIP

6 Layer7 HTTPS VIP

7 LB Rules (HTTP or HTTPS)

• Active Monitor Configuration

• Passive Monitor Configuration

Active Monitor Passive Monitor

Supported health monitor types Failure detection methods

LB Monitor Server Pool 1. LB generates a Monitor request to

LB monitors data plane traffic.

Create ICMP Monitor

How many consecutive failures before How many consecutive successes

Size of ICMP packet

Create TCP Monitor

How many consecutive failures before LB

Optional: TCP Payload packet Optional: Payload LB expects from

How many consecutive failures before LB

How many consecutive successes

Payload LB expects from Pool

Create HTTP Monitor

How many consecutive failures before LB

See next slide

– Add Active Monitor – HTTP (2/2)

Optional: Expected Response code.

HTTP Version (1.0 and 1.1 are supported.

Optional: HTTP Body

Create HTTPS Monitor

How many consecutive failures before LB

See next slides

– Add Active Monitor – HTTPS (2/3)

Optional: Expected Response code.