You are on page 1of 20
VSX CoreXL and CPU affinity [Restricted] ONLY for designated groups and individuals ©2012 Check Point Software

VSX

CoreXL and CPU affinity

VSX CoreXL and CPU affinity [Restricted] ONLY for designated groups and individuals ©2012 Check Point Software

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

Course Timetables

Course Timetables Day 1 Day 2 Day 3 9:00 Course Introduction VSX Clustering VSX Conversion 10:00
 

Day 1

 

Day 2

Day 3

9:00

Course Introduction

 

VSX Clustering

VSX Conversion

10:00

   

RC & QoS

 

R75.40VS VSX

 

vsx_utill

Gaia VS CTX & New Features

11:00

Introduction

 

(Conversion, SNMP, JF)

12:00

Mgmt. Implementation

L2 VS

13:00

 

Lunch Break

14:00

     

15:00

VSX Networking

 

GW Implementation

Meeting with Check Point

16:00

VSX CoreXL Affinity &

 

R&D

17:00

Memory RC

Debug & Troubleshooting

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

2

CoreXL

CoreXL architecture

CoreXL CoreXL architecture  Parallelise security gateway kernel  Leverage modern processor architectures  Suited to
  • Parallelise security gateway kernel

  • Leverage modern processor architectures

  • Suited to medium path

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

3

Security Gateway CoreXL

  • Firewall kernel Replication

Security Gateway CoreXL  Firewall kernel Replication – Firewall kernel is replicated multiple times. Each runs

Firewall kernel is replicated multiple times. Each runs on one processing core.

Each instance is independent FW-1 kernel. Instances can run concurrently – don’t share a global lock.

  • Dispatcher New component introduced in CoreXL. Receives packets and forwards them to the kernel instances.

Acts as a load balancer. The dispatching is based on a hash of the source IP, Destination IP, Destination port and IP protocol (4-tuple)

The dispatcher must maintain core stickiness per connection

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

4

CoreXL - First Packet Flow

CoreXL - First Packet Flow IP stack fw0 conn table WT Queue fw1 conn table WT
IP stack
IP stack
fw0 conn table WT Queue
fw0
conn
table
WT
Queue
fw1 conn table WT Queue
fw1
conn
table
WT
Queue
Record Conn fw2 conn table WT 2 Queue
Record
Conn
fw2
conn
table
WT
2
Queue
Arbitrary Decision Dispatcher global conn table Lookup. Not found
Arbitrary
Decision
Dispatcher
global conn table
Lookup.
Not found

[Restricted] ONLY for designated groups and individuals

PKT
PKT

©2012 Check Point Software Technologies Ltd.

5

CoreXL - Second Packet Flow

CoreXL - Second Packet Flow IP stack fw0 conn table WT Queue fw1 conn table WT
IP stack
IP stack
fw0 conn table WT Queue
fw0
conn
table
WT
Queue
fw1 conn table WT Queue
fw1
conn
table
WT
Queue
fw2 conn table WT Queue
fw2
conn
table
WT
Queue
Dispatcher global conn table Lookup. Found 2
Dispatcher
global conn table
Lookup.
Found
2

[Restricted] ONLY for designated groups and individuals

PKT
PKT

©2012 Check Point Software Technologies Ltd.

6

CoreXL - Parallel Processing
CoreXL - Parallel Processing
CoreXL - Parallel Processing IP stack fw0 conn table WT Queue fw1 conn table WT Queue
IP stack
IP stack
fw0 conn table WT Queue
fw0
conn
table
WT
Queue
fw1 conn table WT Queue
fw1
conn
table
WT
Queue
fw2 conn table WT Queue
fw2
conn
table
WT
Queue
Dispatcher global conn table 0 1 2
Dispatcher
global conn table
0
1
2

[Restricted] ONLY for designated groups and individuals

PKT PKT PKT
PKT
PKT
PKT

©2012 Check Point Software Technologies Ltd.

7

CoreXL

CoreXL Core #0 Dispatcher PPAK SND eth0 Core #1 Dispatcher PPAK SND eth1 Core #2 fw5
Core #0 Dispatcher PPAK SND eth0
Core #0
Dispatcher
PPAK
SND
eth0
Core #1 Dispatcher PPAK SND eth1
Core #1
Dispatcher
PPAK
SND
eth1
Core #2
Core #2
fw5
fw5
Core #2 fw5 Medium Path Queue
Core #2 fw5 Medium Path Queue
 
 

Medium Path

Queue

 
Core #3
Core #3
fw4
fw4
Core #3 fw4 Medium Path Queue
Core #3 fw4 Medium Path Queue
 
 

Medium Path

Queue

 
Core #4
Core #4
fw3
fw3
Core #4 fw3 Medium Path Queue
Core #4 fw3 Medium Path Queue
 
 

Medium Path

Queue

 
Core #5
Core #5
fw2
fw2
Core #5 fw2 Medium Path Queue
Core #5 fw2 Medium Path Queue
 
 

Medium Path

Queue

 
Core #6
Core #6
fw1
fw1
Core #6 fw1 Medium Path Queue
Core #6 fw1 Medium Path Queue
 
 

Medium Path

Queue

 
Core #7
Core #7
fw0
fw0
Core #7 fw0 Medium Path Queue
Core #7 fw0 Medium Path Queue
 
 

Medium Path

Queue

 
  • Accelerated Path Cores are allocated via Interface IRQ Affinity

  • Secure Network Dispatcher queues packets to firewall instances running Firewall and Medium Paths

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

8

Accelerated Path – No Template
Accelerated Path – No Template
Accelerated Path – No Template Core # ... FW Core #4 Medium FW Medium Core #
Core # ... FW Core #4 Medium FW Medium Core # ... FW Medium Path Path
Core # ...
FW
Core #4
Medium
FW
Medium
Core # ...
FW
Medium
Path
Path
Path
Path
Path
Path
Queue
Queue
Queue
Core #0
Dispatcher
Performance Pack
Core #1
Dispatcher
Performance Pack
eth1
eth0
eth0
  • Syn

  • SynAck + subsequent S2C packets

  • Subsequent C2S packets

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

9

Accelerated Path – With Template
Accelerated Path – With Template
Accelerated Path – With Template Core # ... FW Core #4 Medium FW Medium Core #
Core # ... FW Core #4 Medium FW Medium Core # ... FW Medium Path Path
Core # ...
FW
Core #4
Medium
FW
Medium
Core # ...
FW
Medium
Path
Path
Path
Path
Path
Path
Queue
Queue
Queue
Core #0
Dispatcher
Performance Pack
Core #1
Dispatcher
Performance Pack
eth0
eth1
  • Syn + subsequent C2S packets

  • SynAck + subsequent S2C packets

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

10

Medium Path – IPS Traffic
Medium Path – IPS Traffic
Medium Path – IPS Traffic Core # ... FW Core #4 Medium FW Medium Core #
Core # ... FW Core #4 Medium FW Medium Core # ... FW Medium Path Path
Core # ...
FW
Core #4
Medium
FW
Medium
Core # ...
FW
Medium
Path
Path
Path
Path
Path
Path
Queue
Queue
Queue
Core #0
Secure Dispatcher
Performance Pack
Core #1
Secure Dispatcher
Performance Pack
eth1
eth0
eth0
  • Syn + subsequent C2S packets

  • SynAck + subsequent S2C packets

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

11

VSX CoreXL

VSX CoreXL

VSX CoreXL VSX CoreXL  Same idea as applied for SG is applied to VSX CoreXL.
  • Same idea as applied for SG is applied to VSX CoreXL.

  • Main difference, instance in FWK (fw kernel equivalent) are executed by UM threads.

  • VSX CoreXL can be applied for any existing VS simultaneously with different number of instances.

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

12

VSX CoreXL cont.

VSX CoreXL affinity

VSX CoreXL cont. VSX CoreXL affinity  VSX CoreXL does not affine FWK instance per core.
  • VSX CoreXL does not affine FWK instance per core.

  • Affinity can be set manually per instance.

VSX Semi Static affinity

  • Semi Static affinity will assign FWK to run on a default number of cores.

  • The number of cores is calculated using a formula.

  • This number can also be changed by a manual command (fwkall).

  • The cores chosen will have a physical proximity.

  • Any manual settings to FWK or FWK instance will override the semi static affinity.

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

13

VSX CoreXL configuration

VSX CoreXL configuration  CoreXL configuration for VS0 is done using cpconfig This program will let
  • CoreXL configuration for VS0 is done using cpconfig

This program will let you re-configure your Check Point products configuration.

Configuration Options:

----------------------

(1) Licenses and contracts (2) SNMP Extension (3) PKCS#11 Token (4) Random Pool (5) Secure Internal Communication

(6) Enable cluster membership for this gateway

(7) Disable Check Point SecureXL

(8) Configure Check Point CoreXL

(9) Automatic start of Check Point Products

(10) Exit

Enter your choice (1-10) :

  • CoreXL for VS which is not VS0 is done using SmartDashboard

VSX CoreXL configuration  CoreXL configuration for VS0 is done using cpconfig This program will let

Note: changing CoreXL configuration (num of instances) will require downtime of the VS (VS0 or other).

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

14

VSX Affinity

VSX Affinity VSX affinity handles the following VSX entities  VS - Setting affinity for VS
VSX Affinity VSX affinity handles the following VSX entities  VS - Setting affinity for VS

VSX affinity handles the following VSX entities

  • VS - Setting affinity for VS means setting affinity for all processes related to the specific VS (all processes with the specific vrf)

  • Process - Setting affinity is possible for specific process per specific VS.

  • FWK instance Setting affinity for FWK instance means setting affinity per FWK thread (any FWK instance for any VS)

  • FWKALL Setting affinity for all FWK processes by the number of CPUs, the specific cores are chosen by the gateway.

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

15

VSX Affinity

VSX Affinity  Affinity persistency - Vsx affinity is maintained over reboot and over process crash
VSX Affinity  Affinity persistency - Vsx affinity is maintained over reboot and over process crash
  • Affinity persistency - Vsx affinity is maintained over reboot and over process crash cycle using configuration files

  • Affinity Exceptions VSX affinity can handle process exceptions which will be chosen by the user. VSX affinity does not affine kernel threads.

  • Affinity Priority VSX affinity entities can be set together in the same VS, using the following priority

FWK instance Process VS In case of override the user will be prompt for actions

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

16

VSX Affinity Usage

  • Setting Affinity

VSX Affinity Usage  Setting Affinity – Interface Affinity : fw ctl affinity -s -i <interface>

Interface Affinity: fw ctl affinity -s -i <interface> <cpuids | all>

VS affinity (VS,VR,VSW): fw ctl affinity -s -d [-vsid <ranges>] -cpu <ranges>

Process affinity - fw ctl affinity -s -d -pname <process name> [-vsid <ranges>] -cpu <ranges>

pid Affinity - fw ctl affinity -s -p <pid> <cpuids | all> FWK instance affinity - fw ctl affinity -s -d -inst <ranges> -cpu <ranges> All FWKs affinity - fw ctl affinity -s -d -fwkall <num of CPUs> Note: If vsid flag is omitted, the current context will be used.

  • Listing Affinity

Configured affinity - fw ctl affinity -l

Extended Affinity - fw ctl affinity -l -x [-vsid <ranges>] [-cpu <ranges>] [-flags e|k|t|n]

Flags:

e don't print exception processes k don't print kernel threads t print also all process threads n print process name instead of /proc/<pid>/cmdline

h print CPU mask in hex format

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

17

Usage Examples

  • Setting affinity examples

fw ctl affinity -s -d -fwkall 3

fw ctl affinity -i eth0 0 3 7

fw ctl affinity -s -d -inst 0 2 4 -cpu 5

Usage Examples  Setting affinity examples – fw ctl affinity -s -d -fwkall 3 – fw

fw ctl affinity -s -d -pname cpd -vsid 0-12 -cpu 7

fw ctl affinity -s -d -vsid 0-2 4 6-8 -cpu 0-2 4

  • Listing Affinity example

fw ctl affinity -l

Output:

eth0: CPU 1

VS_0 FWK_INSTANCE_0: CPU 0 1 2

VS_0 fwk: CPU 2 3

VS_1 FWK_INSTANCE_0: CPU 0

VS_1 FWK_INSTANCE_1: CPU 1

VS_1 FWK_INSTANCE_2: CPU 2

VS_1 fwk: CPU 2 3

VS_2 cpd: CPU 1 2 3

VS_2 fwk: CPU 2 3

VS_3 fwd: CPU 1 3

VS_3 fwk: CPU 0 3

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

18

Usage Examples (cont)

  • Extended Affinity List example

fw ctl affinity l x vsid 1 flags tnek

Output:

Usage Examples (cont)  Extended Affinity List example – fw ctl affinity – l – x

-------------------------------------------------------

|PID

|VSID |

CPU

|SRC|V|KT |EXC| NAME

-------------------------------------------------------

|

4835 |

1

|

all |

 

|

|

|

| routed

|

21094 |

1

|

all |

|

|

|

| fwk_wd

|

21096 |

1

|

all |

|

|

|

| cpd

|

21241 |

1

|

all |

|

|

|

| |---cpd

|

21244 |

1

|

all |

|

|

|

| |---cpd

|

21245 |

1

|

all |

|

|

|

| |---cpd

|

21107 |

1

|

all |

|

|

|

| mpdaemon

|

21115 |

1

|

2 3

|

P

|

|

|

| fwk1_dev

|

21116 |

1

|

0

|

I

|

|

|

| |---fwk1_0

|

21117 |

1

|

1

|

I

|

|

|

| |---fwk1_1

|

21118 |

1

|

2

|

I

|

|

|

| |---fwk1_2

|

21119 |

1

|

2

3

|

P

|

|

|

| |---fwk1_3

|

21401 |

1

|

all |

|

|

|

| fw

|

21411 |

1

|

all |

|

|

|

| |---fw

|

21412 |

1

|

all |

|

|

|

| |---fw

|

21413 |

1

|

all |

|

|

|

| |---fw

-------------------------------------------------------

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

19

CoreXL with Affinity Example

CoreXL with Affinity Example  Command used for viewing fwk setup in the following example –
  • Command used for viewing fwk setup in the following example

fw ctl affinity l x vsid 1 flags tn | grep fwk | grep v fwk_

  • Before Setting CoreXL

|

21115 |

1

|

2

3

|

P

|

|

|

| fwk1_dev

|

21116 |

1

|

0

|

I

|

|

|

| |---fwk1_0

  • After Setting CoreXL (Used SDB to configure 4 instances)

 

|

21115 |

1

|

2

3

|

P

|

|

|

| fwk1_dev

|

21116 |

1

|

2 3

|

P

|

|

|

| |---fwk1_0

|

21117 |

1

|

2 3

|

P

|

|

|

| |---fwk1_1

|

21118 |

1

|

2 3

|

P

|

|

|

| |---fwk1_2

|

21119 |

1

|

2

3

|

P

|

|

|

| |---fwk1_3

  • Set affinity for instance 0 (fw ctl affinity s d inst 0 cpu 0)

|

21115 |

1

|

2

3

|

P

|

|

|

| fwk1_dev

|

21116 |

1

|

0

|

I

|

|

|

| |---fwk1_0

|

21117 |

1

|

2

3

|

P

|

|

|

| |---fwk1_1

|

21118 |

1

|

2

3

|

P

|

|

|

| |---fwk1_2

|

21119 |

1

|

2

3

|

P

|

|

|

| |---fwk1_3

  • Set affinity for instance 2 and 3 (fw ctl affinity s d inst 2 3 cpu 1 2)

|

21115 |

1

|

2 3

|

P

|

|

|

| fwk1_dev

|

21116 |

1

|

0

|

I

|

|

|

| |---fwk1_0

|

21117 |

1

|

2

3

|

P

|

|

|

| |---fwk1_1

|

21118 |

1

|

1 2

|

I

|

|

|

| |---fwk1_2

|

21119 |

1

|

1 2

|

I

|

|

|

| |---fwk1_3

[Restricted] ONLY for designated groups and individuals

©2012 Check Point Software Technologies Ltd.

20