Arm Overview v4

ARM Replacement
Jeff Cochran
Copyright © 2016 Forcepoint. All rights reserved.

New Under ARM
1. Legacy ARM 1. New ARM
1. Uses in-house kernel modules for 1. Uses open source software for
interception of traffic interception of traffic
2. Transparent interception works via 2. Transparent interception does not modify
Network Address Translation (changing packets
the destination information of packets) 3. Python implementation
3. C/C++ implementation
INTERNAL USE ONLY

Copyright © 2016 Forcepoint. All rights reserved. | 2
ARM AND NETCONTROL
1. A large portion of the ARM features are now facilitated by a new python module
called netcontrol
2. The purpose of netcontrol is to manage and affect configurations of the firewall,

OS, and network adapters on the deployment system for the interception,
manipulation and redirection of network traffic
3. Whereas ARM used to have form as a kernel module, it can now be thought of
as a collection of features, most of which are provided by the netcontrol module
4. netcontrol‘s features can be explored by invoking

/opt/WCG/bin/netcontrol.sh -h
INTERNAL USE ONLY

NETCONTROL ON PERFORCE
1. The netcontrol module is available at
//ws3/WSG/main/src/dev/WCG/util/netcontrol
2. The main module is core.py
3. Other modules are broken down loosely by feature (wccp.py, bypass.py) and
function (iptables.py)
4. Runs on Python 2.7

1. On a customer’s deployment, Python 2.7 is bundled with the product and deployed alongside –
not replacing – the Python installation on the system
INTERNAL USE ONLY

New ARM Underpinnings
/opt/WCG/bin/netcontrol.sh
/opt/WCG/bin/netcontrol
iptables / ip6tables
Linux Kernel
INTERNAL USE ONLY

GRASPING IPTABLES WITH ARM
Chains Tables
1. A sequential list of rules used to manipulate, 1. A collection of chains
differentiate, reject or redirect packets entering or
leaving the system
2. Each table has a broad, but well-defined
2. Packets enter built-in chains at certain points in intended purpose
the linux networking pipeline 1. For example, the NAT table is supposed to
1. Built-in chains have a default policy, such as be used for NATing packets, the Filter table
accepting or dropping packets, in the event that a for restricting which packets the system will
packet matches no rules in the chain accept (traditional firewalling), etc.
3. A system admin can cause packets to enter

user-defined chains by inserting “JUMP” rules in 3. As such, some types of rules can only be
built-in chains used in certain tables
1. You may hear dev refer to these types of rules as 1. For example, NAT rules can only be added
“hooks” to chains in the NAT table
2. If a packet matches no rules in a user-defined
chain, it will return to the chain which we JUMPed
in from 4. You can’t JUMP to a chain in another table
INTERNAL USE ONLY
SIMPLE FILTER TABLE ON NON-WCG SYSTEM
INTERNAL USE ONLY

GETTING THE GOODS – VIEWING IPTABLES CONFIGURATION
sudo iptables -t filter -vnL

List rules in the table
Show numeric output instead of
hostnames and service names
We’d like to operate on the Verbose output – show all info
Must be root to even view filter table. If we wanted to about the rules – very confusing
configuration! see the NAT table, we would without this flag!
type ‘nat’ here instead
INTERNAL USE ONLY

IPTABLES TABLES OF INTEREST
Mangle Table Filter Table
1. Comes before filter table in the network 1. Intended purpose is for making
pipeline accept/reject decisions on packets
2. Intended purpose is for actually modifying 2. The table operated on by default (no -t
packets before they are processed by the flag) – only table necessary for basic
system (not including NATting)
firewalling
3. Used primarily by netcontrol to facilitate
interception or bypass of select traffic 3. Used primarily by netcontrol to allow
proxy-related packets to be accepted
4. Contains the PREROUTING chain, which regardless of end-user firewall
allows us to manipulate packets before a configuration
routing decision is made on them by the
kernel
INTERNAL USE ONLY

THE FILTER TABLE – (MOSTLY) STOCK
INTERNAL USE ONLY

THE MANGLE TABLE - STOCK
INTERNAL USE ONLY

NETCONTROL CHAIN ENTRY POINTS
1. ARM chains all share the

prefix NC_
2. Packet processing is
redirected to these reserved
chains by adding
unconditional JUMP rules to
the top of certain built-in
chains
1. Again, dev refers to these
types of rules as hooks
INTERNAL USE ONLY

ARM CHAINS IN THE MANGLE TABLE
1. NC_RESERVED 3. NC_RESERVED_SPOOFING
1. Main entry point for ARM logic in the 1. Used to facilitate IP spoofing feature
mangle table
2. This is where transparent interception 4. NC_RESERVED_POSTROUTING
rules are handled, as configured in the old 1. Used for loadshedding
NAT page on the WCG UI
3. This chain also contains some small one- 5. NC_TUNNELBACK
off rules for features which have a
1. Used for GRE tunnelback
configuration simple enough to make
creation of a dedicated chain unjustified 
Tunnel-back is a configuration in which
communication from the proxy to the
WCCP router is encapsulated in a GRE
2. NC_BYPASS tunnel. This is different from GRE return,
1. Used for bypassing traffic based on where packets which are rejected by the
configured static and dynamic bypass proxy are sent to the router
rules 
This feature requires the end user to
install additional software that must
be compiled on the box to function
INTERNAL USE ONLY

ARM CHAINS IN THE FILTER TABLE
1. NC_RESERVED_INPUT
1. Responsible for allowing traffic based on redirection rules to be received by the system
1. Product help documentation previously referred to redirection rules as NAT rules. Since we’re no longer
NATting traffic, this name and the corresponding configuration interface on the UI have been renamed
2. Responsible for ensuring that packets to/from Forcepoint ports used by WCG and other first-party
infrastructure is allowed by the firewall
2. NC_RESERVED_FORWARD
1. Used to permit packets that are being bypassed or loadshedded to proceed to their destinations
INTERNAL USE ONLY

TROUBLESHOOTING FIREWALL CONFIGURATIONS
1. Since system administrators can configure iptables, it is possible for them to
cause the firewall to enter a state which damages the operability of the ARM
features
2. It is predicted that the most likely mistakes a customer could make are
1. Flushing the firewall rules or disabling iptables
2. Inserting rules before the JUMP rules which netcontrol uses as hooks
3. Modifying or removing rules in the reserved chains
INTERNAL USE ONLY

FIXING A CUSTOMER-MODIFIED RESERVED CHAIN
1. The reserved chains are flushed and reconstructed when the proxy restarts
2. A troubleshooter could restart WCG from the UI to cause the chains to be

reconstructed
INTERNAL USE ONLY

FIXING HOOKS
1. It is possible that our JUMP statements are no longer in the correct location in
the customer’s built-in chains
2. Re-initialization of netcontrol can be performed by invoking the following

command
1. /opt/WCG/bin/netcontrol.sh --recover
2. Invoking the recover command should cause no loss of configuration in netcontrol, other than
forgetting any dynamic bypass rules that have been triggered
INTERNAL USE ONLY

WHEN RECOVER FAILS, OR NETCONTROL CAN’T INITIALIZE
1. If netcontrol fails when invoking --recover, it is likely that the customer has
deleted parts of the netcontrol configuration from the firewall
2. If this is the case, then forcibly teardown netcontrol configuration

1. From the command line, invoke /opt/WCG/bin/netcontrol.sh -t force
1. Expect to see a lot of error output when invoking a forced teardown
3. After performing the forced teardown, attempt reinitialization of netcontrol

1. From the command line, invoke /opt/WCG/bin/netcontrol.sh -i
1. Upon success, the script will return a success return code and the line “NETCONTROL_DONE” should be
displayed
4. If initialization of netcontrol fails even after a forced teardown, manual cleanup

of the system configuration may be required
INTERNAL USE ONLY

LOOKING FOR ERRORS POSTMORTEM
1. The netcontrol script is invoked by WCG when it restarts, or when configuration
changes happen on the UI
2. In such a scenario, any failures in netcontrol will be logged to /var/log/messages
3. Search the log for netcontrol to identify failures postmortem

1. All netcontrol messages in the system log contain the prefix “netcontrol”
INTERNAL USE ONLY

A SLICE OF THE INPUT PACKET NETWORKING FLOW
INTERNAL USE ONLY

A SLICE OF THE OUTPUT PACKET NETWORKING FLOW
INTERNAL USE ONLY

THE WHOLE SHEBANG – ENTIRE NETFILTER FLOW DIAGRAM
Schematic for the packet flow

paths through Linux networking
and Xtables by Jan Engelhardt
https://commons.wikimedia.org/wiki/File:Netfilter-packet-fl
ow.svg
CC BY-SA 3.0
INTERNAL USE ONLY

PROVIDING DEV WITH INFORMATION WHEN ESCALATING
1. If a customer’s ARM needs investigation, 5. ip route list
as much of the following information 1. This will display more routing configuration. This is
not to be confused with step 4. Make sure to
should be provided to dev as possible clearly differentiate the results of step 4 from step 5
2. Execute iptables-save 6. /opt/WCG/bin/netcontrol.sh -s

>/tmp/iptables-config 1. This will display whether netcontrol believes it has
been initialized
1. This creates a file in /tmp/iptables-config
of the current firewall configuration
7. /opt/WCG/bin/netcontrol.sh -c
1. This will do a quick sanity check to ensure some
3. ip rule list critical features are provided by the deployment
1. This will display policy routing information machine
2. This will not work on an appliance. Skip this step if
on appliance
4. ip route list table nc-reserved
1. This will display some routing 8. Contents of /var/log/messages
configuration
2. This call will fail if netcontrol has not been 9. Contents of the /opt/WCG/config directory,
successfully initialized on the system stripped of private information if necessary
INTERNAL USE ONLY

EVEN MORE INFO TO PROVIDE FOR AN ESCALATION
10. Contents of the /opt/WCG/logs directory
11. ip link
1. This provides info about the configured network adapters
12. ip addr
1. This provides other info about the configured network adapters
13. sysctl -a
1. Wow that’s a lot of output! This step will provide information about kernel options
14. lsmod
1. This step lists kernel modules that are currently loaded
15. If a problem exists relating to certain network traffic, a pcap of the bad behavior should be provided as
well if possible
16. Thank you for collecting all that information! Hopefully this should be everything dev needs to identify
problems with on-box configuration
INTERNAL USE ONLY
NETCONTROL CHECKLIST
http://ssdengwiki1.websense.com/doku.php?id=wiki:wcg_netcontrol_checklist
INTERNAL USE ONLY

Thank you!
Please contact me at jcochran@forcepoint.com with
any questions
FORCEPOINT
INTERNALPROPRIETARY
USE ONLY

Arm Overview v4

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Arm Overview v4

Uploaded by

Copyright:

Available Formats

ARM Replacement

Copyright © 2016 Forcepoint. All rights reserved.

INTERNAL USE ONLY

2. The purpose of netcontrol is to manage and affect configurations of the firewall,

4. netcontrol‘s features can be explored by invoking

INTERNAL USE ONLY

2. The main module is core.py

4. Runs on Python 2.7

INTERNAL USE ONLY

INTERNAL USE ONLY

3. A system admin can cause packets to enter

INTERNAL USE ONLY

sudo iptables -t filter -vnL

INTERNAL USE ONLY

INTERNAL USE ONLY

INTERNAL USE ONLY

INTERNAL USE ONLY

1. ARM chains all share the

INTERNAL USE ONLY

INTERNAL USE ONLY

INTERNAL USE ONLY

INTERNAL USE ONLY

2. A troubleshooter could restart WCG from the UI to cause the chains to be

INTERNAL USE ONLY

2. Re-initialization of netcontrol can be performed by invoking the following

INTERNAL USE ONLY

2. If this is the case, then forcibly teardown netcontrol configuration

3. After performing the forced teardown, attempt reinitialization of netcontrol

4. If initialization of netcontrol fails even after a forced teardown, manual cleanup

INTERNAL USE ONLY

2. In such a scenario, any failures in netcontrol will be logged to /var/log/messages

3. Search the log for netcontrol to identify failures postmortem

INTERNAL USE ONLY

INTERNAL USE ONLY

INTERNAL USE ONLY

Schematic for the packet flow

INTERNAL USE ONLY

2. Execute iptables-save 6. /opt/WCG/bin/netcontrol.sh -s

INTERNAL USE ONLY

INTERNAL USE ONLY

You might also like