NEXT GENERATION FIREWALL TEST REPORT

Forcepoint NGFW 3301 Appliance v6.1.2

JUNE 6, 2017
Authors – Jeff Bowermon, Devon James, Ty Smith

This report is Confidential, for internal use only, and is expressly limited to NSS Labs’ licensed users.

NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.1.2_060617

Overview
NSS Labs performed an independent test of the Forcepoint NGFW 3301 Appliance v6.1.2. The product was
subjected to thorough testing at the NSS facility in Austin, Texas, based on the Next Generation Firewall (NGFW)
Test Methodology v7.0, which is available at www.nsslabs.com. This test was conducted free of charge and NSS did
not receive any compensation in return for Forcepoint’s participation.

While the companion Comparative Reports on security, performance, and total cost of ownership (TCO) will
provide information about all tested products, this Test Report provides detailed information not available
elsewhere.

NSS research indicates that NGFW devices are typically deployed to protect users rather than data center assets,
and that the majority of enterprises will not separately tune intrusion prevention system (IPS) modules within their
NGFWs. Therefore, during NSS testing, NGFW products are configured with the vendor’s pre-defined or
recommended (i.e., “out-of-the-box”) settings in order to provide readers with relevant security effectiveness and
performance dimensions based on their expected usage.

NSS-Tested
Product Exploit Block Rate1 3-Year TCO (US$)
Throughput
99.95% 9,952 Mbps $76,159
Forcepoint NGFW 3301 Appliance Firewall Policy Application Evasions Stability and
v6.1.2 Enforcement Control Blocked Reliability
PASS PASS 137/1372 PASS

Figure 1 – Overall Test Results

Using the recommended policy, the NGFW 3301 Appliance blocked 99.95% of attacks. The device proved effective
against all evasion techniques tested. The device also passed all stability and reliability tests

The Forcepoint 3301 Appliance is rated by NSS at 9,952 Mbps, which is higher than the vendor-claimed
performance; Forcepoint rates this device at 9 Gbps. NSS-Tested Throughput is calculated as an average of all the
“real-world” protocol mixes and the 21 KB HTTP response-based capacity test.

1 Exploit block rate is defined as the number of live exploits (CAWS) and exploits from the NSS Exploit Library blocked under test.
2In accordance with the industry standard for vulnerability disclosures and to provide vendors with sufficient time to add protection where
necessary, NSS Labs will not publicly release information about which previously untested evasion techniques were applied during testing until
90 days after the publication of this document.

This report is Confidential, for internal use only, and is expressly limited to NSS Labs’ licensed users. 2

...............................................................15 Real-World Traffic Mixes ...............................................15 HTTPS Capacity with HTTPS Persistent Connections ..........................................16 Stability and Reliability ............................................................................... and is expressly limited to NSS Labs’ licensed users.................................................................................................................................................................9 Coverage by Target Vendor ..................................................................................... 11 Raw Packet Processing Performance (UDP Throughput) ....................11 Raw Packet Processing Performance (UDP Latency) ..........................................12 Maximum Capacity ........................................................6 NSS Exploit Library ..........................................................................................................................................14 Application Average Response Time – HTTP ................................................................................................................................................................................................................................................................................................................17 Total Cost of Ownership (TCO) .................................................................................................................................................................................................................................................................................................................................................................................................8 Coverage by Date............................................................................................................................................................................................... 3 .........................................................................................................................12 HTTP Capacity ..................................................................18 Total Cost of Ownership .......................................................................... 26 Contact Information ......... 2 Security Effectiveness ........................ 26 This report is Confidential..................................................................................................................................................8 Coverage by Impact Type...............................................................................................................................................................................................19 Appendix A: Product Scorecard ................................................................................. 5 Firewall Policy Enforcement ........................ 20 Test Methodology .........................................7 False Positive Testing ............................2_060617 Table of Contents Overview...........................................................................................................................................................................................................................................................................................................................................................5 Application Control ........10 Performance ......................................................................6 CAWS (Live Exploits) ...........................................................................................................7 Coverage by Attack Vector ....................................................................................................................................................................................................................................................................................................................................9 Resistance to Evasion Techniques .................... for internal use only.......................................................................................................1................................................................................................................. 18 Installation Hours .........NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.....................14 HTTP Capacity with HTTP Persistent Connections .....................................

..................................................................................................................................... and is expressly limited to NSS Labs’ licensed users...........13 Figure 13 – HTTP Capacity .......9 Figure 8 – Product Coverage by Target Vendor .....................................................................................................................16 Figure 18 – Stability and Reliability Results .............................................9 Figure 9 – Resistance to Evasion Results ........................................................................................................................................................................................................11 Figure 11 – UDP Latency in Microseconds ................19 Figure 21 – Detailed Scorecard ..........................................................................6 Figure 4 – Number of Threat Encounters Blocked (%) ....................................................................................................................................................................................................................................7 Figure 6 – Coverage by Attack Vector ................................25 This report is Confidential..............................................................17 Figure 19 – Sensor Installation Time (Hours) ..........................................................................................14 Figure 14 – Average Application Response Time (Milliseconds) ......................................................15 Figure 16 – HTTPS Capacity with HTTPS Persistent Connections ...................................................................................................................................................................5 Figure 3 – Application Control ......................NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6...........................15 Figure 17 – “Real-World” Traffic Mixes ...........................................................................................8 Figure 7 – Product Coverage by Date ..........................................................................................................................................................................10 Figure 10 – Raw Packet Processing Performance (UDP Traffic) ........2_060617 Table of Figures Figure 1 – Overall Test Results.......................................................................................................................................................12 Figure 12 – Concurrency and Connection Rates ..................................1..........................................................................................................6 Figure 5 – Number of Exploits Blocked (%) .............................14 Figure 15 – HTTP Capacity with HTTP Persistent Connections ..........................................................................................................18 Figure 20 –3-Year TCO (US$) .............................................................................................................2 Figure 2 – Firewall Policy Enforcement ..................................................................................... for internal use only............................................................................................. 4 ......................................................................................................................

and service.2_060617 Security Effectiveness This section verifies that the device is capable of enforcing the security policy effectively. An example of an untrusted network would be the Internet. based on identifying criteria such as source.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. destination.  DMZ – This is a network that is being isolated by the firewall. a network that is considered secure and protected. Firewall Policy Enforcement Policies are rules that are configured on a firewall to permit or deny access from one network resource to another. restricting network traffic to and from hosts contained within the isolated network. Test Procedure Result Baseline Policy PASS Simple Policy PASS Complex Policy PASS Static NAT PASS Dynamic/Hide NAT PASS SYN Flood Protection PASS IP Address Spoofing Protection PASS TCP Split Handshake Spoof PASS Figure 2 – Firewall Policy Enforcement This report is Confidential.e. Policies are typically written to permit or deny network traffic from one or more of the following zones:  Untrusted – This is typically an external network and is considered to be unknown and not secure.  Trusted – This is typically an internal network. 5 . and is expressly limited to NSS Labs’ licensed users. for internal use only. The NSS firewall tests verify performance and the ability to enforce policy between the following:  Trusted to Untrusted  Untrusted to DMZ  Trusted to DMZ Note: Firewalls must provide at least one DMZ interface in order to provide a DMZ or “transition point” between untrusted and trusted networks.. A term typically used to define the demarcation point of a network where policy is applied is demilitarized zone (DMZ). i.1.

NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.2 Figure 4 – Number of Threat Encounters Blocked (%) 3 See the NSS Cyber Advanced Warning System™ for more details.1. and applications. as it enables the administrator to define security policies based on both applications and ports. granular application control is a requirement of an NGFW. This capability is needed to re- establish a secure perimeter where unwanted applications are unable to tunnel over HTTP/S. CAWS (Live Exploits) Total Number Block Product Threat Encounters Blocked Percentage Forcepoint NGFW 3301 Appliance 4534 4529 99. “drive-by” attacks first exploit a vulnerable application then silently download and install malware. This report is Confidential. objects. For more information. As such. can be effectively measured in NSS’ unique live test harness through a series of procedures that measure the stages of protection. see the Comparative Report on Security – CAWS (Live Exploits). CAWS (Live Exploits) This test uses NSS’ Cyber Advanced Warning System (CAWS) to determine how effectively products are able to block exploits that are being used in active attack campaigns.89% v6. for internal use only.2_060617 Application Control An NGFW must provide granular control based on applications as well as ports. Unlike traditional malware that is downloaded and installed. 6 . 3 Protection from web-based exploits targeting client applications.1. NSS engineers verified that the device successfully determined the correct application and took the appropriate action based on the policy. and is expressly limited to NSS Labs’ licensed users. Test Procedure Result Block Unwanted Applications PASS Block Specific Actions PASS Figure 3 – Application Control Our testing found that the NGFW 3301 Appliance correctly enforced complex outbound and inbound policies consisting of multiple rules. also known as “drive-by” downloads.

1. With 2. including NSS’ network live stack test environment 4 as appropriate.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.097 exploits.0% v6. this is the industry’s most comprehensive test to date. open-source.097 2. for internal use only. 7 . Most notably.2 Figure 5 – Number of Exploits Blocked (%) False Positive Testing The Forcepoint 3301 Appliance correctly identified traffic and did not fire alerts for non-malicious content.2_060617 NSS Exploit Library NSS’ security effectiveness testing leverages the deep expertise of our engineers who utilize multiple commercial. 4 See the NSS Cyber Advanced Warning System™ for more details.1. This report is Confidential. and is expressly limited to NSS Labs’ licensed users. and proprietary tools. allowing the attacker to execute arbitrary commands ● Arbitrary code is executed ● A malicious payload is installed ● A system is rendered unresponsive ● Etc. all of the exploits and payloads in this test have been validated such that: ● A reverse shell is returned ● A bind shell is opened on the target.097 100. Total Number of Product Total Number Blocked Block Percentage Exploits Run Forcepoint NGFW 3301 Appliance 2.

100% 1150 90% Exploits Attempted/Caught 80% 1100 70% 1050 Block Rate 60% 50% 1000 40% 30% 950 20% 900 10% 0% 850 Attacker Initiated Target Initiated Attempted 974 1123 Caught 974 1123 Coverage 100. but not arbitrary system-level command execution. Attacker-initiated exploits are threats executed remotely against a vulnerable application and/or operating system by an individual. providing the attacker with the ability to execute arbitrary system-level commands. there are attacks that result in a system.0% 100. Clients can contact NSS for more information about these tests. Exploits can be categorized as either attacker- initiated or target-initiated. and the attacker has little or no control as to when the threat is executed. NGFWs should be evaluated against a broad set of exploits. for internal use only. Slightly less serious are attacks that result in an individual service compromise. Most exploits in this class are “weaponized” and offer the attacker a fully interactive remote shell on the target client or server.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. Target-initiated exploits are the most common type of attack experienced by the end user.00% Figure 6 – Coverage by Attack Vector Coverage by Impact Type The most serious exploits are those that result in a remote system compromise. 8 . This report is Confidential. and is expressly limited to NSS Labs’ licensed users.1. while target-initiated exploits are initiated by the vulnerable target.or service-level fault that crashes the targeted service or application and requires administrative action to restart the service or reboot the system.2_060617 Coverage by Attack Vector Because a failure to block attacks could result in significant compromise and could severely impact critical business systems. Finally.

0% 100. and is expressly limited to NSS Labs’ licensed users.0% 100.2_060617 Coverage by Date Figure 7 provides insight into whether or not a vendor is aging out protection signatures aggressively enough to preserve performance levels.0% 100.0% 100.0% 100. Figure 8 depicts the coverage offered by the NGFW 3301 Appliance for five of the top vendors targeted in this test. 100. More than 70 vendors are represented in the test.0% 100.0% 100.0% 40% 30% 20% 10% 0% <=2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Caught % Missed % Figure 7 – Product Coverage by Date Coverage by Target Vendor Exploits within the NSS Exploit Library target a wide range of protocols and applications.1. 100% 90% 80% 70% 60% 50% 100. It also reveals whether a product lags behind in protection for the most current vulnerabilities.0% 100.0% 100.0% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Adobe Apple IBM Microsoft Oracle Figure 8 – Product Coverage by Target Vendor This report is Confidential. for internal use only.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.0% 100. Exploits older than ten years are grouped together.0% 100. 9 .0% 100.0% 100. NSS reports exploits by individual years for the past ten years. Clients can contact NSS for more information about this test.0% 100.0% 100.0% 100.

2_060617 Resistance to Evasion Techniques Evasion techniques are a means of disguising and modifying attacks at the point of delivery to avoid detection and blocking by security products. Test Procedure Result IP Packet Fragmentation PASS TCP Stream Segmentation PASS RPC Fragmentation PASS URL Obfuscation PASS HTML Obfuscation PASS HTTP Compression PASS FTP/Telnet Evasion PASS Payload Padding PASS IP Packet Fragmentation + TCP Segmentation PASS HTTP Evasion PASS Figure 9 – Resistance to Evasion Results This report is Confidential. For further detail. for example. 10 . HTML obfuscation. Many of the techniques used in this test have been widely known for years and should be considered minimum requirements for the NGFW product category.) Lower-level evasions will potentially impact a wider number of exploits. stream segmentation. The NGFW 3301 Appliance blocked all 137 evasions it was tested against. Figure 9 provides the results of the evasion tests for the NGFW 3301 Appliance. Providing exploit protection results without fully factoring in evasion can be misleading. Furthermore. and is expressly limited to NSS Labs’ licensed users.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. RPC fragmentation. For example. the less effective the device. Failure of a security device to correctly identify a specific type of evasion potentially allows an attacker to use an entire class of exploits for which the device is assumed to have protection. than one technique in each category. please reference Appendix A. it is better to miss all techniques in one evasion category. missing TCP segmentation. URL obfuscation. IP packet fragmentation. which would result in a broader attack surface. and FTP evasion). This renders the device virtually useless. such as FTP evasion. is a much more serious issue than missing FTP obfuscation. The more classes of evasion that are missed (such as HTTP evasion. evasions operating at the lower layers of the network stack (IP packet fragmentation or stream segmentation) have a greater impact on security effectiveness than those operating at the upper layers (HTTP or FTP obfuscation. for internal use only.1.

2_060617 Performance There is frequently a trade-off between security effectiveness and performance.635 50 0 - 64 Byte 128 Byte 256 Byte 512 Byte 1024 Byte 1514 Byte Packets Packets Packets Packets Packets Packets Mbps 3.000 250 186 200 Megabits per Second Latency (μs) 15.350 22.860 100 9.350 10.860 21. No TCP sessions are created during this test.1. it is important to judge a product’s security effectiveness within the context of its performance and vice versa.635 7.169 14. with variable source and destination IP addresses transmitting from a fixed source port to a fixed destination port. is transmitted bidirectionally through each port pair of the device. Raw Packet Processing Performance (UDP Throughput) This test uses UDP packets of varying sizes generated by test equipment. and there is very little for the state engine to do. 25. This ensures that new security protections do not adversely impact performance and that security shortcuts are not taken to maintain or improve performance.127 5.000 3. in order to provide the highest level of network performance with the least amount of latency.960 Latency (μs) 121 123 126 186 223 252 Figure 10 – Raw Packet Processing Performance (UDP Traffic) This report is Confidential. and to determine the device’s effectiveness at forwarding packets quickly.000 300 252 223 20. This traffic does not attempt to simulate any form of a “real-world” network condition.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.960 150 21. and is expressly limited to NSS Labs’ licensed users. A constant stream of the appropriate packet size.127 14. The aim of this test is to determine the raw packet processing capability of each inline port pair of the device.169 9. Each packet contains dummy data and is targeted at a valid port on a valid IP address on the target subnet. Because of this trade-off.000 7. Multiple tests are run and averages are taken where necessary. for internal use only.000 121 123 126 22. The percentage load and frames per second (fps) figures across each inline port pair are verified by network monitoring tools before each test begins. 11 .

00 256-Byte Packets 126. and these tests provide an excellent representation of a live network at various connection/transaction rates.2_060617 Raw Packet Processing Performance (UDP Latency) NGFWs that introduce high levels of latency lead to unacceptable response times for users. ● Unsuccessful HTTP transactions – Normally. especially where multiple security devices are placed in the data path.00 512-Byte Packets 186. it is an indication that excessive latency within the NGFW is causing connections to time out. This report is Confidential. 12 . Latency – UDP Microseconds 64-Byte Packets 121. Figure 11 depicts UDP latency (in microseconds) as recorded during the UDP throughput tests at 90% of maximum load.00 128-Byte Packets 123. Once these appear. there should be zero unsuccessful transactions. The aim of these tests is to stress the inspection engine and determine how it copes with high volumes of TCP connections per second. and concurrent open connections. for internal use only. application layer transactions per second. ● Excessive concurrent HTTP connections – Latency within the NGFW is causing excessive delays and increased response time.1. and is expressly limited to NSS Labs’ licensed users.00 1024-Byte Packets 223. Note that in all tests the following critical “breaking points”—where the final measurements are taken—are used: ● Excessive concurrent TCP connections – Latency within the NGFW is causing an unacceptable increase in open connections.00 1514-Byte Packets 252.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.00 Figure 11 – UDP Latency in Microseconds Maximum Capacity The use of traffic generation appliances allows NSS engineers to create “real-world” traffic at multi-Gigabit speeds as a background load for the tests. All packets contain valid payload and address data.

358.000 12.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.000.000 120.100 Figure 12 – Concurrency and Connection Rates This report is Confidential.000 0 0 without data with data Maximum TCP Connections per Second 139.000.000 14.000.1.000. and is expressly limited to NSS Labs’ licensed users.000 2. for internal use only.900 Maximum HTTP Connections per Second 75.000 8.000.358.000 10.000.000 16.000.000 20.000 6.000 3.000 160.000 80.000.700 Maximum HTTP Transactions per Second 130.000 140. 13 .2_060617 20.000.000 4.100 40.120.665 3.000 Concurrent Connections Connections per Second 100.000 60. Concurrent TCP Connections 18.470 Max.665 18.000 18.120.000.

482 60. the device is forced to track valid TCP sessions.760 2.000 6.410 68.000 10.000 Connections per Second – 4.38 Figure 14 – Average Application Response Time (Milliseconds) This report is Confidential. while ensuring absolute accuracy and repeatability.1.89 5.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.000 1. and is expressly limited to NSS Labs’ licensed users. Each transaction consists of a single HTTP GET request.75 10.000 Connections per Second – 21 KB Response 2.482 6.7 KB Response Response Response Response Response CPS 34. thus ensuring a higher workload than for simple packet-based background traffic.900 52.000 Connections per Second 70.000 3.760 Figure 13 – HTTP Capacity Application Average Response Time – HTTP Application Average Response Time – HTTP (at 90% Maximum Load) Milliseconds 2.000 4. 16.000 13.820 79.000 40.40 40. for internal use only.960 10.000 Megabits per Second 12.5 KB Response 1.000 0 0 44 KB 21 KB 10 KB 4. This provides a test environment that is as close to “real-world” conditions as possible.882 3.960 14.000 10.000 80. All packets contain valid payload (a mix of binary and ASCII objects) and address data. 14 . By creating genuine session- based traffic with varying session lengths.500 Connections per Second – 44 KB Response 3.2_060617 HTTP Capacity The aim of the HTTP capacity tests is to stress the HTTP detection engine and determine how the device copes with network loads of varying average packet size and varying connections per second.000 Connections per Second – 1.000 6.7 KB Response 1.14 20.000 20.000 90. This test provides an excellent representation of a live network (albeit one biased toward HTTP traffic) at various network loads.5 KB 1.400 Mbps 13.981 30.000 Connections per Second – 10 KB Response 2.000 10.000 8.610 70.981 1.882 50.

672 4.2_060617 HTTP Capacity with HTTP Persistent Connections This test will use HTTP persistent connections.672 2.000 2.1.437 9. with each TCP connection containing 10 HTTP GETs and associated responses.000 5.874 7.198 Figure 15 – HTTP Capacity with HTTP Persistent Connections HTTPS Capacity with HTTPS Persistent Connections This test will use HTTPS persistent connections.368 6.284 Mbps 5.500 5.000 0 0 HTTPS 250 CPS HTTPS 500 CPS HTTPS 1000 CPS CPS 1.198 10.500 3.000 1. 15 .472 18.874 9.000 5.284 Figure 16 – HTTPS Capacity with HTTPS Persistent Connections This report is Confidential.418 2.000 10.000 2. All packets contain valid payload (a mix of binary and ASCII objects) and address data.000 2.068 2.000 500 1.472 12.000 6. and this test provides an excellent representation of a live network at various network loads. 6.000 4.000 Connections per Second 4.034 2.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.000 2. The stated response size is the total of all HTTP responses within a single TCP session.000 9.000 16.000 Connections per Second Megabits per Second 14. for internal use only.000 9.198 Mbps 17.068 Megabits per Second 4.000 8.000 6. 20.000 12.000 17. and is expressly limited to NSS Labs’ licensed users.000 3.284 1.000 4.000 2.000 8.000 1.000 0 0 HTTP 250 CPS HTTP 500 CPS HTTP 1000 CPS CPS 4. with each TCP connection containing 10 HTTPS GETs and associated responses.000 12.

000 6. for internal use only.000 5.2_060617 Real-World Traffic Mixes This test measures the performance of the device in a “real-world” environment by introducing additional protocols and real content. while still maintaining a precisely repeatable and consistent background traffic load.965 4.nsslabs. 18.390 16.280 16.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. and is expressly limited to NSS Labs’ licensed users.1.203 5.280 6.000 0 Real World Real-World Real-World Real World Real World Protocol Mix Protocol Mix Protocol Mix Protocol Mix Protocol Mix (US (Enterprise (European (Internal (Financial) Mobile Carrier) Perimeter) Mobile Carrier) Segmentation) Mbps 14.com. available at www.392 6.203 6. European Mobile Carrier mix. and the Internal Segmentation mix. Different protocol mixes are utilized based on the intended location of the device (network core or perimeter) to reflect real use cases.000 Mbps 8.000 12.000 2.392 Figure 17 – “Real-World” Traffic Mixes The NGFW 3301 Appliance was tested by NSS and performed above the throughput claimed by the vendor for the Enterprise Perimeter and Financial Protocol “real-world” traffic mixes and below vendor-claimed throughput for the US Mobile Carrier mix.390 14.000 14. see the NSS Labs Next Generation Firewall Test Methodology.000 10. For details about real-world traffic protocol types and percentages.965 6. This report is Confidential.000 16. 16 .

If any non-allowed traffic passes successfully. and is expressly limited to NSS Labs’ licensed users. raising an alert for each. it will fail the test.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. All NGFW devices must choose whether to risk denying legitimate traffic or risk allowing malicious traffic once they run low on resources. or when traffic loads exceed its capacity. and to block 100% of previously blocked traffic. Stability and Reliability Result Blocking under Extended Attack PASS Passing Legitimate Traffic under Extended Attack PASS Behavior of the State Engine under Load  Attack Detection/Blocking – Normal Load PASS  State Preservation – Normal Load PASS  Pass Legitimate Traffic – Normal Load PASS  State Preservation – Maximum Exceeded PASS  Drop Traffic – Maximum Exceeded PASS Protocol Fuzzing and Mutation PASS Power Fail PASS Persistence of Data PASS Figure 18 – Stability and Reliability Results These tests also determine the behavior of the state engine under load. These tests verify the stability of the device along with its ability to maintain security effectiveness while under normal load and while passing malicious traffic. for internal use only.2_060617 Stability and Reliability Long-term stability is particularly important for an inline device. This report is Confidential. caused either by the volume of traffic or by the device failing open for any reason. Products that cannot sustain legitimate traffic (or that crash) while under hostile attack will not pass. where failure can produce network outages. In theory. this means the NGFW will block legitimate traffic but maintain state on existing connections (and prevent attack leakage).1. 17 . An NGFW device will drop new connections when resources (such as state table memory) are low. The device is required to remain operational and stable throughout these tests.

including software and hardware support.1. and so on. Each of the following should be considered over the course of the useful life of the solution: ● Product Purchase – The cost of acquisition. software. and set up desired logging and reporting. policy updates. ● Management – Day-to-day management tasks. ● Installation – The time required to take the device out of the box. policy deployment. put it into the network. ● Upkeep – The time required to apply periodic updates and patches from vendors. and other updates. The table accurately reflects the amount of time that NSS engineers. This closely mimics a typical enterprise deployment scenario for a single device. For the purposes of this report. for internal use only. capital expenditure (capex) items are included for a single device only (the cost of acquisition and installation). passed legitimate traffic. The installation cost is based on the time that an experienced security engineer would require to perform the installation tasks described above.1. Product Installation (Hours) Forcepoint NGFW 3301 Appliance 8 v6. including hardware. maintenance.2_060617 Total Cost of Ownership (TCO) Implementation of security solutions can be complex. including device configuration. ● Product Maintenance – The fees paid to the vendor. configure it.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. and blocked and detected prohibited or malicious traffic. needed to install and configure the device to the point where it operated successfully in the test harness. 18 . and is expressly limited to NSS Labs’ licensed users. with the help of vendor engineers. and upkeep. This approach allows NSS to hold constant the talent cost and measure only the difference in time required for installation. alert handling. Installation Hours This table depicts the number of hours of labor required to install each device using only local device management options.2 Figure 19 – Sensor Installation Time (Hours) This report is Confidential. maintenance. Readers should substitute their own costs to obtain accurate TCO figures. and other updates. with several factors affecting the overall cost of deployment. apply updates and patches.

costs for central management solutions (CMS) may be extra.109 $10.150 $55.150 $76. Purchase Maintenance/ Year 1 Year 2 Year 3 3-Year Product Price Year Cost Cost Cost TCO Forcepoint NGFW 3301 Appliance $45. ● Year 2 Cost consists only of maintenance/support fees. updates for the first year are included in the initial purchase price and are not counted again in Year 1 Cost.859 $10. the 24/7 maintenance and support option with 24-hour replacement is utilized.150 $10.2 Figure 20 –3-Year TCO (US$) For the Forcepoint NGFW 3301 Appliance.1. ● Year 1 Cost is calculated by adding installation costs (US$75 per hour fully loaded labor x installation time) + purchase price + first-year maintenance/support fees. Where possible. and is expressly limited to NSS Labs’ licensed users. 19 . For additional TCO analysis.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. ● Year 3 Cost consists only of maintenance/support fees. Prices are for single device management and maintenance only.159 v6. since this is the option typically selected by enterprise customers.2_060617 Total Cost of Ownership Calculations are based on vendor-provided pricing information. refer to the TCO Comparative Report. for internal use only. This report is Confidential. including for the CMS.1.

duplicate last packet PASS Ordered 8-byte fragments.00% Combined Total 100.00% Coverage by Impact Type System Exposure Contact NSS Service Exposure Contact NSS System or Service Fault Contact NSS Coverage by Date Contact NSS Coverage by Target Vendor Contact NSS Coverage by Result Contact NSS Coverage by Target Type Contact NSS Evasions and Attack Leakage Resistance to Evasion PASS IP Packet Fragmentation PASS Ordered 8-byte fragments PASS Ordered 16-byte fragments PASS Ordered 24-byte fragments PASS Ordered 32-byte fragments PASS Out of order 8-byte fragments PASS Ordered 8-byte fragments.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. interleaved duplicate packets scheduled for later delivery PASS Ordered 8-byte fragments.89% NSS Exploit Library Block Rate 100. duplicate last packet PASS Out of order 8-byte fragments. This report is Confidential.95% CAWS (Live Exploits) Block Rate 99. for internal use only. duplicate packet with an incrementing DWORD in the options field.00% Target-Initiated 100. fragment overlap (favor old) PASS Out of order 8-byte fragments.2_060617 Appendix A: Product Scorecard Description Result Security Effectiveness Firewall Policy Enforcement PASS Baseline Policy PASS Simple Policy PASS Complex Policy PASS Static NAT PASS Dynamic / Hide NAT PASS SYN Flood Protection PASS Address Spoofing Protection PASS TCP Split Handshake PASS Application Control PASS Block Unwanted Applications PASS Block Specific Action PASS Intrusion Prevention False Positive Testing PASS Exploit Block Rate 99. fragment overlap (favor new) PASS Ordered 16-byte fragments. The PASS duplicate packet has random payload. reorder fragments in reverse PASS Ordered 16-byte fragments. 20 .00% Coverage by Attack Vector (NSS Exploit Library) Attacker-Initiated 100.1. and is expressly limited to NSS Labs’ licensed users.

segment overlap (favor new) with random data PASS Reverse Ordered 1024-byte segments. interleaved duplicate segments with null TCP control flags PASS Ordered 1-byte segments. segment overlap (favor new) PASS Ordered 1-byte segments. including Last Fragment (LF) will be sent in one TCP segment (ONC) PASS All frags except Last Fragment (LF) will be sent in one TCP segment. interleaved duplicate segments with requests to resync sequence numbers PASS mid-stream Ordered 1-byte segments. and is expressly limited to NSS Labs’ licensed users. duplicate last packet PASS Ordered 2-byte segments. TCP Stream Segmentation PASS Ordered 1-byte segments. segment overlap (favor new) PASS Out of order 1-byte segments.2_060617 Ordered 16-byte fragments. LF will be sent in separate TCP seg PASS (ONC) One RPC fragment will be sent per TCP segment (ONC) PASS One LF split over more than one TCP segment. Ordered 32-byte fragments. The PASS duplicate packet has random payload. segment overlap (favor new (Unix)) PASS Ordered 32-byte segments PASS Ordered 64-byte segments PASS Ordered 128-byte segments PASS Ordered 256-byte segments PASS Ordered 512-byte segments PASS Ordered 1024-byte segments PASS Ordered 2048-byte segments (sending MSRPC request with exploit) PASS Reverse Ordered 256-byte segments. Initial TCP sequence PASS number is set to 0xffffffff . duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new) with random data PASS Out of order 1024-byte segments. duplicate packet with an incrementing DWORD in the options field. Initial TCP sequence PASS number is set to 0xffffffff . interleaved duplicate segments with faked retransmits PASS Ordered 1-byte segments. segment overlap (favor new) with random data. The PASS duplicate packet has random payload.4294967295 Out of order 2048-byte segments. segment overlap (favor new) with random data. interleaved duplicate segments with invalid TCP checksums PASS Ordered 1-byte segments. segment overlap (favor new) with random data PASS Reverse Ordered 2048-byte segments. PAWS elimination (interleaved duplicate segments with older TCP PASS timestamp options) Ordered 16-byte segments.1. segment overlap (favor new) with random data PASS Reverse Ordered 512-byte segments. 21 .4294967295 RPC Fragmentation PASS One-byte fragmentation (ONC) PASS Two-byte fragmentation (ONC) PASS All fragments. In this case no RPC fragmentation is performed (ONC) PASS Canvas Reference Implementation Level 1 (MS) PASS Canvas Reference Implementation Level 2 (MS) PASS Canvas Reference Implementation Level 3 (MS) PASS Canvas Reference Implementation Level 4 (MS) PASS Canvas Reference Implementation Level 5 (MS) PASS Canvas Reference Implementation Level 6 (MS) PASS Canvas Reference Implementation Level 7 (MS) PASS Canvas Reference Implementation Level 8 (MS) PASS Canvas Reference Implementation Level 9 (MS) PASS This report is Confidential. for internal use only. Ordered 24-byte fragments.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. interleaved duplicate segments with out-of-window sequence numbers PASS Out of order 1-byte segments PASS Out of order 1-byte segments. The PASS duplicate packet has random payload. duplicate packet with an incrementing DWORD in the options field.

for internal use only. 22 .2_060617 Canvas Reference Implementation Level 10 (MS) PASS URL Obfuscation PASS URL encoding – Level 1 (minimal) PASS URL encoding – Level 2 PASS URL encoding – Level 3 PASS URL encoding – Level 4 PASS URL encoding – Level 5 PASS URL encoding – Level 6 PASS URL encoding – Level 7 PASS URL encoding – Level 8 (extreme) PASS Directory Insertion PASS Premature URL ending PASS Long URL PASS Fake parameter PASS TAB separation PASS Case sensitivity PASS Windows \ delimiter PASS Session splicing PASS HTML Obfuscation PASS UTF-16 character set encoding (big-endian) PASS UTF-16 character set encoding (little-endian) PASS UTF-32 character set encoding (big-endian) PASS UTF-32 character set encoding (little-endian) PASS UTF-7 character set encoding PASS Chunked encoding (random chunk size) PASS Chunked encoding (fixed chunk size) PASS Chunked encoding (chaffing) PASS Compression (Deflate) PASS Compression (Gzip) PASS Base-64 Encoding PASS Base-64 Encoding (shifting 1 bit) PASS Base-64 Encoding (shifting 2 bits) PASS Base-64 Encoding (chaffing) PASS Combination UTF-7 + Gzip PASS HTTP Compression PASS FTP Evasion / Telnet Evasion PASS Inserting spaces in FTP command lines PASS Inserting non-text Telnet opcodes – Level 1 (minimal) PASS Inserting non-text Telnet opcodes – Level 2 PASS Inserting non-text Telnet opcodes – Level 3 PASS Inserting non-text Telnet opcodes – Level 4 PASS Inserting non-text Telnet opcodes – Level 5 PASS Inserting non-text Telnet opcodes – Level 6 PASS Inserting non-text Telnet opcodes – Level 7 PASS Inserting non-text Telnet opcodes – Level 8 (extreme) PASS Payload Padding PASS Layered Evasions PASS IP Fragmentation + TCP Segmentation PASS Ordered 8-byte fragments + Ordered TCP segments except that the last segment comes first PASS Ordered 24-byte fragments + Ordered TCP segments except that the last segment comes first PASS This report is Confidential.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. and is expressly limited to NSS Labs’ licensed users.1.

duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). The duplicate packet has random payload + Out of order TCP segments. The duplicate packet has random payload + Out of order TCP segments. The duplicate packet has random payload + Out of order TCP segments. The duplicate packet has random payload + Out of order TCP segments.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. duplicate packet with an incrementing DWORD in the options field.1. duplicate packet with an incrementing DWORD in the options field. The duplicate packet has random payload + Out of order TCP segments. duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). PASS Overlapping data is set to random bytes HTTP Evasion PASS Test Case 1 PASS Test Case 2 PASS Test Case 3 PASS Test Case 4 PASS Test Case 5 PASS Test Case 6 PASS Test Case 7 PASS Test Case 8 PASS Test Case 9 PASS Test Case 10 PASS Test Case 11 PASS Test Case 12 PASS Test Case 13 PASS Test Case 14 PASS Test Case 15 PASS Test Case 16 PASS Test Case 17 PASS Test Case 18 PASS Test Case 19 PASS Test Case 20 PASS This report is Confidential. PASS Overlapping data is set to zero bytes Ordered 32-byte fragments. segment overlap (favor new). PASS Overlapping data is set to random alphanumeric Ordered 32-byte fragments. The duplicate packet has random payload + Out of order TCP segments. duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). duplicate packet with an incrementing DWORD in the options field. duplicate packet with an incrementing DWORD in the options field. PASS Overlapping data is set to random bytes Ordered 24-byte fragments. PASS Overlapping data is set to random alphanumeric Ordered 16-byte fragments. PASS Overlapping data is set to random bytes Ordered 32-byte fragments. segment overlap (favor new). The duplicate packet has random payload + Out of order TCP segments. duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). segment overlap (favor new). The duplicate packet has random payload + Out of order TCP segments. duplicate packet with an incrementing DWORD in the options field. duplicate packet with an incrementing DWORD in the options field. segment overlap (favor new). and is expressly limited to NSS Labs’ licensed users. The duplicate packet has random payload + Reverse order TCP segments.2_060617 Ordered 32-byte fragments + Ordered TCP segments except that the last segment comes first PASS Ordered 8-byte fragments. PASS Overlapping data is set to zero bytes Ordered 8-byte fragments. PASS Overlapping data is set to zero bytes Ordered 24-byte fragments. segment overlap (favor new). PASS Overlapping data is set to random bytes Ordered 16-byte fragments. PASS Overlapping data is set to zero bytes Ordered 16-byte fragments. PASS Overlapping data is set to random alphanumeric Ordered 8-byte fragments. The duplicate packet has random payload + Out of order TCP segments. for internal use only. segment overlap (favor new). The duplicate packet has random payload + Out of order TCP segments. 23 .

7 KB Response 70.75 10.00 512-Byte Packets 186.860 1024-Byte Packets 21.470 Maximum HTTP Transactions per Second 130. and is expressly limited to NSS Labs’ licensed users. for internal use only.00 128-Byte Packets 123. Concurrent TCP Connections 18.635 128-Byte Packets 7.500 Connections per Second – 44 KB Response 3.034 1000 Connections per Second 2.000 Connections per Second – 21 KB Response 2.350 1514-Byte Packets 22.610 40.169 256-Byte Packets 9.358. 24 .960 Latency – UDP Microseconds 64-Byte Packets 121.38 HTTP Capacity with HTTP Persistent Connections CPS 250 Connections per Second 4.000 Connections per Second – 10 KB Response 2.820 20.00 Maximum Capacity CPS Theoretical Max.5 KB Response 79.00 256-Byte Packets 126.00 1514-Byte Packets 252.89 5.368 500 Connections per Second 6.700 Maximum HTTP Connections per Second 75.900 5.437 1000 Connections per Second 9.000 Connections per Second – 1.390 “Real-World” Protocol Mix (Financial) 16.100 Maximum TCP Connections per Second 139.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6.665 Theoretical Max.400 Application Average Response Time – HTTP (at 90% Max Load) Milliseconds 2.000 Connections per Second – 4.000 Connections per Second – 10 KB Response 68.410 10.198 HTTPS Capacity with HTTPS Persistent Connections CPS 250 Connections per Second 1.127 512-Byte Packets 14.1.284 “Real-World” Traffic Mbps “Real-World” Protocol Mix (Enterprise Perimeter) 14.00 1024-Byte Packets 223. Concurrent TCP Connections w/Data 3.2_060617 Test Case 21 PASS Test Case 22 PASS Test Case 23 PASS Test Case 24 PASS Test Case 25 PASS Test Case 26 PASS Performance Raw Packet Processing Performance (UDP Traffic) Mbps 64-Byte Packets 3.000 Connections per Second – 4.000 Connections per Second – 21 KB Response 52.900 HTTP Capacity CPS 2.5 KB Response 1.14 20.280 This report is Confidential.500 Connections per Second – 44 KB Response 34.7 KB Response 1.120.000 Connections per Second – 1.418 500 Connections per Second 2.40 40.

25 .150 Annual Cost of Updates (IPS/AV/etc.150 3-Year Total Cost of Ownership $76.159 Figure 21 – Detailed Scorecard This report is Confidential.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. for internal use only.965 “Real-World” Internal Segmentation Mix 6.) $0 Initial Purchase (enterprise management system) See Comparative Annual Cost of Maintenance and Support (enterprise management system) See Comparative Total Cost of Ownership Year 1 $55.109 Installation Labor Cost (@$75/hr) $600 Annual Cost of Maintenance and Support (hardware/software) $10.392 Stability and Reliability Blocking under Extended Attack PASS Passing Legitimate Traffic under Extended Attack PASS Behavior of the State Engine under Load Attack Detection/Blocking – Normal Load PASS State Preservation – Normal Load PASS Pass Legitimate Traffic – Normal Load PASS State Preservation – Maximum Exceeded PASS Drop Traffic – Maximum Exceeded PASS Protocol Fuzzing and Mutation PASS Power Fail PASS Persistence of Data PASS Total Cost of Ownership Ease of Use Initial Setup (Hours) 8 Time Required for Upkeep (Hours per Year) See Comparative Time Required to Tune (Hours per Year) See Comparative Expected Costs Initial Purchase (hardware as tested) $45.2_060617 “Real-World” Protocol Mix (US Mobile Carrier) 6.150 Year 3 $10.859 Year 2 $10.203 “Real-World” Protocol Mix (EU Mobile Carrier) 5. and is expressly limited to NSS Labs’ licensed users.1.

or guarantee of any of the products (hardware or software) tested or the hardware and/or software used in testing the products. 3. and trade names used in this report are the trademarks. recommendation. IN NO EVENT SHALL WE BE LIABLE FOR ANY DIRECT.nsslabs. Inc. and trade names of their respective owners. Contact Information NSS Labs. ALL IMPLIED WARRANTIES. copied/scanned. OR FOR ANY LOSS OF PROFIT. for internal use only. sponsorship.nsslabs. 4. Suite 200 Austin. service marks. TX 78746 USA info@nsslabs. NO WARRANTIES. “You” or “your” means the person who accesses this report and any entity on whose behalf he/she has obtained this report. 6. All use of and reliance on this report are at your sole risk. This report does not constitute an endorsement. This report is Confidential. ARE HEREBY DISCLAIMED AND EXCLUDED BY US. OR OTHER ASSETS. Inc.1.com. or verification by or with any organizations mentioned in this report. service marks. DATA.com www. 206 Wild Basin Road Building A. CONSEQUENTIAL. needs. INCIDENTAL. FITNESS FOR A PARTICULAR PURPOSE. but is not guaranteed. 5. 1. The information in this report is believed by us to be accurate and reliable at the time of publication. Inc. losses. EXEMPLARY. you should not read the rest of this report but should instead return the report immediately to us. AND NON-INFRINGEMENT. or specifications.nsslabs. We are not liable or responsible for any damages. requirements. and we disclaim any obligation to update it. affiliation. All trademarks. 2. The information in this report is subject to change by us without notice. Please read the disclaimer in this box because it contains important information that binds you. COMPUTER PROGRAMS. and is expressly limited to NSS Labs’ licensed users. 26 . e-mailed or otherwise disseminated or transmitted without the express written consent of NSS Labs. EVEN IF ADVISED OF THE POSSIBILITY THEREOF. To receive a licensed copy or report misuse. This report does not imply any endorsement.2_060617 Test Methodology Next Generation Firewall (NGFW) Test Methodology v7.0 A copy of the test methodology is available on the NSS Labs website at www. All rights reserved. No part of this publication may be reproduced. The testing does not guarantee that there are no errors or defects in the products or that the products will meet your expectations. PUNITIVE. INCLUDING IMPLIED WARRANTIES OF MERCHANTABILITY. stored on a retrieval system. or expenses of any nature whatsoever arising from any error or omission in this report. EXPRESS OR IMPLIED ARE GIVEN BY US. (“us” or “we”). or that they will operate without interruption.NSS Labs Next Generation Firewall Test Report – Forcepoint NGFW 3301 Appliance 6. © 2017 NSS Labs.com. If you do not agree to these conditions.com This and other related documents are available at www. REVENUE. OR INDIRECT DAMAGES. please contact NSS Labs.