Professional Documents
Culture Documents
COMPARATIVE
ANALYSIS
Security
Value
Map
(SVM)
2013
Frank
Artes,
Thomas
Skybakmoen,
Bob
Walder,
Vikram
Phatak,
Ryan
Liles
Tested
Products
Barracuda
F800,
Check
Point
12600,
Cyberoam
CR2500iNG,
Dell
SonicWALL
NSA
4500,
Fortinet
FortiGate
800c,
Juniper
SRX550,
NETASQ
ng1000-A,
NETGEAR
ProSecure
UTM9S,
Palo
Alto
Networks
PA-5020,
Sophos
UTM
425,
Stonesoft
StoneGate
FW-1301,
WatchGuard
XTM
1050
Overview
Empirical
data
from
the
individual
Product
Analysis
Reports
(PAR)
and
Comparative
Analysis
Reports
(CAR)
is
used
to
create
the
unique
Security
Value
Map
(SVM).
This
high-level
document
illustrates
clearly
the
relative
value
of
security
investment
options
by
mapping
security
effectiveness
and
value
(cost
per
protected
Mbps)
of
tested
product
configurations.
The
SVM
is
designed
to
provide
a
high-level
overview
of
the
detailed
findings
from
NSS
Labs
group
tests.
Having
examined
the
high-level
picture,
it
is
then
possible
to
dig
deeper
into
individual
products
and
capabilities
as
required
via
the
PAR
and
CAR
documents.
Individual
PARs
are
available
for
every
product
tested.
CARs
provide
detailed
comparisons
across
all
tested
products
in
the
areas
of:
Security
Performance
Management
Total
cost
of
ownership
(TCO)
NSS Labs
WatchGuard XTM
1050
80%
Barracuda F800
NETASQ NG1000-A
70%
60%
Cyberoam
CR2500iNG
50%
Sophos UTM 425
40%
30%
20%
Average
NETGEAR ProSecure
UTM9S
Juniper SRX550
10%
0%
$256
$128
$64
$32
$16
$8
$4
$2
$1
NSS Labs
Key Findings
Protection
varied
between
4%
and
100%,
with
half
of
the
tested
devices
achieving
greater
than
80%.
Price
per
Protected-Mbps
varied
from
$2
to
$994
with
most
tested
devices
costing
below
$12.41
per
Protected-Mbps.
Median
throughput
was
3.6
Gbps
with
a
spread
from
231
Mbps
to
9.7
Gbps.
NETGEAR
is
clearly
not
an
enterprise
product,
and
thus
was
excluded
from
calculations
of
average
to
prevent
excessive
skewing
of
the
results
to
the
overall
detriment
of
the
report.
Product
Guidance
NSS
Labs
recommendations
are
based
solely
on
empirical
test
data,
validated
over
multiple
iterations.
While
some
products
fall
within
Neutral
quadrants,
the
table
below
will
indicate
Caution
if
the
DUT
scored
below
90%
of
the
average
of
all
devices
tested
with
regard
to
Protection
and
Value.
The
overall
quadrant
score
may
remain
Neutral,
but
the
Protection
or
Value
will
be
flagged
appropriately.
Product
Protection
&
Management
Value
Overall
Barracuda F800
Neutral
Recommended
Neutral
Recommended Recommended
Recommended
Cisco Systems
Caution
Caution
Caution
Cyberoam CR2500iNG
Caution
Neutral
Neutral
Recommended Neutral
Neutral
Fortinet FortiGate-800c
Recommended Recommended
Recommended
Juniper SRX550
Recommended Recommended
Recommended
NETASQ NG1000-A
Caution
Caution
Caution
Caution
Caution
Caution
Recommended Neutral
Neutral
Caution
Caution
Stonesoft FW-1301
Recommended Recommended
Recommended
Recommended Neutral
Neutral
Caution
NSS Labs
NSS Labs
Table
of
Contents:
Overview
______________________________________________________________
1
Key
Findings
___________________________________________________________________
3
Product
Guidance
______________________________________________________________
3
Security
Effectiveness
&
Cost
_____________________________________________________
4
SVM
___________________________________________________________________
6
What
Do
The
Values
Mean?
______________________________________________________
6
How
To
Use
The
SVM
____________________________________________________________
7
Analysis
of
Data
_________________________________________________________
8
Recommended
_________________________________________________________________
8
Check
Point
12600
____________________________________________________________
8
Fortinet
FortiGate-800c
________________________________________________________
8
Juniper
SRX
550
______________________________________________________________
8
Stonesoft
StoneGate
FW-1301
___________________________________________________
9
Neutral
______________________________________________________________________
10
Barracuda
NG
Firewall
F800
___________________________________________________
10
Cyberoam
CR2500iNG
________________________________________________________
10
DELL
SonicWALL
NSA
4500
_____________________________________________________
11
Palo
Alto
Networks
PA-5020
___________________________________________________
11
WatchGuard
XTM
1050
_______________________________________________________
11
Caution
______________________________________________________________________
12
Cisco
Systems
_______________________________________________________________
12
NETASQ
NG1000
A
___________________________________________________________
12
NETGEAR
ProSecure
UTM9S
____________________________________________________
13
Sophos
UTM
425
____________________________________________________________
13
Test
Methodology
______________________________________________________
14
Contact
Information
_____________________________________________________
14
Table
of
Figures
Figure
1
-
2013
Firewall
Security
Value
Map
(SVM)
_______________________________________________
2
Figure
2
-
NSS
Labs'
2013
Firewall
Recommendations
_____________________________________________
3
Figure
3
-
Example
SVM
_____________________________________________________________________
6
NSS Labs
SVM
What
Do
The
Values
Mean?
The
SVM
depicts
the
value
of
a
deployment
of
ten
firewall
devices
and
the
appropriate
enterprise/central
management
console
for
each
vendor.
The
Management
Comparative
Analysis
Report
(CAR)
outlines
the
pricing
and
structure
needed
for
the
management
infrastructure
if
a
more
robust
setup
is
desired.
Additionally,
the
2013
Firewall
Management
CAR
outlines
multiple
cost-modeled
deployments
for
those
interested
in
distributed
management
deployment
scenarios.
Figure
3
-
Example
SVM
The
x-axis
charts
the
Total
Cost
of
Ownership
per
Protected
Mbps,
a
value
that
incorporates
the
3
year
TCO
with
measured
performance
to
provide
a
single
figure
that
can
be
used
to
compare
the
real
cost
of
each
device
tested.
Further
to
the
right
(lower
cost)
is
better.
The
y-axis
charts
the
enterprise
management
capabilities
and
security
effectiveness
as
measured
via
the
NSS
security
management
review
and
effectiveness
tests.
The
security
effectiveness
of
a
product
as
tested
is
multiplied
by
the
score
for
enterprise
management
as
tested.
Devices
that
are
missing
critical
security
OR
management
capabilities
will
have
a
reduced
score
on
this
axis.
Further
up
(higher
effectiveness)
is
better.
NSS Labs
NSS Labs
Analysis
of
Data
Recommended
Check
Point
12600
1
The
12600
was
rated
by
NSS
Labs
at
8.4
Gbps
out
of
the
10
Gbps
claimed
by
the
vendor.
The
12600
scored
100%
for
Stability,
100%
for
Evasion,
100%
for
Leakage,
and
100%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$13
per
protected
megabit,
and
100%
for
security
and
management
effectiveness.
Check
Points
management
system
is
flexible
and
granular,
allowing
for
a
high
degree
of
customization.
With
this
level
of
flexibility,
however,
comes
some
complexity.
For
current
or
experienced
enterprise
users
of
Smart-1
who
have
been
managing
NGFW
and
IPS
through
Check
Points
SmartDashboard,
there
will
not
be
a
significant
learning
curve.
New
administrators
should
take
their
time
learning
the
features
and
building
the
foundation
of
their
object
groups.
Check
Point
currently
only
offers
the
management
client
as
a
Windows
executable,
but
the
management
system,
overall,
is
the
most
mature
and
feature-complete
in
its
class.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Check
Point
12600
Product
Analysis
Report
(PAR).
Fortinet
FortiGate-800c
1
The
FortiGate-800c
was
rated
by
NSS
Labs
at
9.7
Gbps
out
of
the
20
Gbps
claimed
by
the
vendor.
The
800c
scored
100%
for
Stability,
100%
for
Evasion,
100%
for
Leakage,
and
100%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$4
per
protected
megabit,
and
100%
for
security
and
management
effectiveness.
Fortinets
management
interface
was
reasonably
well
designed,
although
the
organization
of
items
and
menus
proved
less
than
intuitive.
The
policy
is
based
on
a
Virtual
Domain
(VDOM)
organization,
grouping
policy
objects
based
on
their
area
of
effect,
which
may
create
confusion
for
administrators
that
are
not
familiar
with
this
method.
For
users
of
Fortinet
firewalls
or
IPS,
there
will
not
be
a
significant
learning
curve.
Tuning
and
maintenance
is
achieved
easily,
once
the
VDOM
organizational
method
is
understood.
Management
currently
does
not
support
event
correlation,
and
log
aggregation
requires
an
additional
purchase.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Fortinet
FortiGate-800C
Product
Analysis
Report
(PAR).
Juniper
SRX
550
1
The
Juniper
SRX550
12.1r2
was
rated
by
NSS
Labs
at
2.1
Gbps
out
of
the
5.5
Gbps
claimed
by
the
vendor.
The
SRX
5500
scored
100%
for
Stability,
100%
for
Evasion,
100%
for
Leakage,
and
100%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$20
per
protected
megabit,
and
100%
for
security
and
management
effectiveness.
NSS Labs
Junipers
Junos
Space
combined
with
the
Security
Threat
Response
Manager
(STRM)
provides
the
replacement
for
the
NSM/STRM
management
system.
Space
is
a
graphically
pleasing,
feature-rich
user
interface.
The
management
interface
is
easy
to
navigate
and
the
layout
is
reasonably
intuitive.
From
failsafe
features
like
rollback
policies
to
prescheduling
policy
and
rule
changes,
Space
is
straightforward
to
use.
There
is
a
lack
of
integration
between
Space
and
STRM
at
this
time,
but
Juniper
has
stated
it
is
working
to
resolve
this
issue.
Regardless,
the
system
has
very
robust
logging
and
audit
/
change
logs
that
are
easy
to
navigate
and
filter.
The
help
documentation
included
is
very
useful,
and
administrators
should
have
few
problems
learning
the
system.
Space
can
manage
as
few
as
25
devices
and
new
devices
can
be
added
via
100
devices
license
packs.
There
is
no
stated
limited
as
to
the
maximum
number
of
managed
devices
that
can
actively
be
connected
to
the
management
server
at
once,
but
Juniper
should
be
consulted
for
information
on
CPU,
RAM,
and
storage
requirements
as
the
managed
licenses
are
increased.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Juniper
SRX
550
Product
Analysis
Report
(PAR).
Stonesoft
StoneGate
FW-1301
1
The
StoneGate
FW-1302
was
rated
by
NSS
Labs
at
5.1
Gbps
out
of
the
7.5Gbps
claimed
by
the
vendor.
The
FW-
1301
scored
100%
for
Stability,
100%
for
Evasion,
100%
for
Leakage,
and
100%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$13
per
protected
megabit,
and
100%
for
security
and
management
effectiveness.
The
Stonesoft
Management
Center
has
been
designed
from
the
ground
up
as
a
flexible
and
powerful
large
enterprise
or
service
provider
management
system.
Administrator
access
is
via
extremely
granular
role-based
mechanisms.
Policy
management
and
deployment
is
straightforward
and
extremely
flexible,
with
grouping
and
inheritance
capabilities
providing
the
ability
to
deploy
complex
policies
across
multiple
devices
with
ease.
The
ability
to
deploy
sub-policies
for
individual
devices
beneath
a
hierarchy
of
global
policy
templates
makes
this
product
ideal
for
multi-tenanted
service
provider
environments.
Alert
handling
is
powerful,
with
multiple
means
of
achieving
the
same
end.
Flexible
real-time
filter
definition
provides
rapid
drill-down
to
pertinent
information
and
the
ability
to
save
ad
hoc
filters
for
reuse
later
is
very
useful.
Unique
to
Stonesofts
Management
Center
are
robust
investigation
and
forensic
capabilities.
The
only
drawback,
in
certain
environments,
is
the
lack
of
direct
device
management
capabilities.
All
Stonesoft
deployments
even
for
a
single
device
require
the
three-tier
management
system,
making
this
solution
less
cost
effective
for
some
SMB
environments.
Those
customers
who
don't
wish
to
take
on
enterprise
level
management
have
the
option
to
work
through
one
of
Stonesoft's
managed
security
service
providers
(MSSP).
However,
for
large-scale
enterprise
and
service
provider
environments,
the
Stonesoft
management
solution
is
well
suited.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Stonesoft
StoneGate
FW-1301
Product
Analysis
Report
(PAR).
NSS Labs
Neutral
Barracuda
NG
Firewall
F800
The
NG
Firewall
F800
remained
functional
through
most
of
NSS
Labs
performance
and
security
testing,
though
some
stability
issues
were
noted
initially.
The
device
was
rated
by
NSS
Labs
at
7.8
Gbps
out
of
the
9.2
Gbps
1
claimed
by
the
vendor.
The
F800
scored
80%
for
Stability,
100%
for
Evasion,
100%
Leakage,
and
95%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$11
per
protected
megabit,
and
76%
for
security
and
management
effectiveness.
Barracuda
Networks
management
interface
is
reasonably
well
designed
and
intuitive,
although
there
were
some
instances
where
options
and
configuration
parameters
were
difficult
to
find
or
appeared
out
of
place.
Tuning
and
maintenance
is
straightforward
once
the
complexities
of
the
interface
have
been
mastered.
For
users
of
Barracuda
Networks
firewalls,
there
will
not
be
a
significant
learning
curve.
However,
there
may
be
a
steep
learning
curve
for
those
new
to
the
interface.
The
good
news
is
that
there
is
excellent
documentation
to
help
overcome
any
difficulties.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO
please,
see
the
Barracuda
NG
Firewall
F800
Product
Analysis
Report
(PAR).
Cyberoam
CR2500iNG
1
The
CR2500iNG
was
rated
by
NSS
Labs
at
8.7
Gbps
out
of
the
32
Gbps
claimed
by
the
vendor.
The
CR2500iNG
scored
100%
for
Stability,
100%
for
Evasion,
100%
Leakage,
and
55%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$19
per
protected
megabit,
and
55%
for
security
and
management
effectiveness.
CCC
presents
a
clean
user
interface
with
tabs
across
the
top
of
the
screen
for
all
areas
of
firewall
management.
The
interface
is
intuitive
and
easy
to
use
by
any
experienced
administrator.
Reminiscent
of
the
Cisco
device
management
webUI,
the
interfaces
are
clean
and
technical.
The
interface
lacks
cross-connect
functionality,
however.
Administrators
are
forced
to
copy
and
paste
information
between
screens,
or
enter
redundant
information
multiple
times
in
different
places.
The
system
is
Java-based,
and
this
often
causes
compatibility
issues
with
different
browsers,
as
well
as
rendering
the
interface
useless
for
some
mobile
tools,
such
as
an
iPad.
iView
is
actually
quite
a
robust
and
feature-rich
logging
and
reporting
tool.
It
provides
an
alternative
for
those
environments
without
a
dedicated
SIM/SIEM.
However,
with
the
logs
and
the
firewall
management
implemented
via
two
different
interfaces
with
no
integration,
it
makes
the
system
more
difficult
to
manage
than
it
should
be.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Cyberoam
CR2500iNG
Product
Analysis
Report
(PAR).
10
NSS Labs
DELL
SonicWALL
NSA
4500
1
The
SonicWALL
NSA
4500
was
rated
by
NSS
Labs
at
850
Mbps
out
of
the
990Mbps
claimed
by
the
vendor.
The
NSA
4500
scored
100%
for
Stability,
100%
for
Evasion,
100%
Leakage,
and
95%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$35
per
protected
megabit,
and
95%
for
security
and
management
effectiveness.
DELL
SonicWALLs
management
interface
is
well
designed
and
comprehensive,
though
the
breadth
of
advanced
features
comes
at
the
cost
of
complexity.
This
is
certainly
not
an
interface
that
will
be
mastered
quickly.
However,
it
does
offer
some
highly
evolved
features
suitable
for
large
enterprise
and
multi-tenanted/service
provider
deployments,
making
it
straightforward
to
apply
complex
policies
in
a
targeted
manner
across
multiple
nested
groups
in
large-scale
deployments.
DELL
plans
to
offer
an
iOS
client
that
will
provide
administrators
the
ability
to
review
logs
and
activity
in
real-time.
Tuning
and
maintenance
is
straightforward
once
the
complexities
of
the
interface
have
been
mastered,
and
deployment
of
complex,
fine-grained
policies
across
large
organizations
is
made
easy
thanks
to
the
implementation
of
advanced
features
such
as
nested
groups
and
inheritance.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Dell
SonicWALL
NSA
4500
Product
Analysis
Report
(PAR).
Palo
Alto
Networks
PA-5020
1
The
Palo
Alto
Networks
PA-5020
was
rated
by
NSS
Labs
at
4.1
Gbps
out
of
the
5
Gbps
claimed
by
the
vendor.
The
PA-5020
scored
100%
for
Stability,
100%
for
Evasion,
100%
Leakage,
and
95%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$24
per
protected
megabit,
and
95%
for
security
and
management
effectiveness.
Palo
Alto
Networks
management
interface
was
reasonably
intuitive
for
most
tasks,
making
it
relatively
straightforward
to
use
without
extensive
training.
Tuning
and
maintenance
is
straightforward,
making
it
suitable
for
environments
where
only
occasional
updates
are
expected
or
where
there
is
a
lack
of
extensive
on-site
expertise.
However,
certain
features,
such
as
the
lack
of
identification
of
application
dependencies
during
policy
creation
and
lack
of
support
for
group
management,
made
it
clumsy
to
use.
Those
used
to
the
more
traditional
port-
and
protocol-based
security
ACL
rules
will
struggle
with
the
lack
of
granularity
in
Palo
Altos
rules.
The
simple,
single
detection
engine
approach
will
find
favor
with
SMB
users,
however.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Palo
Alto
Networks
PA-
5020
Product
Analysis
Report
(PAR).
WatchGuard
XTM
1050
1
The
WatchGuard
HTM
1050
was
rated
by
NSS
Labs
at
2.2
Gbps
out
of
the
10
Gbps
claimed
by
the
vendor.
The
XTM
1050
scored
100%
for
Stability,
100%
for
Evasion,
100%
Leakage,
and
85%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$40
per
protected
megabit,
and
60%
for
security
and
management
effectiveness.
11
NSS Labs
WatchGuard
provides
a
suite
of
management
applications
for
use
with
its
centralized
server
and
managed
devices,
but
while
the
firewalls
all
are
capable
of
high
availability
(HA),
the
management
server
lacks
any
fault
tolerance
/
redundancy
features.
While
it
is
possible
to
review
prior
firewall
configurations
following
modifications,
there
is
no
delta
view.
Administrators
are
forced
to
manually
identify
changes
from
one
saved
configuration
to
another.
Other
features
such
as
drag-and-drop
VPN
construction,
and
the
ability
to
pre-configure
new
firewalls
using
a
free
cloud-based
configuration
service,
allow
for
rapid
deployment
at
remote
offices.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
WatchGuard
XTM
1050
Product
Analysis
Report
(PAR).
Caution
Cisco
Systems
Cisco
was
not
included
in
the
2013
firewall
test
since
it
does
not
currently
have
an
enterprise
class
firewall
in
its
product
line.
The
Adaptive
Security
Appliances
are
unified
threat
management
(UTM)
devices
and
thus
not
optimized
for
deployment
as
dedicated
firewalls.
According
to
Cisco
representatives
there
is
a
dedicated
firewall
device
in
development,
and
NSS
is
looking
forward
to
testing
this
shortly.
Until
that
time,
NSS
recommends
that
enterprises
looking
to
purchase
a
dedicated
firewall
solution
should
consider
other
alternatives.
NETASQ
NG1000
A
1
The
NETASQ
NG1000
A
was
rated
by
NSS
Labs
at
2.5
Gbps
out
of
the
7
Gbps
claimed
by
the
vendor.
The
NG1000
A
scored
100%
for
Stability,
70%
for
Evasion,
100%
Leakage,
and
100%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$27
per
protected
megabit,
and
70%
for
security
and
management
effectiveness.
Administrators
are
presented
with
a
worldview
showing
deployed
firewalls
and
indicating
VPN
tunnels,
all
of
which
are
color
coded
to
reflect
the
health/status
of
each
managed
device
and
tunnel.
Administrators
can
drill
down
into
each
device,
and
are
presented
with
a
clean
interface
allowing
for
quick
management
of
the
devices.
While
the
management
console
centralizes
the
logs
from
the
managed
devices,
a
second
product,
NETASQ
Event
Analyzer,
is
required
to
view
this
data.
Event
Analyzer
is
also
used
to
generate
all
standard
firewall
event
reports.
Centralized
Manager
automatically
creates
backups
of
firewall
configurations
during
an
update
and
administrators
are
able
to
navigate
through
these
backup
images
to
restore
a
firewall
to
a
prior
state.
The
interface
is
crisp
and
quick
to
respond,
and
intuitive
for
an
experienced
administrator.
Rule
creation
is
robust,
allowing
administrators
to
define
complex
rules
quickly
and
without
the
interference
of
restrictive
parameters
often
found
on
such
interfaces.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
NETASQ
NG1000
A
Product
Analysis
Report
(PAR).
2013
NSS
Labs,
Inc.
All
rights
reserved.
12
NSS Labs
NETGEAR
ProSecure
UTM9S
1
The
NETGEAR
ProSecure
UTM9S
was
rated
by
NSS
Labs
at
231
Mbps
out
of
the
850
Mbps
claimed
by
the
vendor.
The
device
had
issues
during
the
stability
and
reliability
tests,
however.
The
UTM9S
scored
40%
for
Stability,
70%
for
Evasion,
100%
Leakage,
and
15%
in
the
management
interface
review
(there
is
no
central
management
option).
All
of
which
resulted
in
a
TCO
of
$5,950
per
protected
megabit,
and
4%
for
security
and
management
effectiveness.
NETGEAR
does
not
have
a
centralized
management
console,
forcing
administrators
to
manage
all
deployed
firewalls
one-on-one
through
direct
device
management
(DDM).
This
does
not
scale
in
an
enterprise
environment.
Administrators
are
limited
by
the
DDM
interface,
which
uses
inflexible
web-forms
for
reporting,
access
control
list
development,
NAT,
etc.
The
interface
is
overly
restrictive
with
its
use
of
drop-down
and
check-box
menu
items
that
have
been
migrated
from
NETGEARs
home
firewall
appliances.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
NETGEAR
ProSecure
UTM9S
Product
Analysis
Report
(PAR).
Sophos
UTM
425
1
The
Sophos
UTM
425
was
rated
by
NSS
Labs
at
3
Gbps
out
of
the
6
Gbps
claimed
by
the
vendor.
The
UTM
425
scored
100%
for
Stability,
70%
for
Evasion,
100%
Leakage,
and
65%
in
the
central
management
review.
All
of
which
resulted
in
a
TCO
of
$44
per
protected
megabit,
and
46%
for
security
and
management
effectiveness.
Sophos
UTM
Manager
has
some
limitations,
and
the
administrator
must
open
a
session
to
the
direct
device
management
port
of
the
firewall
to
fulfill
certain
tasks.
These
include
access
to
change
control
logs,
granular
firewall
transaction
log
data,
front
panel
view,
and
port
status
and
utilization
information
for
the
device.
Logs
are
presented
as
tab
delimited
text
files
in
new
windows
within
the
browser.
Administrators
are
required
to
compile
and
normalize
such
data
into
spreadsheets,
or
feed
it
into
a
SIM/SIEM
to
filter,
sort
and
parse
the
output.
Sophos
has
included
UTM
features,
such
as
Insight,
into
the
anti
virus
report
data
coming
from
protected
endpoints.
However,
the
system
is
missing
features
commonly
found
on
enterprise-class
central
management
systems,
such
as
transaction
roll-back
and
failsafe
checks
on
new
configurations
prior
to
deployment.
Administrators
are
required
to
make
DDM
connections
to
firewalls
to
correct
these
issues.
For
an
in-depth
evaluation
of
security,
management,
performance
and
TCO,
please
see
the
Sophos
UTM
425
Product
Analysis
Report
(PAR).
13
NSS Labs
Test
Methodology
Methodology
Version:
Firewall
v4
A
copy
of
the
test
methodology
is
available
on
the
NSS
Labs
website
at
www.nsslabs.com
Contact
Information
NSS
Labs,
Inc.
206
Wild
Basin
Rd,
Suite
200A
Austin,
TX
78746
USA
+1
(512)
961-5300
info@nsslabs.com
www.nsslabs.com
v2013.02.07
This
and
other
related
documents
available
at:
www.nsslabs.com.
To
receive
a
licensed
copy
or
report
misuse,
please
contact
NSS
Labs
at
+1
(512)
961-5300
or
sales@nsslabs.com.
2013
NSS
Labs,
Inc.
All
rights
reserved.
No
part
of
this
publication
may
be
reproduced,
photocopied,
stored
on
a
retrieval
system,
or
transmitted
without
the
express
written
consent
of
the
authors.
Please note that access to or use of this report is conditioned on the following:
1. The information in this report is subject to change by NSS Labs without notice.
2.
The
information
in
this
report
is
believed
by
NSS
Labs
to
be
accurate
and
reliable
at
the
time
of
publication,
but
is
not
guaranteed.
All
use
of
and
reliance
on
this
report
are
at
the
readers
sole
risk.
NSS
Labs
is
not
liable
or
responsible
for
any
damages,
losses,
or
expenses
arising
from
any
error
or
omission
in
this
report.
3.
NO
WARRANTIES,
EXPRESS
OR
IMPLIED
ARE
GIVEN
BY
NSS
LABS.
ALL
IMPLIED
WARRANTIES,
INCLUDING
IMPLIED
WARRANTIES
OF
MERCHANTABILITY,
FITNESS
FOR
A
PARTICULAR
PURPOSE,
AND
NON-INFRINGEMENT
ARE
DISCLAIMED
AND
EXCLUDED
BY
NSS
LABS.
IN
NO
EVENT
SHALL
NSS
LABS
BE
LIABLE
FOR
ANY
CONSEQUENTIAL,
INCIDENTAL
OR
INDIRECT
DAMAGES,
OR
FOR
ANY
LOSS
OF
PROFIT,
REVENUE,
DATA,
COMPUTER
PROGRAMS,
OR
OTHER
ASSETS,
EVEN
IF
ADVISED
OF
THE
POSSIBILITY
THEREOF.
4.
This
report
does
not
constitute
an
endorsement,
recommendation,
or
guarantee
of
any
of
the
products
(hardware
or
software)
tested
or
the
hardware
and
software
used
in
testing
the
products.
The
testing
does
not
guarantee
that
there
are
no
errors
or
defects
in
the
products
or
that
the
products
will
meet
the
readers
expectations,
requirements,
needs,
or
specifications,
or
that
they
will
operate
without
interruption.
5.
This
report
does
not
imply
any
endorsement,
sponsorship,
affiliation,
or
verification
by
or
with
any
organizations
mentioned
in
this
report.
6.
All
trademarks,
service
marks,
and
trade
names
used
in
this
report
are
the
trademarks,
service
marks,
and
trade
names
of
their
respective
owners.
14