Troubleshooting

1. Define
2. Analyse
3. Verify

Define
Identify specific symptoms of the issue.

Identify any error messages related to the issue.

Confirm steps required to reproduce the error or symptom.

Analyse/ Root Cause Analysis

Use the Sophos knowledge base and community for the defined symptoms or error.

Check the configuration of the affected function and the log files for additional information.

Use troubleshooting tools for further investigation as required

Verification Testing
Apply the resolution of the issue.

Repeat the replication steps of the issue to test the solution.

Confirm the issue has been resolved.

Device Access
Web Admin Console
Console and Advanced shell can be accessed through SSH, Web Admin Console or Physical
Connection.

Can be configured in System > Administration > Device Access.

Alternatively, you can configure a Local ACL Service Exception rule to access from a specific host
or network (this is recommended when enabling access to the firewall through the WAN zone.

The local ACL Service Exception rules override the behaviour of the local service ACL table.

SSH Access to the CLI can also be secured using public key authentication.

The thumbprint of the key used to authenticate is logged in /log/sshd.log

Accessing the Console

Access to the advanced shell is restricted to licenced commercial versions of the product.

In advanced shell, you can run the command cish to start a Console session.

Logging
Access to real-time logs using the log viewer.

You can add up to 5 external syslog servers.

Manage which events are logged.

You can apply structured filters to the logs and perform free text searches.

Data in logs can be exported to a CSV file.

Each log entry has a link to open an Open Live Product Packet Capture.

Packet information is also available in HEX and ASCII values

Packet capture can also be accessed from the Diagnostics menu with the same options
available.

Wrap capture buffer once full option allows you to continue capturing packets even after the
buffer is full

A Berkeley Packet Filter (BPF) can also be defined, and it is protocol independent. It uses a filter
before buffering approach.

tail can be used to monitor a log file.

SURF – Sophos Universal Reader and Finder
It provides an integrated view of all files to aid troubleshooting and resolution of issues.

It reads data from an SDU (endpoint) or a CTR (firewall) and creates a local database.

It extracts relevant data from the database and displays that data on the SURF dashboard.

Can be downloaded from the Sophos Partner Portal only.

Only available on Windows 10 or later.

Routing
Routing Precedence
1. Health Check Route
2. Static Route
3. SD-WAN Route
4. IPSec VPN Route
5. Default Route (WAN Link Manager)

3 main types of routes

1. Static Routes
2. SD-WAN Routes
3. VPN Routes

Health Check routes are used by Sophos Firewall to ensure that health probe traffic is sent
through the gateway that is being monitored and not being matched on any other route.

Static routes define the gateway to use based on the destination network e.g. directly connected
networks, routes added by dynamic routing protocols, and routes created for SSL VPNs.

SD-WAN Policy routes make decisions based on the properties of the traffic, such as source,
destination and service.

VPN routes are created automatically when policy-based IPSec VPN connections are
established with the Sophos Firewall.
When no other routing rule has been matched, the Sophos Firewall will send the packets on the
default route, which is the active gateway derived from the load balancing configuration across
active gateways.

The precedence of the static routes, policy routes and VPN routes can be modified on the
command line.

Packet Routing
The packet is matched in the firewall based on the post-NAT zone and pre-NAT IP.

If the traffic is destined for the WAN zone and no PBR (policy-based route) or RTG (route through
gateway) has been matched, the parker is marked MLM (Multi Link Management).

MLM is the gateway derived from the load balancing configuration across active gateways.

A NAT lookup is performed lastly.

To show the routing table run ip rule list on the advanced shell.

If a packet is marked for RTG, the Sophos Firewall will still traverse the full route precedence but
will not be able to match PBR because the firewall-mark will be different.

RTG will always have a lower precedence than VPN and static.

You can look up the route table associated with each gateway e.g. ip route list table wanlink1

By using the ip rule list and ip route list table commands you can navigate the routing table tree
to identify how traffic is being routed.

You can use conntrack command in the advanced shell to see how packets are being marked for
routing and then use the previous commands to look up the gateway.

Route Precedence
Route precedence can be managed on the console.

By default, static routes have the highest precedence.

The command system route_precedence show displays the current precedence of the routes.

Set new route precedence using the command: system route_precedence set
sdwan_policyroute vpn static

Default route precedence is Static > SD-WAN policy routes > VPN routes
SD-WAN policy routes
Policy routes on Sophos Firewall upgraded from v17.5 or earlier do NOT match on reply packets
or system generated traffic

Enable new behaviour via the console

Policy routes are applied to reply packets and system generated traffic.

As this was not the behaviour in earlier version of Sophos Firewall these behaviours are not
enabled when the Sophos Firewall has been upgraded.

They can be enabled through the console using the commands set routing sd-wan-policy-route
reply-packet enable and routing sd-wan-policy-route system-generate-traffic enable.

For new installations of Sophos Firewall v18.0 and later these options are enabled by default.

Health Checks
Sophos firewall checks the health of gateways, which is used by the default routing and SD-WAN
routes when configured to use primary and backup gateways.

SD-WAN profiles also have health checks; these are independent of the gateway health checks,
and the health check for each profile is also independent of each other.

Gateway Health Check

- Used by default routing (MLM), RTG
- SD-WAN primary and backup

SD-WAN Profile Health Check

- Independent of gateway health check
- Each profile does independent health checks or probing for all the selected gateways
- One does not override other’s decision

SD-WAN Profile Probe Targets

SD-WAN profiles support 2 probe targets.
When 2 probe targets are configured, Sophos Firewall will use the first probe target for
performing the health checks on the connections.

If the first probe target is unavailable, Sophos Firewall will use the second probe target.

If both probe targets are unavailable, the health check fails as the gateway is unavailable.

In some circumstances it is possible that all the configured gateways in an SD-WAN profile fail to
meet the configured SLA.

In this case Sophos Firewall will use the first available gateway.

It is necessary to use a custom SLA with higher values that the links can achieve to get the
desired routing outcomes where the links are of poor quality.

SD-WAN Log Viewer

In the log viewer you can add columns to the firewall log to show which SD-WAN profile and SD-
WAN route is being used for each connection.

You can also see events for the SD-WAN profile health checks and when routes change.

When you are troubleshooting routing, you can check the log viewer to see which SD-WAN
profile, route and gateway are being used, and the interface that the traffic is leaving the firewall
on.

You can use the command conntrack to see the gateway mark and outbound port.

Application Based Policy Routing Behaviour

Application-based routing uses learned routes which means for the very first connection from an
application it will be routed via the default route.

The DPI (Deep Packet Inspection) supports application- based routing for all applications;
however, the legacy web proxy does not support this for micro-applications.

Application-based routes require an active Web Protection license and one of the following:

• Application classification is on, which it is by default.

• An application filter is applied to the firewall rule
• Or the application is part of the offload signatures and is flowing through snort.

In HA (High Availability), the cached application-based routing information is synchronized over

the dedicated HA link using multicast IP 226.1.1.1 on port 4455
Troubleshooting Application Based Policy routing
You can review the application policy route information with the command ipset -L appset in the
Advanced Shell.

Please note that application IDs above 10000 are applications identified by Synchronised
Application Control. Other application IDs are in the set that Sophos Firewall has detection for by
default.

Network Resources Inaccessible

The first thing to do is to check the logs using the log viewer.

If the traffic is being allowed on the gateway(s) you can perform a packet capture to see what is
happening e.g. using the BPF (Berkley Packet Filter) String host 192.168.16.30 and port 80 to
select relevant traffic.

Additional Troubleshooting Tools

There are a couple more useful tools for troubleshooting routing issues. The first is tcpdump.

tcpdump is a command line packet capture tool that can be used on the Console or Advanced
Shell.

You can use a BPF string with tcpdump in the same way as with the packet capture in the web
admin console.

tcpdump output can be saved to a pcap file, which allows you to view the output of the tcpdump
command and analyse it using the other packet analyser tools, such as WireShark, on an
external computer.

For example, on the Console you can run tcpdump ‘host 192.168.16.30 and port 80 -w
/tmp/dump.pcap’

You can then use SCP to copy the output to another server from the Sophos Firewall e.g. scp
/tmp/dump.pcap user@servername: /folder/location/

When using tcpdump in the advanced shell you use 11h to print the link-level header on each
dump line.

conntrack -L will output all the current connections at that point in time. The output can be used
with grep for filtering.
conntrack -E will continuously write updates to the screen. Each entry will be tagged as a new
connection, an UPDATE to an existing connection, a connection that is being DESTROYed, and so
forth.

You can also apply filters to the command such as the protocol, port and destination.

Lost Admin Password

To reset the admin password back to the default ‘admin’ requires physical access to the device
console.

At the login prompt enter RESET and this will give access to a reset menu where you can choose
to reset the admin password for the device.

Unable to Access the Web Admin Console

• Check the connectivity to the firewall
• Connect to the firewall from the CLI
• Check the tcpdump output and logs
• Check and restart services
• Reset the web admin console certificate to default device certificate

Check Connectivity to Sophos Firewall

1. Verify that the IP and port you are accessing the firewall are correct
2. Ping the LAN/DMZ/WAN IP of Sophos Firewall
3. Try to access the User Portal using https://LAN_IP_OF_FIREWALL:Port (the default port is
4443
4. Try to use another browser
5. Try to connect using SSH to the firewall’s LAN interface IP on port 22 via putty.

Locked out of Sophos Firewall

It is possible to become locked out of the Sophos Firewall in various ways, for example:

1. Misconfiguring device access so there is no access to admin services from any zone
2. Misconfiguring spoof protection such that traffic is being dropped.
For this type of issue, you can enable appliance access mode from the console. This will enable
access to all services on all interfaces.

All internet traffic is dropped while this mode is enabled.

To enable appliance access, connect to the CLI, and select option 4, Device Console.

Verify if the appliance access is disabled using the command: system appliance_access show

To enable appliance access, use the command: system appliance_access enable

If successful go to Administration > Device Access to enable HTTPS on the required zones.

Once the connectivity is restored disable the appliance access using the command: system
appliance_access disable

If the web admin console is still inaccessible, in the CLI go to Device Management > Advanced
Shell

Verify the destination port configured for HTTPS web admin console using the command: psql -U
nobody -d corporate -c "select destinationport from tbllocalservicedetails WHERE
localserviceid =2"

For further analysis, you can do a packet capture when you are trying to access the web admin
console to find out more about the root cause.

To check the tcpdump output and logs, use the advanced shell to run the command: tcpdump -
nei any port 4444 (or any port which you have configured to access the firewall).

Check and Restart Services

Check the status of tomcat service (used for the web admin console) using the command:
service -S | grep -iE 'tomcat|apache’

If the tomcat or apache service shows STOPPED, they can be started using the service
command:

• service tomcat:start -ds nosync

• service apache:start -ds nosync

If tomcat or apache service shows DEAD, use the restart command:

• tomcat:restart -ds nosync

• service apache:restart -ds nosync
Reset the Web Admin Console Certificate
If problems persist you can reset the web admin console certificate to the default device
certificate.

From the CLI main menu, select option 2. System Configuration > 4. Reset Default Web Adim
Certificate

The steps above should reset the admin certificate to its default which is ApplianceCertificate.

Failsafe Mode
If there is a problem during the startup of the Sophos firewall, it will go into failsafe mode.

This mode provides command line access to the firewall for troubleshooting only.

To see why the firewall went into failsafe mode you must run the command: show failure reason
on the Device Console

Failsafe Mode reasons

Unable to start Config database - Requires a factory reset and then restore from a backup

Unable to start Firewall framework - Reboot the Sophos Firewall

RAM size lower than minimum required - Increase the RAM assigned to the virtual appliance

Problems with any of these services can cause the Sophos Firewall to go into failsafe mode:

• Configuration database
• Signature database
• Firewall framework
• Logging (Garner)
• Network

If you are unable to resolve the failsafe mode issue using the actions here, you will need to
contact Sophos Support for further assistance.

Troubleshooting Firewall Rules

If bidirectional rules to allow traffic in and out (for example for a site-to-site VPN) are in use and
traffic only works in one direction it is likely a misconfigured firewall rule.
Common causes of firewall rules are:
Rule ordering – where a firewall rule is created either too high or too low in the rule table. This can
lead to traffic matching on the wrong firewall rule, and either being unintentionally allowed or
blocked.

Rule configuration – sometimes a firewall rule is incorrectly configured. There are many ways this
can be done, but common mistakes include selecting the wrong zones and networks or using
objects that are misconfigured.

Policies applied – the issue may not always be directly with the firewall rules itself but be caused
by the policies selected in the rule, such as web, application or intrusion prevention policy.

When you need to start troubleshooting firewall rules the first step is always to confirm which
firewall rule the traffic is matching on. It may not be marching the rule you expect or may not
match any firewall rules. Review the configuration of the firewall rule, including any objects used,
and policies selected.

If you have configured firewall rules for bidirectional traffic, such as for a site-to-site scenario,
you may find traffic works one way but not the other, this can often indicate a misconfigured
firewall rule.

Servers Cannot download Updates

You can use the built in packet capture toll in the web admin console for troubleshooting. It
shows the routing taking place, the firewall and the NAT rule IDs matched, the status and reason.

Another way of checking is using the log viewer. It provides additional information as it also
shows the other policies that are applied to the matched firewall rule. You can click on the IDs to
go directly to the rules in the web admin console.

You can reset data transfer counter for the firewall rules involved.

There is another tool that can be useful when troubleshooting firewall rules and routing issues
which is conntrack.

The command conntrack -L in the Advanced Shell and grep can be used to filter the output. In
the output you can see all the routing information as well as all the policies that apply to the
connection.
NAT Process
When a packet arrives and the marking has been done, the Sophos firewall performs a NAT
lookup for DNAT or Full NAT rules.

If a NAT rule has been matched the destination zone is translated before the packed goes to the
firewall.

The firewall will be matching rules based on the post-NAT destination zone and the pre-NAT IP
address.

After the firewall either the DNAT or Full NAT rule matched in NAT lookup is used to do the
translation, or a second NAT lookup is done for SNAT rules or linked rules, and this translation is
applied.

Finally, the packet is delivered.

Troubleshooting NAT rules

Common causes or NAT rule issues are:
• Rule ordering – creating a NAT rule either too high or too low in the rule table so that the
traffic matches on the wrong rule.
• Rule configuration – this can be caused by simply selecting the wrong option by mistake,
or by not fully understanding the scope and effect of selecting specific options.
• Firewall rules – NAT rules define the source and destination for traffic, changing it where
necessary, there still needs to be a correctly configured firewall rule to allow the traffic.

It is also important to note that DNAT rules take precedence over device access e.g. a DNAT rule
on port 22 will translate the destination and the Sophos Firewall will be accessible on that port.

Firewall rules for DNAT must use the post-NAT zone and the pre-NAT IP address for the
destination.

Sophos Firewall cannot be accessed via SSH

You can login to the web admin console with the password that is not working on SSH.

It could be possible that the traffic is matching a firewall rule and a NAT rule with the SSH
connection being NATed out of a different Port on a different zone e.g. DMZ.

When troubleshooting NAT rules, you can also use the conntrack command in the Advanced
Shell since it shows the original and translated information for the traffic.
TLS Decryption Errors
TLS connection errors can cause the webpages to fail to load properly or problems with
applications.

Clicking the widget will display more information about the errors, including the website, number
of errors encountered and the number of affected users.

Clicking on the errors of the Fix errors link will allow you to take action to correct them.

When creating an exclusion, you have options that affect the scope e.g. you can exclude the
subdomain, the whole domain, or the web category.

To exclude using other properties, such as IP address, you need to modify the TLS inspection
rules.

Excluding the subdomain will add the subdomain to the Local TLS exclusion list URL group.

You may not want to exclude the subdomain entirely; you may just want to apply a different TLS
decryption profile.

To do this you can create a URL group including the subdomain.

Creating a new TLS inspection rule above the rule that caused the error is also a good solution.
The rule should only match on the URL group you created and use a TLS decryption profile that
does not cause an error.

When there are TLS decryption errors, users may see issues using websites and applications.

You can check the decryption capacity and workload of the Sophos firewall from the system
widget in the Control centre.

If the decryption capacity is constantly above 90% it can result in timeout errors.

On the right-hand side of the Control centre, you can see the TLS connections widget which
quickly shows if there are many errors and clicking on the widget allow you to see more detailed
information.

The decryption limit of the Sophos firewall is set depending on the number of CPUs and the
amount of RAM.

If an issue is caused by misconfigured TLS inspection rule that was decrypting traffic from the
LAN zone to any zone, the rule is corrected by limiting the scope of the correct destination zones.
Further TLS troubleshooting
Check that the TLS rules are not matching traffic you do not want to decrypt

Check for causes of increased TLS traffic

Allow TLS connections when the limit is exceeded

Increase the resources of the Sophos Firewall

FastPath
FastPath Driver Support
i40e

igb

ixgbe

Hardware / Virtual
E1000

E1000e

Virtual
vmxnet3 (VMware ESX)

Unsupported
Hardware
rtl8169

Virtual
virtio_net (KVM)
pcnet32 (VMware ESX)

vmxnet2 (VMware ESX)

Legacy Network Adapter (Hyper-V)

Synthetic Network Adapter (Hyper-V)

If FastPath is enabled when the system has no supported network cards, FastPath will fail to
load. The system will be fully functional without the performance enhancements by FastPath.

Manage FastPath
You can check the status of FastPath on the Sophos firewall with the command system firewall-
acceleration show in the console.

If FastPath is disabled when you check the status it shows the message ‘Firewall Acceleration is
Disabled. Fastpath Unload Failed.’

You can also enable fastpath using the command system firewall-acceleration enable.

Having fastpath enabled with no supported network cards does not cause any problem with the
device.

Verify the Kernel Modules are loaded

You can verify whether the vfp modules are loaded by running the command lsmod | grep vfp in
the Advanced Shell.

Check if a connection is being offloaded to fastpath

You can check if a connection is being offloaded to fastpath using conntrack -L in the Advanced
Shell.

If the fastpath connection ID (conn_fp_id) equals NOT_OFFLOADED, then the connection is not
being offloaded to the fastpath.

If the connection is being offloaded to the fastpath, the fastpath connection ID (conn_fp_id) will
display the connection ID.
The fastpath connection ID is based on the current state of conntrack, i.e., it will tell you whether
it is being offloaded at that very moment.

Counters
There are several debug counters for fastpath that can be useful in troubleshooting.

The counters in the path are useful in verifying traffic that is using FastPath.

The list of counters is long, but a few interesting ones are:

• The one ending WIRE_TO_WIRE: this counter should increase if the application
classification is off, and traffic is not being handled by IPS.
• The one ending WIRE_TO_IPS_INSPECT: if application classification is on ad the traffic is
handled by IPS.
• The one ending IPS_TO_WIRE: for packets that are sent to fastpath from IPS.

The counter contains information on packet header parsing done by fastpath, which can be
useful information for packets that have errored.

Counters: Slow Path

Traffic from slow path to fastpath.

1. Counter 1 (FPCNTR_RX_WIRE) shows traffic being received on the wire.

2. Counter 2 (FPCNTR_FROM_WIRE_TO_KN_MFLOW_NOT_ACTIVE) is a packet flow we
have never seen before, and so uses the slow path for application classification and
routing.
3. Counter 3 (FPCNTR_RX_KN) is the traffic being received by the FastPath from the kernel.
4. Counter 4 (FPCNTR_FROM_KN_TO_WIRE) is traffic being received from the slow path to
the wire.

Counters: FastPath without IPS

Offloading to fastpath with no IPS

1. Counter 1 (FPCNTR_RX_WIRE) shows traffic being received on the wire.

2. Counter 2 (FPCNTR_FROM_WIRE_TO_WIRE) show packets taking the FastPath where IPS
is disabled, and the packet flow has previously been classified.
FPCNTR_FROM_WIRE_TO_KN_FW_REV_MISMATCH
FPCNTR_FROM_WIRE_TO_KN_NHOP_UNRESOLVED- route changes and ARP timeouts move the
flow back to the slow path

FPCNTR_FROM_WIRE_TO_KN_CONN_RECLAIMED
FPCNTR_FROM_WIRE_TO_KN_TCP_FIN_SYN_RST – FIN and RST packets go through the slow
path.

When there are route changes, firewall changes and ARP timeouts, the packet flow is moved
back to the slow path.

FIN and RST packets for tearing down a connection always go through the slow path.

Counters: Fastpath with IPS

1. Counter 1 (FPCNTR_RX_WIRE) shows the packets being received.
2. Counter 2 (FPCNTR_FROM_WIRE_TO_IPS_INSPECT) Shows the packets being sent to IPS
using zero copy.
3. Counter 3 (FPCNTR_RX_IPS) shows the packets returning from IPS.
4. Counter 4 (FPCNTR_FROM_IPS_TO_WIRE) shows the packets going to FastPath.

Unsupported Configurations
Using tcpdump forces all traffic to flow through the slow path for full visibility.

tcpdump does not support FastPath.

IPS
IPS Configuration
The behaviour of IPS is configured by default based on the device that Sophos firewall is running
on.

To verify the settings being used you can run the command show ips-settings in the console.

stream

• controls the packet streaming, which is used to restrict the streaming of packets in
situations where the system is experiencing memory issues.
 • If stream is set to on, which is the default setting, the IPS engine builds an internal table
during a session and deletes them at the end of each session.
• It also reassembles all incoming packets and checks the data for any known signatures.

lowmem

• Affects the memory and performance of the appliance.

• This setting does not have any effect on how the signatures operate.
• By default, this is set to on where the devices memory is limited and off on high-end
devices where there is abundant memory to handle IPS signatures.

maxsesbytes
• Sets the maximum allowed file size to be scanned by IPS.
• The default setting zero means the device will check all the data in the session.
• This value is applied per session.

maxpkts
• Sets the number of packets to be sent for an application classification.
• By default, this is set to 8 but can be changed to send all packets or any number of
packets above 8.

enable_appsignatures
• Turns app-based signatures on or off for IPS.
• App signatures determine the application that is using a data stream to help determine if
the traffic is malicious.
• By default, app-based signatures are enabled.

http_response_scan_limit
• Sets the scan limit for HTTP response packets.
• Available are 0-262144, for full scanning this should be set to 0.

search_method
• Sets the search method to be used for IPS signature pattern matching based on the
device hardware.

sip_prepoc
• Sets whether the SIP preprocessor should be enabled or not.
• Enabling this will scan all SIP sessions to prevent any network attacks.

sip_ignore_call_channel
• Sets whether the audio and video data channels should be ignored.
inspect_untrusted_content
• The flow will be offloaded to fast path when engine detects that all IPS, App, Web and AV
inspections can be offloaded.

The optimal value for MXPKTS and stream on/off will vary from case to case, and depends on
many factors including:

• Deployment type and size

• Number of signatures being used
• Network traffic being generated
• ISP bandwidth
• Number of published servers/resources

If you feel this value need to be adjusted, we recommend consulting Sophos Support.

Signature search methods:

• ac-bnfa: low memory usage, high performance
• ac-q: high memory usage, best performance
• hyperscan: low memory usage, best performance

By default, Sophos firewall will automatically run one IPS instance per CPU core.

IPS Log File

IPS logs to /log/ips/log.

When troubleshooting you may need to enable debug logging to get information.

You can also do this by running the command service ips:debug -ds nosync in the Advanced
Shell.

Remember to disable debug logging when you are done as it will generate a lot of data.

For each packet from the client or server on the debug log output you can see:

• the verdict, which is what to do with the packet

• the packet number, pktnum
• which functions have been offloaded to the FastPath, flowoff [IPS]
• and the application ID and application category, appcat [0:0]

You can also see the application being identified.

Once the application has been categorized, we can see that the verdict changes from 0, allow
the packet, to 4, drop the session.

If the session is offloaded to fastpath, look for flowoff [Session].

IPS Verdict:
• 0 - allow the packet
• 1 - drop the packet
• 4 - drop the session
• 7 - hold the packet until next verdict
• 8 - Offload the session

RDP Not Being Blocked by IPS

IPS is fundamental in application identification.

First, we must check if the IPS service is running by using the command service -S | grep ips in
the advanced shell.

Next, we check the IPS settings to identify id the application signatures have been disabled.

We can enable them with the command set ips enable_appsignatures on.

Incorrect IPS settings with application signatures off may result in RDP not being blocked by IPS.

Active Threat Response Detections

You may see sources blocked by Active Threat Response in the Control Centre.

Active Threat Response has 2 threat feeds:

• MDR to detect and block IoCs, indicators of compromise, on your network

• Sophos X-Ops to detect and block access to C2, command-and-control, servers by
compromised devices on your network.

Note that the widget in the control centre resets the counter at midnight.

Cleaning the cause of the detection will not reset the counter.
Detection Details
On control centre under Active threat response tab, you can the computer that was the source of
the blocked traffic, and which feed it was detected by i.e. Sophos X-Ops feed or MDR threat feed.

As part of MDR, if you are getting detections from the MDR threat feed, the MDR team will
respond according to how you have setup MDR.

By clicking on the blocked sources widget in the Control Centre you can check the hostname,
threat name and the number of times it was blocked.

Check the Sophos Website

For Sophos X-Ops threat feed detections, if you do not have MDR, you will need to search the
Sophos Website, knowledgebase, and community pages for the latest information about the
detected threat.

If a detection is related to a computer trying to connect to a server that is categorised as a

command-and-control server used by malware we recommend the following remediation steps.

Recommended remediation steps:

1. Identify the compromised machine. The IP address of the machine attempting to connect
to the C&C server will be visible within the alert.
2. Perform a full system scan on the compromised machine using the Sophos Virus
Removal tool.

In some very limited cases, it may be necessary to create an exception, either for a host or a
specific threat. We recommend consulting Sophos Support before doing this to minimise loss of
protection.

Sophos Central Registration

The first step in using Synchronised Security is to register the Sophos firewall with Sophos central
in SYSTEM > Sophos Central.

You have 2 options, using OTP or email address to authenticate.

The email address option can be used by admins who have access to both the Sophos Central
account and the firewall web admin consoles.
If you are a firewall admin but not the Sophos Central admin, then you can have the central
admin generate an OTP for you. The OTP will be valid for 14 days and it allows you to join the
firewall without having to be given an admin account on Sophos Central.

If you review the sophos-central.log using the command tail -f /log/sophos-central.log in the
advanced shell, you will see the error ‘Email address or password incorrect. Sophos Central
rejected credentials.’ This Error is usually caused by an incorrect username or password.

If you are attempting to join the firewall via an OTP, you may encounter the error ‘Couldn’t register
the firewall with Sophos Central. Verify your Sophos Central Credentials.’ This can happen if the
OTP string was modified in any way. This can be caused by not copying the entire string, copying
extra characters, or adding/removing anything from the string.

If the string is too heavily modified, you may also get an error indicating that it could not be
recognised as a valid OTP string.

If the error is more generic it is usually caused by an incomplete OTP request which then results
in the cancellation of the registration process.

Re-entering the correct username and password OTP allows the Sophos firewall to successfully
register with Sophos central.

You can also experience an issue where you have entered the credentials to register, but after the
activity indicator disappears, nothing seems to have happened with no error being displayed.

If your review the log file sophos-central.log in advanced shell, you will see connection errors
being reported. For example, it can fail to connect to utm.cloud.sophos.com.

After failing 3 times it will stop attempting to register. All of this indicates an issue
communicating with the Sophos central registration site.

To continue troubleshooting this, confirm if you can connect to https://utm.cloud.sophos.com

using a browser. While there is no page at this location, you will notice in the address bar of the
browser, it redirects you to www.central.sophos.com to perform an authentication request. If
the test fails, it will time out to reach the site utm.cloud.sophos.com.

Check the route to the internet (sophos.com) in the advanced shell using traceroute. Please
note that utm.cloud.sophos.com cannot be pinged. Once you have identified the devices
between the Sophos firewall and the internet, locate and correct the configuration that is
blocking access to utm.cloud.sophos.com.

With the configuration blocking access corrected, the Sophos Firewall will be able to register
with Sophos Central.

Security Heartbeat
Establishing a Security Heartbeat with Sophos Firewall
Security heartbeat works by using Sophos central to broker trust between the Sophos Firewall
and the endpoints.

When a Sophos firewall registers with Sophos Central it receives certificates, signed by a Sophos
Central Certificate Authority, the heartbeat IP address and port, as week as a list of IDs for
computers registered in that Sophos Central Account.

After the Sophos firewall is registered the endpoints will receive the certificate authorities and
certificates for the Sophos firewalls, as well as the heartbeat IP address and port.

The following conditions must be met:

• Computers must be connected to the local network or via a VPN
• Sophos firewall must be the default gateway or public IP must be added to the VPN
profile.

Once the endpoint has all the data from Sophos central it will try to establish a heartbeat using
the IP address and port provided. The IP address is always a public IP address.

The Sophos firewall will intercept the traffic to the IP address and respond to establish the
heartbeat connection.

As the heartbeat IP address is public, traffic for it will be sent to the default gateway, so the
Sophos firewall must be on the route to the internet which means that an endpoint must either
be on the network behind a Sophos firewall or connected via VPN.

Where the endpoint is connected via VPN, it must either be the default gateway or route traffic
for the heartbeat IP address.
Computer Not Establishing a Heartbeat
First check the heartbeat configuration on the endpoint in:
C:\ProgramData\Sophos\Heartbeat\config\heartbeat.xml

There could be a basic configuration but a missing configuration for any Sophos Firewalls.

If the issue is with servers, check that Sophos security heartbeat is enabled in the policy.

There is no separate option to enable and disable Security Heartbeat for endpoints.

When the policy is enabled and there are Sophos firewalls registered with the Central account,
the heartbeat.xml on the endpoint should include the client’s private key and the Sophos
central certificate authorities.

The Sophos Firewall ID and certificate are used for Identification, authentication, and encryption
while the Client Certificate is used for user authentication and secure communication between
the client and the firewall.

You can also review the logs on the endpoint for errors by navigating to
C:\ProgramData\Sophos\Heartbeat\logs\heartbeat.log.

Failure to make a secure connection from the endpoint to the firewall will result in errors.

In the log viewer you can filter for traffic on port 8347 to check that the Sophos firewall is
receiving the traffic.

In advanced shell, reviewing the heartbeatd.log using the command tail -f heartbeat.log may
show an SSL error.

The other reason for errors may be that the Sophos firewall is being registered in a separate
Sophos central account to the endpoint. This is resolved by registering both the endpoint and the
firewall on the same Sophos central account.

If separate Sophos central accounts are being used at different locations, then it will not be
possible for a laptop taken from one location to establish a heartbeat with the Sophos firewall at
another location.

You can see the list of registered firewalls in Sophos central in General Settings > Registered
Firewall Appliances.
If you see a ‘Connection failed’ error in the heartbeat.log on the endpoint, this means that there
was no response from trying to establish the Security Heartbeat. This is usually caused by
Sophos firewall not being on the default route to the internet from the endpoint.

Use the trace route command (tracert -d 52.5.76.173) on the endpoint to check whether the
path to the heartbeat IP address is a public IP.

If the Sophos firewall has been deployed as a web proxy and not the default gateway, then you
may have issues with establishing a heartbeat.

To resolve this, you can add a route to the gateway device that will send traffic for the heartbeat
IP address to the Sophos firewall.

Health Status
Green – endpoint agent is running and there is no active or inactive malware or PUAs detected.

Yellow – endpoint agent is running, however, inactive malware or PUA has been detected.

Red – endpoint agent may not be running, and devices may not be protected; or active malware,
malware has not been cleaned up, malicious network traffic or communication to known bad
hosts may have been detected.

Missing Heartbeat – Sophos firewall will also report endpoints with missing heartbeat. These are
endpoints that had established a heartbeat but are no longer sending it.

Ipset
Ipsets for health status and missing heartbeat. The MAC addresses of the computer’s status are
also stored in ipsets that can be checked in the advanced shell and they can be checked by
using the command:

ipset -L [hb_green | hb_yellow | hb_red | hb_missing].

This can be useful when troubleshooting behaviour that might be related to a device’s health
status.

Malicious Traffic Detections

Outbound malicious traffic, for example, connections to command-and-control servers, can be
detected by both the Sophos firewall, through active threat response, or the endpoint’s Network
threat protection service.
Devices that have had a malicious traffic detection will show as ‘At risk’ if they support Security
Heartbeat.

In the ACTIVE THREAT RESPONSE section you can see the computer involved, the threat
detected, and the process name.

On the endpoint you may see multiple detections, one for the endpoint and another for Sophos
firewall.

C2/Generic-A – Sophos firewall detected malicious traffic from an endpoint

• To a known C&C server

• Performing a DNS lookup for a known C&C server
• Triggered IPS rule

Note: C2/Generic-A is only reported on the Sophos firewall UI and not on the endpoint.

C2/Generic-B – the endpoint detected a process that attempted to contact a known C&C server
URL or IP address.

C2/Generic-C – Sophos firewall detected a C2/Generic-A threat and notified the endpoint to
report a C2/Generic-C threat in the quarantine for the identified process.

As malicious traffic detections are based on network traffic and not on a file there is nothing
directly to clean up.

This type of detection always requires some manual investigation to determine the cause of the
detection to be sure there is nothing malicious present then the detections must be marked as
resolved to return the health status of the device to GREEN.

Virus Detections
Virus detections are not always cause for a RED health status; in many cases they will be
YELLOW status, e.g. inactive malware on the system may only cause YELLOW status.

In the majority of cases malware will be cleaned up automatically, so the health status of the
device will automatically return to GREEN.

However, if automatic clean-up for malware fails, this will likely cause the health status of the
device to change from YELLOW to RED.
Lateral Movement Protection
In a scenario where computers are connected to the same section of network as a device that
has detected malware, the computers on this section of the network can communicate with
each other without the traffic passing through the Sophos firewall.

When the Sophos firewall receives a red health status for a laptop on the network, it shares the
MAC address of the laptop with all the endpoints it has a heartbeat with.

The computers can use the MAC address to drop traffic from the computer with the RED health
status.

Currently, only Windows endpoints will drop traffic from computers with a red health status.

It is important to note that because this relies on the other computers being able to see the MAC
address of computers with a red health status, this would not work if we replaced the switch with
a router.

Lateral Movement protection is enabled in Sophos Central in General Settings > Reject
Network Connections.

In the configuration you can exclude specific computers from dropping traffic from computers
with a red health status. This may include an update cache or message relay for Sophos Central,
or an admin computer that will be used to remotely access the device.

You can exclude devices from lateral movement protection in Sophos Firewall.

Cannot Connect to Device

A scenario where a share cannot be accessed by a computer on the same network and the
computer may be able to access other resources without any problem.

If the computer hosting the share has a red health status it will show up in the control centre, and
by clicking on the security heartbeat widget you can see which computer it is.

If you review the heartbeat.log you can see the health status being update from the endpoint
and the notification to other endpoints of the change.

To identify the cause of the red health status you will need to login to Sophos Central.

In the computer’s details, more information can be found, and if appropriate, the alert can be
marked as resolved to restore the GREEN health status.

If the computer still has an active thread, marking it as resolved will change it to green until the
threat is detected again.
Limitations of Lateral Movement Protection
If traffic has passed through a router the endpoint will not see the original MAC address and will
not be able to drop the traffic.

Where the traffic passes through the Sophos firewall, security heartbeat settings in the firewall
rules can protect other computers that cannot see the original MAC address because the firewall
uses the IP address of devices.

The solution to this limitation is to route all traffic back through the Sophos firewall where it is
passing through a router using VLANs which allows the Sophos firewall to enforce protection
through firewall rules.

SSL VPNs
Log Files
Review the log file from both the server and the client side of the SSL VPN connection from
/log/sslvpn.log.

The SSL VPN between 2 Sophos firewalls uses a client-server model.

SSL VPN Cannot Connect

If you check the logs and see that the host address cannot be resolved it could be because of 2
things.

1. The host address has been misconfigured.

2. The host address is correct but cannot be resolved.

In such a case, we check the host address and if it is correct, the next step would be to verify the
DNS configuration.

The hostname can be corrected or overridden in the VPN settings on the Sophos Firewall web
console.

It is also possible to override the peer hostname for the site-to-site VPN. If DNS resolution is an
issue, you can override that with an IP address.

With the host address either corrected or overridden, the VPN will be able to connect.
If a site-to-site VPN fails to connect to server, the first step in troubleshooting is to determine if
the traffic is reaching the Sophos firewall, and if so, what is happening to it.

Checking the log viewer and applying a filter for the destination port 8443, the default port for SSL
VPN on Sophos firewall, allows us to see what is happening. You can see that the appliance
access is denied so we look at the device access configuration.

If Device access to SSL VPN is not enabled for the WAN zone for the VPN to connect, we have to
enable it.

Remember to check the local ACL exception rules as well; even if the SSL VPN is enabled for the
WAN zone, there could be an exception rule denying access.

If the problem was not resolved by enabling SSL VPN for the WAN zone in the device access
settings, the next step would be to determine the route the traffic is taking to the Sophos firewall
by checking the configuration of the devices on the route to ensure the port is not being blocked.
This is done using trace route (tracert) command.

If the port is being blocked by a device outside of your control and you need to change the port, it
can be done in the SSL VPN settings on the web console, however, this would require you to
download and install the updated configuration.

You need to enable SSL VPN in device access for the zones in which you want to use it.

IPsec Site-to-Site VPNs

IPsec Logs
When troubleshooting IPsec VPN issues, it is important to get the logs from both sides of the
connection as they can report different error information.

/log/stongswan.log.

Each connection has its own log; however, the strongswan.log contains all of the connection
information and is generally best for troubleshooting.

Connection Configuration
You can view the configuration of all the IPsec connections from the Advanced Shell by running
the command: cat /_conf/ipsec/connections/*.conf.
This can be a useful view when troubleshooting connections to third party devices as it contains
all the IPsec policy information as well as the connection information.

Connection Status
You can view and monitor the connection status from the Advanced Shell with the command:
watch -5 “ipsec statusall”.

This uses watch to refresh the view of the ipsec statusall command every 5 seconds.

IPsec Connection Could Not be Established

There are a number of reasons you might get this error.

The first cause of this issue and one of the most common, is where the pre-shared keys do not
match on either end of the VPN connection.

If you check the logs and see that there is a generic authentication failed error, it is because of
the pre-shared keys not matching.

You can also see what was written to the log viewer by looking at the GARNER-LOGGING line.

Mismatching Pre-Shared Key

Using IKEv2
If we see an error written “tried 1 shared key” followed by “MAC mismatched” on either machine
on each VPN end, it is a clear indication that the issue is a mismatch of the pre-shared key and
demonstrates why having the logs from both sides is important.

If the GARNER-LOGGING shows “peer authentication failed” it is also a clear indication of

mismatching pre-shared key. This is all when using IKEv2.

Using IKEv1
The error here will be “Invalid HASH_V1 payload length, decryption failed”.

On the other end of the VPN the error might be “Invalid ID_V1 payload length, decryption failed?”.

For IKEv1 connections these errors indicate a problem with the shared key.
Incorrect ID
There are scenarios where the ID is misconfigured on side of the connection.

On one end of the connection, we might get the error “no matching peer config found”. This error
indicates that the issue is related to the connection ID.

Policy Mismatch
There could be a mismatch in the IPsec policies between the ends of the connection.

In the logs you will see “NO_PROPOSAL_CHOSEN”.

This issue is clearly logged in the log viewer, and you can see the error “the received IKE_SA
proposals did not match,” along with a list of the IPsec settings enabled for this side of the
connection.

IPsec Connection Has a Yellow Status

You may find that an IPsec connection can be established, but that on one side of the tunnel the
connection status is yellow. You can click the [i] next to the connection status for more
information.

For an example, you can see the local and remote subnets that the VPN is trying to create
associations for, with the ones marked in RED having failed. You can see that they are all for a
local subnet called ‘Another Network.’ If you compare this to the associations on the other end of
the VPN you can see that the ‘Another Network’ subnet is not shown at all.

This means that either one side has an extra subnet in the configuration that should not be there,
or the other side is missing a subnet that should be there.

Failover
IPsec VPNs can be configured in failover groups to allow alternative VPNs to be established if the
primary fails.

In some circumstances when the primary VPN fails the VPN may not seem to failover, or the
second VPN may not seem to have established.

1. Check the VPN connection configuration for the secondary VPN. (Always check that each
VPN works when adding it to a VPN group)
2. Check the failover group condition.
 3. Check that the dead gateway detection (dgd) is running.

If we consider the failover condition of the failover group, you can choose to base this on PING or
a TCP connection to a specified port on the remote VPN server.

You can optionally combine the two tests. You need to check that the remote VPN server
supports the failover conditions you are using.

You can check the status of the dead gateway detection service from the Advanced Shell with
the command: service -S | grep dgd.

The issue might be caused by a misconfigured secondary VPN. When the primary VPN was
cloned, the IP address of the remote VPN server was not updated, so both VPNs in the failover
group were trying to reach the same VPN server on the same IP address. When that connection
failed neither VPN would work.

Route-Based IPsec Connections

When troubleshooting route-based, or tunnel IPsec connections, there are some additional logs
that may be useful.

The first is /log/applog.log which shows the creation, connection and disconnection of the xfrm
interfaces.

The second log is /log/xfrmi.log which contains information about the interface configuration
and routing.

Remote Ethernet Devices (RED)

Log Files
There are two log files on the Sophos firewall that should be checked when troubleshooting
issues with RED which are: /log/csc.log and /log/red.log.

Cannot Enable RED on Sophos Firewall

You can see an error that indicates a problem accessing the provisioning service on port 3400.

This error can be seen in the csc.log.

To troubleshoot this issue, you need to test the connectivity from the Sophos firewall to the
provisioning server, red.astro.com. This can be done from the Advanced shell using the telnet
command: telnet red.astro.com 3400.

If you see a ‘Connection timed out’ error, it indicates that something is blocking access to port
3400 most likely an upstream router or gateway.

If you see a ‘Name or service not known’ error, it indicates that there may be an issue with then
DNS configuration on the Sophos firewall.

RED Cannot Connect

Troubleshooting a RED that cannot connect may require someone at the remote location, but we
will start with the steps that can be completed on the Sophos Firewall.

On the Advanced Shell, check that the RED services (red_client & red) are running using the
command service -S | grep red.

If they are not running use the command: service red: start -ds nosync.

If the services are running, verify that the red_server process is running with the command ps |
grep red_server.

The next step is to perform a packet capture for traffic on port 3400 to see if the RED has been
able to reach the Sophos Firewall.

The RED device uses the following ports: TCP port 3400 and UDP port 3410.

The next steps should be completed at the remote location of the RED.

Test that there is access to the provisioning server using the command telnet red.astaro.com
3400, and the Sophos firewall.

telnet <Sophos FIREWALL FQDN/IP ADDRESS> 3400.

If the RED configuration uses the hostname of the Sophos firewall, be sure to use that in your test
to check that it can be resolved at the remote site.

The picture below shows the operating guide that covers the startup process of the RED and
what the lights and messages shown mean. This can be used to identify possible causes for
being unable to connect.

The lights meaning and indications can be looked up on the Sophos website.
Azure AD SSO
Log Viewer
The Azure AD SSO login to the web admin console log entries can be found int the ‘Admin’
module on the log viewer. This will show successful authentication logins use Azure AD SSO.
Unsuccessful logins happen in Azure AD and so cannot be logged here.

Azure AD SSO logins to the captive portal are shown in the ‘Authentication’ module of the log
viewer.

There are 2 log files for Azure SSO, one for each portal where it is supported: the web admin
console and the captive portal: /log/oauth_sso_webadmin.log and
/log/oauth_sso_captive.log.

Here you can find the extended details from the login, and the details of any errors that can occur
after the administrator has authenticated in Azure and been redirected to the Sophos Firewall.
Services
As the log files, there are separate services for each of the supported portals.

If you use the ‘service -S’ command, note that the service name is truncated to 15 characters.

service oauth_sso_webadmin:status -ds nosync and service oauth_sso_captive:status -ds

nosync are the commands used to check the status of the Azure authentication services.

Client Authentication Domains

Domains for authenticating with Azure AD are:
• login.microsoftonline.com
• *.aadcdn.msftauth.net
• *.aadcdn.msftauthimages.net
• *.aadcdn.msauthimages.net
• *.logincdn.msftauth.net
• login.live.com
• *.msauth.net
• *.aadcdn.microsftonline-p.com
• *.microsoftonline-p.com

When a user is authenticating with Azure AD, they need to be able to access the domains shown
here for the authentication to work.

To enable this, you will need to create a firewall rule above the firewall rule that requires
authentication that allows users access to these resources.

Internal Server Error

Once an administrator has authenticated with Azure and is directed back to the Sophos firewall,
they might see an interval server error 500.

If you look at the logs, we can see the error message ‘insufficient privileges to complete the
operation.’

As the error references permission, the first place to check is the API permissions in the app
registration in Azure AD.

To allow the correct information to be passed back to Sophos Firewall the API permissions
should also include the ‘User.Read.All’ delegated permission and ‘Group.Read.All’ delegated
and application permissions.
We also recommend granting admin consent for the permissions so that administrators are not
prompted when logging in.

Please note that the ‘Group.Read.All’ delegated permission is only required if you are using
groups to assign users to the role, and the ‘Group.Read.All’ application permission is used for
group import.

Mismatching Redirect URI

The error message for mismatching redirect URIs is ‘the redirect URI specified in the request
does not match the redirect URIs configured for the application.’

In the app registration configuration, we can check the configured URIs.

On Sophos firewall, you can check the redirect URI you have configured in the Azure AD SSO
authentication server.

This is the URL that the administrator will be redirected to after authenticating with Azure AD.

You can modify the hostname or IP address that will be used in the redirect URL. Copy the
redirect URL from the Sophos firewall web console under CONFIGURE > Authentication >
Servers > Redirect URI.

In Azure AD, update the redirect URI to match what was in Sophos Firewall.

Note that you can have multiple redirect URIs; however, each Sophos firewall can only have one
redirect URI for an Azure AD SSO authentication server.

Not Assigned to a Role

The error message will be ‘You don’t have permission to sign in to the firewall. Contact the
firewall’s super administrator.’

If we look in the logs, we can see the roles that the administrator is assigned to. If it is ‘null’ it
means the user is not assigned to any role.

Review the role in Azure AD and assign the administrator role the specified user.
Authentication Failure
Causes of Authentication Failure
1. The external authentication server has not been configured in the Sophos Firewall.
2. The external authentication server is not available.
3. The time profile is preventing access.

User Cannot Authenticate

When a user tries to log into the captive portal, they may see the error message. ‘Login failed.
Invalid username/password. Please contact the administrator.’

If the user is sure that they have entered their details correctly, the first thing to do is to review the
log file. You can do this using the log viewer and selecting the Authentication module.

Alternatively, you can check the file /log/access_server.log in the Advanced Shell.

The important thing we can take from this log entry is that the auth_mechanism is ‘Local’. This
means that the Sophos firewall is trying to authenticate the user against only local users. In most
deployments users are authenticated using an external source such as Active Directory.

In the firewall console navigate to CONFIGURE > Authentication > Services > Firewall
authentication methods to enable authentication from the failing authentication server.

You can reorder the selected authentication servers in the firewall web console by dragging and
dropping them.

If the user fails to login and the log file it shows auth_mechanism is ‘Loca,AD’, showing that the
Sophos firewall is trying to authenticate the user against both local users and the external Active
Directory server. In this instance, the next step is to check the Active Directory server.

Check the configuration of the AD server. You can do this by using the Test Connection button. If
the server cannot be reached it can either be caused by misconfiguration, the server being down
or another firewall or router blocking access.

If you have multiple domain controllers, we recommend adding more than one to Sophos firewall
for redundancy.
If the error message says ‘Login failed. Access not permitted at this time.’ This means that an
access time profile is limiting the times at which the user can login.

In the log file you will see an error that says, ‘Access Time Policy violation.’

There are two places that access time policies can be applied for a user, directly in the user or in
the group.

To allow the user to login, a different access time policy will need to be set, either at the user
level or at the group level. Changing it to ‘Allowed all the time’ will resolve the problem.

If you are still seeing authentication issues check that the authentication service
(access_server) is running using the command: service -S | grep access_server.

To start the service use the command service access_server:start -ds nosync.

Captive Portal
The captive portal can be used to authenticate users through the browser when they try to
access resources.

Captive Portal not Displayed

The captive portal is used to authenticate users accessing servers in the intranet zone if the user
is not already authenticated to the firewall.

If the captive portal is not being displayed, first you should check that the captive portal is
enabled for the zone the user is accessing the network from in the device access settings
located in SYSTEM > Administration > Device access.

If the captive portal is enabled for the zone, check that it is enabled in the firewall rule that the
traffic is matching.

The option ‘Use web authentication for unknown users’ enables the captive portal.

If the captive portal is still not being displayed, check the status of the authentication service.
This can be done in web admin console in CONFIGURE > System services > Services.

You can also check the status of the authentication from the Advanced Shell. The authentication
service is called access_server.
Synchronised User ID
User not Authenticated
Synchronised user identity leverages the already deployed Sophos central software and Security
Heartbeat to provide simple, robust user authentication with Sophos firewall.

The log files needed to examine for Synchronised User Identity are /log/heartbeatd.log and
/log/access_server.log which are communication between endpoint and firewall and
authentication logs respectively.

Heartbeat service debug logging

• Debug level 1: service -t json -b ‘{“debug”:1}’ -ds nosync heartbeat:debug
• Debug level 2(trace): service -t json -b ‘{“debug”:2}’ -ds nosync heartbeat:debug
• Debug off: service -t json -b ‘{“debug”:0}’ -ds nosync heartbeat:debug

To see the detail behind what is happening you will need to enable debug logging for both the
heartbeat and access_server services.

The heartbeat service supports 2 levels of debug (debug and trace). In most cases debug level 1
will be sufficient.

In the access_server.log you can see the request being processed. It shows the username,
domain name and IP address being extracted from the login request. It also shows that the error
message e.g. ‘Invalid username/password’ error.

This is a common issue is where multiple domains are being used, or where the default domain
is not used.

The solution to this issue it to create additional authentication servers for each domain being
used.

As all of the domains will be using the same domain controller(s), you will need to use a unique
IP address or hostname when defining the external server on Sophos firewall, as only one
authentication server can be created per IP or hostname.

The simplest solution is to create multiple A records in DNS for the domain controller.

The NetBIOS domain will most likely remain the same, as will the search queries, it is only the
‘Domain Name’ field that needs to be updated.

Remember to enable the new authentication server on the Services tab so that it will be used to
authenticate users.
If we review the access_server.log and filter for lines containing the username of the person
logging in, we can see them being authenticated successfully and the user being inserted into
the live users.

STAS
STAS Communication
It provides single sign-on for Windows domain users using software installed on every domain
controller.

When a user logs into the domain the login details are written to the event log with ID 4768.
Please note that auditing of login events must be enabled on the domain controller.

The Sophos Transparent Authentication Suite Agent (STA Agent) on the domain controller uses
hooking to monitor the event log.

The agent reads the login information and passes it to the STA Collector on port 5566. This can
either be on the same server or a separate server.

The STA Collector maintains a list of live users and sends the login information to the Sophos
Firewall on port 6060.

An alternative deployment of STAS can have the STA Agent installed on a separate server to the
domain controller, where it will poll the event log to get the login events. Apart from the Agent
polling the domain controller, the process works in the same way.

If a user tries to access the network, and they are not in the live users list on the Sophos Firewall,
the Sophos firewall will query the STA Collector on port 6677 to get the users’ details.

The STA Collector will poll the computer to get the user session information. This can either be
done using Windows Management Instrumentation (WMI or reading the registry).

TCP ports 135 and 445 should be accessible on the computer from the STA Collector for this
polling to work.

STA Client Logs

C:\Program Files (x86)\Sophos\Sophos Transparent Authentication Suite\logging_events.log
The STAS software has two logs. The first, logging_events.log, logs the information read from the
event log on the domain controller.

The log files can be opened from the Advanced tab on the STAS software.

The other log, stas.log, logs the interaction between the agent, collector and Sophos Firewall.

C:\Program Files (x86)\Sophos\Sophos Transparent Authentication Suite\stas.log

On the Sophos firewall, you should see an entry in /log/csc.log when STAS reports a logon or
logoff.

STAS Service Fails to Start

Failed. Can not start service.: STAS
Fatal Error: The service did not start due to a logon failure.
The first issue you might encounter is the STAS service being unable to start and the error above
is shown.

To ensure the STAS service account has the correct permissions, it should be added to the
Backup Operators and Event Log Readers Groups in AD, and the local Administrators groups on
endpoints, (which can be done via a group policy and is required for WMI logoff detection to
work).

The account should also be granted ‘Logon as a service’ permission on the domain controller,
and the Full Control NTFS permission on the STAS folder.

The most common cause is that the account does not have ‘Log On As A Service’ permission on
the server it is being installed on.

In the properties of the STAS service, re-enter the account details in the Log On tab, and click
Apply. Windows will then grant the required ‘Log On As A Service’ permissions.

User is Not Authenticated

In an event the user is not being authenticated by STAS, the user may find that they cannot
access network resources, or if captive portal is enabled, they may be prompted to authenticate.

Starting at the beginning, when a user logs into the domain an event should be written to the
Windows event log of the domain controller.

In the case where there are multiple domain controllers you will first need to check which
domain controller they logged into.
You can do this using PowerShell to query WMI on the computer. On the logged in machine, you
should be able to find a corresponding event log entry for the user authentication.

If there is no log entry, check that auditing of successful logins is enabled. This can be found in
the Local Security Policy in Security Settings > Local Policies > Audit Policy > Audit logon
events.

When deploying STAS we recommend enabling auditing of logon events through Group Policy for
all domain controllers.

If the event log entry is being created, check the STAS log file on the STA Agent to ensure it is
being picked up.

If you see the message where dca_filter_by_authnet is filtering out a workstation IP, it may
indicate a misconfiguration in the monitored subnets.

Check the monitoring subnets on the STA Agent and in the top entry it should read the LAN
subnet. You can also check that the STA Collector IP is correct.

If the STA Agent is getting the logon event, check that the agent is able to contact the STA
Collector. You can do this in the Advanced tab of the STAS software. Enter the IP address of the
Collector and click Test.

If the test fails, check that the Agent is configured with the correct port settings for the Collector,
in case they have been customised. Also check for any router or firewall that could be blocking
access.

STA Collector has received the user logon event and added it to the Live Users. This can be
accessed from the Advanced tab of the STA Software.

Check the stas.log on the Collector to ensure that the logon event has been sent to the Sophos
firewall.

If the logon event is not sent to the Sophos firewall you may see the entries shown here. This
indicates that there is a subnet filter for the networks that will be authenticated to the Sophos
firewall.

Only users that are authenticated within the filtered subnet will be logged into the Sophos
firewall.

While you are on the Collector you can also test the connectivity to the Sophos firewall from the
Advanced tab on the STAS software.

On the Sophos firewall you can use tcpdump to check that the logon event is being received
from the Collector.
You should also see an entry in the csc.log for the user.

You can check that the user is associated with the IP address of their computer by checking the
ipset in the Advanced Shell. ipset -L lusers is the command.

If the user is not being authenticated, confirm that Clients is enabled under Authentication
services for the zone in device access on the Sophos firewall web console.

The next step is to check the Collector group configuration.

Collector groups are a way of providing redundancy in the development. However, only one of the
Collectors is active at a time.

In a scenario where the full STA suite (Agent & Collector) is installed on the domain controllers
they must be in separate Collector groups so that logon events for both are processed by Sophos
firewall.

In larger environments it may be that the Identity probe for users that are not in the live users
table is timing out. In most environments this should not be set lower than around 45 seconds,
but larger environments may require longer. Default timeout is 12o seconds. Larger
environments may require larger timeouts.

Users Disconnected
You may see the same symptoms when a user is getting disconnected from STAS. This can be
caused when the Collector is unable to poll the computer.

You can test whether the collector is able to poll a computer from the Advanced tab in the STAS
software.

Check that TCP ports 135 and 445 are accessible from the Collector.

You can perform separate tests for the WMI verification and registry read verification to match
what you are using.

Multi-Factor Authentication
Users Cannot Authenticate
Sophos firewall supports multi-factor authentication using one-time passwords.
There are different types of one-time passwords. You can use either software tokens, such as the
Sophos Intercept X App that is available for Android and iOS, or hardware tokens, if they conform
to RFC 6238.

Authentication problems with one-time passwords are almost always caused by a time
difference between the Sophos Firewall and the device with token, usually a mobile phone.

In /log/access_server.log you will see that the OTP token is rejected because it is a bad code, or
the token is not active.

First check that the token is enabled in the web console under CONFIGUR > Authentication >
Multi-factor authentication > Issued tokens.

If the token is enabled and you are seeing an error, compare the time on Sophos firewall with the
device the token is being generated on for the user.

If they are not in sync, correct the time.

It may not always be possible to correct the time on the token. Alternatively, you perform the
following steps:

1. Click on the OTP time-offset synchronization icon for the token

2. Enter the current token code and click Check.
3. Sophos firewall will show how far out of sync it is. Click Apply to set the offset for the
token.

Additional OTP Codes

As a temporary workaround you can also add additional one-time codes to the token. These can
be given to the user so that they can login even if their token is not working or they have lost it.

These codes should be communicated to the users in a secure manner. It is important to note
that the codes do not expire until they are used or unless an administrator manually removes
them.

Web Polices
Unexpected Policy Action
The most common issue encountered with web policies is when a website, or part of a webpage
is blocked by a policy. This may be intended, but it could be a site that users need to access but it
is included in a category that is blocked.
You should first review the web filter log in the log viewer. You can filter this using username, IP
address and URL to help you find the relevant entries.

By hovering over the log entry, we can find more information. We can see the firewall rule, and
web policy IDs, and we can see that the category was part of the ‘Unproductive Browsing’ user
activity.

In the advanced log view, you can click on the web policy ID to open it in the parent web admin
console window.

There is no separate rule in this web policy for the Social Networking category, but from the log
we saw that it was part of the Unproductive Browsing user activity.

There are several ways to resolve this issue, depending on whether you want to allow the site for
all users or a subset of users. Here we are removing Social Networking from the Unproductive
Browsing user activity.

When we save it there is the option to ‘Save for all’ which will update the original user activity, or
‘Save copy’ so that other policies using this user activity are not affected.

If you want to allow Social Networking for a subset of users, you could also create a web policy
rule above the Unproductive Browsing rule to explicitly allow Social Networking for the desired
users and groups.

Web Scanning
Logging for DPI Scanning
If web protection is configured to use DPI scanning, the log viewer will provide useful information
for troubleshooting, and this should always be the place to start.

If you are not able to determine what is happening to resolve the issue from the log viewer, you
may need to enable web debug logging for the DPI engine.

The DPI engine relies on Snort and the IPS service, and so it is in the IPS service that debug
logging is enabled for web scanning.

When you enable debug logging, it can log such a vast volume of data that it makes it hard to see
what is going on.

Before enabling the debug logging, you can optionally create a mask that will enable debugging
for only specific components, allowing you to focus your troubleshooting.
To create a mask, you echo the values for the components into a configuration file using the
Advanced Shell. You then need to toggle debug logging.

Please note that this uses debugp instead of plain debug. This will then log to the file
/log/ips.log.

Slow Browsing Through Web Proxy

When using the web proxy, it is expected that there is a reduction in throughput of up to 25%.

When troubleshooting reports of slow browsing, start by comparing the throughput with and
without the web proxy.

The advanced shell can be used for testing. The first command (curl -x http://172.16.16.16:3128
-L http://ipv4.download.thinkbroadband.com/100MB.zip > /dev/nul) configures proxy settings
before using curl to download a .zip file. The second command (curl -x
http://172.16.16.16:3128 -L http://ipv4.download.thinkbroadband.com/100MB.zip >
/dev/nul) downloads the file without a proxy.

To get more detailed information, enable debug logging for awarerenhttp – this is the web proxy.

Transactions will be logged to /log/awarrenhttp_access.log

service awarrenhttp:debug –ds nosync to check the status of the service.

For each transaction you can see how long different parts of the transaction took. These are
measured in microseconds.

The five times recorded for transactions are:

• authtime – the time for authenticating the request. It should not exceed 5 million
microseconds.
• dnstime – the time for DNS resolution. This should not exceed 50 thousand
microseconds.
• attime – the time for identifying the category of the site. This should not exceed 1 million
microseconds.
• avscantime – the time for malware scanning. This can depend on the size of the content.
• fullreqtime – the total time to process the request. This can depend on the size of the
content.

If you are seeing a high authentication time, check the configuration for authentication servers.
For each authentication server, check the connectivity and responsiveness and ensure that the
most responsive is at the top of the list.
Check the resource utilization on the authentication servers. If they are overloaded this could
cause delays.

If you are seeing a high DNS time, check the time to resolve various domains using the Sophos
firewall. Try to include domains that are unlikely to be in the cache to get the most rea-world
figures.

You can do this from the advanced shell with the command: host -a <domain> <Sophos
Firewall IP address>

After the issue has been resolved, the time with the proxy should be within 25% of the time
without the proxy.

Web Categorization
Categorization Process
Sophos firewall uses the Sophos eXtensible Lookup service to categorize URLs for both DPI
scanning and the web proxy. Sophos firewall also uses this service for IP reputation in email
protection and web server protection.

To use SXL, Sophos firewall needs to be able to access 4.sophosxl.net on port 443. When
Sophos firewall uses SXL for a lookup, the SXL service (nSXLd) first checks the local cache.

If there is no answer in the local cache or it has expired, it will check against local custom
categories. If there is no match, it performs a cloud lookup and caches the response.

You can find the category for each web request in the web filter log in the log viewer.

Categorization Logging
service nSXLd:debug -ds nosync to check the status of the service.

tail –f /log/nSXLd.log to view the log file.

If you are seeing an issue with categorization you may need to enable debug logging for the
nSXLd service to see what is happening.

SXL responds with the category information that will be used to allow or deny the traffic based on
the policy.
Testing Categorization
When investigating categorization, another useful tool is the Policy Test. This is available in
separate tab in the log viewer.

The policy test allows you to quicky and easily test URLs to show how they are classified. It will
also indicate whether the DPI engine or web proxy is being used for the matching firewall rule.

Category Reassessment
If you encounter a URL that you think is being miscategorized, you can submit it for
reassessment from the Sophos website.

When doing this, include the URL being accessed, the category being reported on the Sophos
firewall and the expected category for the URL.

Incorrect Action for Website

Check the log viewer. Some URLs may be categorized as either ‘None’ or ‘Uncategorized’ by
Sophos. To manage these instances, you can use the URL groups to assign actions for the
specific domains or URLs.

Alternatively, you can set an action for the ‘None’ and ‘Uncategorized’ categories in your policy.
You can also choose to submit the URL for assessment.

Application Control
Cannot Use Application
Application control does not display a block page or error message.

In the application filter log in the log viewer, you will see the application being denied, the
application category and the policy that blocked the application.

Using the advanced view in the log viewer you can click on the policy ID to open it in the parent
web admin window.

There are a couple of ways to allow the application; the first is to create a new rule in the
application filter policy to specifically allow the application. You will need to ensure that it is
above the rule that is blocking it.
The other option is to use the ‘Select individual application’ option in the rule that is blocking
the application so that you can deselect it. The disadvantage to this approach is that the
applications in the rule will no longer be automatically updated based on the selected filters.

Remote Access
Sophos Connect Client Log Files
When troubleshooting remote access VPN issues, you will need to collect the logs from the
client. For Sophos Connect, the log file can be opened from the menu.

You can also download a zipped bundle of the configuration and log files. Open About from the
menu then click Generate technical support report.

Sophos Firewall Log Files for IPsec VPNs

On Sophos firewall, the relevant logs will depend on whether it is an IPsec or SSL VPN. For IPsec,
the log file is /log/strongswan.log; this is the same as the IPsec site-to-site VPN.

Sophos Firewall Log Files for SSL VPNs

For SSL VPNs, the log file is /log/sslvpn.log

Issues Using Default IPsec VPN Configuration

When the default IPsec configuration that is generated is used, and a lot of users start
connecting, sometimes there will be a larger than expected impact to the bandwidth utilization
and Sophos firewall performance.

This can be because the default configuration tunnels all traffic back to the Sophos firewall
including all internet traffic.

In the advanced section of the IPsec configuration, you can disable ‘Use as default gateway’
and define split networks that will be routes over the VPN.

There are also several other options in the advanced section, including for security heartbeat,
two-factor authentication, and running logon scripts.

The updated configuration file will need to be imported into the Sophos Connect client.

If split tunnelling has been configured, you also need to ensure that you are using the .scx file,
which is the only file that contains the advanced configuration. The .tgb file will always tunnel all
traffic back over the VPN.
With split tunnelling enabled, only selected traffic is tunnelled back to the Sophos firewall
reducing the impact on bandwidth and performance.

Cannot Connect to SSL VPN

If we check the Sophos Connect log file, you might see an error connecting to the Sophos
firewall. Check the IP address or hostname and port number shown if they match those in the
global SSL VPN settings.

If the IP address and port number are correct, check that SSL VPN is enabled for the WAN zone. If
it is not enabled, then enable it. If it is enabled, you should also check that there is not a local
ACL exception rule that is blocking access to it.

Access Point Deployment

The troubleshooting issues with access point deployment it is useful to have a clear picture in
your mind of the steps in the process so that you can walk through it to identify the root cause.

1. When you connect an access point to the network it must be able to configure itself using
DHCP.
2. The access point will then send a discovery packet to the magic IP of 1.2.3.4 on port 2712
– the management port for wireless.
3. The Sophos Firewall intercepts the discovery packet and responds.
4. An administrator needs to login to the WebAdmin to manually approve the access point
in the pending list.
5. The access point will be moved to the active and inactive access points list, but it will be
in the inactive state.
6. The access point will then update its firmware from the Sophos Firewall and reboot.
7. Once the access point has started it will get the configuration from the Sophos Firewall.
8. The access point can now become active and start broadcasting the configured networks

Scenario 1
The access point is not in the pending access point list.

The first step in the access deployment is that it will configure itself with DHCP, so the first thing
to check is that the access point has received a DHCP lease.
On the Sophos firewall you can do this in CONFIGURE > Network > DHCP. You should also see
an entry in the log viewer in the system log. You can filter for the DHCP server log component.

If the Sophos firewall is not in the DHCP server, check the lease table for the DHCP server.

You can find the MAC address of the access point on the label of the device.

If the access point has received a DHCP lease then the next step is to check for traffic on port
2712, the management port.

If it is being dropped as invalid traffic it means that the traffic does not match either a firewall rule
or device access rule.

Check the device access table to the status of the wireless protection settings.

Scenario 2
The access point is not in the pending access point list.

Here you can see that a packet capture of port 2712 doesn’t return any traffic when the access
point is connected to the network.

This may be caused by a router or device between the access point and Sophos firewall blocking
port 2712.

You can test this by trying to connect to the Sophos firewall on port 2712 using telnet from where
the access point has been plugged in.

If it fails, you can traceroute to help identify the devices along the path to the Sophos firewall
using the command tracert -d 172.16.16.16 (using the option -d to make sure the IP addresses
are not resolved to names).

Scenario 3
The access point is not in the pending access point lists.

No traffic will be shown in the packet capture. When the telnet test is done it works.

You can use traceroute to test the route to the magic IP address 1.2.3.4 using the command
tracert -h 2 -d 1.2.3.4 (using -h 2 to limit the maximum number of hops checked). We can see
that the last hop is not the IP address of the Sophos firewall.

The access point uses the magic IP address 1.2.3.4 to communicate with the Sophos firewall. If
the Sophos firewall is not on the route to the internet, for example if it is being used as web proxy
or VPN concentrator, but not the default gateway, the Sophos firewall cannot intercept this
traffic.

This can be resolved by adding a route to a device along the path to send traffic for 1.2.3.4 to the
Sophos firewall.

You can also resolve the issue by providing the IP address of the Sophos firewall in the DHCP
options.

Please note that if the Sophos firewall is being used as a DHCP server it will include its own IP
address in the DHCP options.

For example, on a Windows Server you can define a new option.

• Right-click on the IPv4 server and select Set Predefined Options…

• Click Add.
• Provide a name for the option, we have used Firewall here.
• Set the ‘Data type’ to IP Address.
• Enter 234 as the ‘Code’.
• You can then configure the value as the IP address of the Firewall.

Wireless Networks

Scenario 1
The first wireless network issue we will look at is where users cannot see the wireless network to
connect to it.

The first thing to check is if the access point has any wireless networks assigned to it. If it does
not, assign the wireless network.

If the access point is in a group, the wireless network that it broadcasts will be defined there. Add
any missing wireless networks to the group.

If the wireless network is assigned to the access point or group, the next step is to check the
configuration of the wireless network itself.

There is an option to hide the SSID of the wireless network. If this is enabled users must
manually enter the SSID that they want their device to connect to.
Hiding the SSID does not add any real-world security as it can still be found through sniffing
wireless traffic.

Another setting to check is whether time-based access is enabled. If so, the schedule may be
incorrectly configured to prevent access to the network at specific times.

Scenario 2
Another issue might be that the user cannot connect to a wireless network, even if they are
entering the correct details and it is working for other users.

The most common cause of this is a misconfigured MAC address block list.

Wireless Performance
Coverage – poor location can leave holes in coverage that have a weak signal.

Interference – can come from other electrical devices, or access points being deployed too
densely.

Crowded frequencies – in built-up areas frequencies can become crowded.

If you are having issues with wireless performance the three things to check are coverage,
interference and frequency utilization.

If access points are not located well, you can end up with holes in coverage that have a weak
signal. This can along lead to too many clients trying to connect through a single access point.

Having access points located too densely can cause interference. Interference can also come
from pretty much any electrical device, including wireless phones and microwaves.

In built up areas, frequencies can become crowded with adjacent office’s wireless networks.

To form a good picture of what is happening you can use tools such as InSSIDer (Windows, Mac)
or Wi-Fi Analyzer (Android) to scan for wireless networks.

You will also be able to see what networks are being broadcast and on which channels. You will
be able to see how strong the signal is for your network in different locations to help you site the
access points correctly.

In the case of crowded frequencies, you could either try selecting the broadcast channel
manually or have the access point perform scans and change channel dynamically.
When an access point changes channel, it will cause all clients to reconnect to the wireless
network. To limit this, you can schedule when these scans take place, for example once in the
morning between 8 and 9.

You may have a wireless deployment that uses a high density of access points where the
coverage overlaps to support a high number of users.

If your access point coverage is overlapping, you may need to reduce the transmission power to
improve performance by reducing channel interference.

Wireless Ports
• TCP:2712 - Access point management
• TCP:414/UDP:414 - RADIUS requests from the access point
• UDP:415 - SYSLOG from the access point
• TCP:4501- Hotspot portal
• TCP:8472 - VxLAN tunnel for separate zone configuration

AWE tool
For troubleshooting the most complex issues you may need to connect to the access point to
check the configuration and read the log, this can be done from the advanced shell of the
Sophos firewall using the awe tool.

Running the awe tool using the command awetool will present a menu that allows you to select
an access point to connect to.

Once you are connected you will see the command prompt for the access point. The commands
below can be run on the access point and the most useful one is logread.

• iw dev List wireless interfaces

• iw wlan# scan iw wlan# survey dump Run a neighbourhood scan
• iw wlan# station dump Return connected stations
• iw phy State and information about the radio
• logread Connection status of the access point to XG Firewall

logread will output the local access point log file which contains detailed information about the
operation of the access point and the clients connecting to it.
Web Server Protection
Refining the Configuration
Place the Protection Policy in Monitor mode while refining the configuration. When deploying
web server protection, we recommend initially configuring it for the highest level of security,
however, this increases the risk of false positives.

To allow you to identify and resolve these issues, first test the configuration in monitor mode,
then review the log file and update the configuration. You will need to repeat this until the logs no
longer show errors.

You can then switch the protection policy into reject mode, retest and review the log files and
make any final adjustments to the configuration.

Log File
The web server protection log file can be viewed in the log viewer; however, this does not always
provide the detail necessary to accurately troubleshoot the rules being triggered.

The full web server protection log file, reverseproxy.log, can be accessed in the advanced shell.

You can use the command tail -f /log/reverseproxy.log to follow the output of the log, with the
new entries being written to screen.

URL Hardening
Static URL Hardening
Static URL hardening prevents users from manually constructing deep links that lead to
unauthorized access. When a client requests a website, all static URLs of the website are signed
using a hash of the URLs and a secret only known to the Sophos firewall.

In addition, the response from the web server is analysed regarding which links can be validly
requested next.

To configure static URL hardening, you specify the URLs you want to serve. These URLs can be
accessed without requiring a URL hardening signature. The URLs added here are case sensitive.

This isn’t effective for dynamic URLs created by the client, for example, using JavaScript, and
these will require exceptions to be created.
Page Cannot Be Accessed
In the log viewer you can see a reason for a page being blocked being ‘Static URL Hardening’ and
that no signature was found. We can see the URL that was being accessed, and the firewall rule
ID.

The same information is reported in the reverseproxy.log in the Advanced Shell.

If we check the static URL hardening configuration in the protection policy, you may need to
check the firewall rule to see which protection policy is applied, we can see that it contains two
URLs.

As the entry URLs list is case sensitive, it is important to include upper and lower-case URLs that
the user may use if they are both valid on the web server.

In this case we add a lowercase copy of the URL to the entry URLs.

Form Hardening
The Sophos firewall saves the original structure of a web form and signs it. If the structure has
changed when the form is submitted, the Sophos firewall rejects the request.

Error Submitting a Form

A user may submit a form and get the error message ‘Your browser sent a request that this server
could not understand.’

In the log viewer we can see that this is a form hardening issue and that the form hardening token
is invalid. This is generally caused when a form is generated dynamically.

We can see the URL and the firewall rule involved with this problem, but there are also other bits
of information, such as the PHP session ID that can be useful if you need to check the logs on the
web server. There will be several entries in the reverseproxy.log for this event.

For form hardening errors you will need to create an exception in the firewall rule.

If we look at the URL where the error occurred, we can see that it includes a parameter that is
variable. To account for this, we need to include a wildcard in the path we create the exception
for.

In the exception configuration you add one or more paths for the exception to apply to, select
which checks should be skipped, and for this issue we also need to select the options in the
Advanced section. Never change HTML during static URL hardening or form hardening. When
this option is selected, no data matching the defined exception settings will be modified by
Sophos Firewall. For example, binary data wrongly supplied with a text/HTML content type by the
web server will not be corrupted. On the other hand, web requests may be blocked due to
activated URL hardening, HTML rewriting, or form hardening. Those three features use an HTML
parser and therefore to some extent depend on the modification of webpage content. To prevent
undesired blocking, skip URL hardening and/or form hardening for requests affected by blocking;
you might need to do this in another/new exception to reflect dependencies between web
servers and/or webpages. Accept unhardened form data. Form Hardening exceptions rely on the
FH (Form Hardening) token, and if this is missing it will always block the data, even if the Form
Hardening check is disabled. This token is added to forms that are presented to the client
through the web application firewall. If the form is presented to the user through an external
source, but submitted through Sophos Firewall, the FH token will not be present, causing the
data to be blocked. With this option, forms submitted without the FH token will be accepted.

Threat Filter Rules

You can also filter strength, which determines how strictly you want to protect against attacks,
this goes from level 1, the most permissive, to level 4, the most restrictive.

• Level 1: Standard security requirements, produces minimal false positives. Provides a

standard level of security.
• Level 2: For better security coverage and higher security requirements, can generate
additional false positives. It provides additional protection, such as from regaxp-based
SQL and XSS injection and checks extra words for code injections.
• Level 3: For higher security requirements and based on your experience handling false
positives and turns on additional rules and keyword lists and it also sets additional limits
on the use of special characters.
• Level 4: For maximum security requirements, generates a high level of false positives and
places additional restrictions on special characters.

Level 1 is not logged. Levels 2 and higher are logged to /log/reverseproxy.log. When you
encounter a false positive you can add the rule ID to the skip filter rules fields to create an
exception.

Application attacks. Performs tight security checks on requests, such as attempts to traverse
prohibited paths.
Sophos Firewall also searches for attempted command executions common to most attacks.
After breaching a web server, an attacker usually tries to execute commands on the server like
expanding privileges or manipulating data stores.
Checking for these post-breach execution attempts allows Sophos Firewall to detect attacks that
may go unnoticed, for example, attackers targeting a vulnerable service after gaining legitimate
access.
SQL injection attacks. Checks for embedded SQL commands and escape characters in request
arguments.
Most attacks on web servers target input fields that can be used to direct embedded SQL
commands to the database.
XSS attacks. Checks for embedded script tags and code in request arguments.
Typical cross-site scripting attacks aim at injecting script code into input fields on a target web
server.
Protocol enforcement. Enforces adherence to RFC standards for HTTP and HTTPS protocols.
Violating these standards usually indicates malicious intent. Searches for common usage
patterns. The absence of such patterns often indicates malicious requests. These patterns
include HTTP headers, such as Host and User-Agent. Enforces reasonable limits on the number
and range of request arguments. Overloading request arguments is a typical attack vector.
Narrows the allowed usage of HTTP protocol. Web browsers typically use only a limited subset of
the possible
HTTP options. Disallowing the rarely used options prevents attacks that use these options.
Scanner detection. Checks for usage patterns characteristic of bots and crawlers. When you
deny them access, possible vulnerabilities on your web servers are less likely to be discovered.
Data leakage. Prevents web servers from leaking information to the client. This includes error
messages sent by servers, which attackers can use to gather sensitive information or detect
specific vulnerabilities.

Forbidden Error
When posting on a website the error displayed is ‘You don’t have permission to access this
resource.’

In the log viewer we can see a WAF anomaly error, the URL and firewall ID, however, we cannot
identify from this which threat filter rule(s) were triggered.

In the reverseproxy.log there are several log entries related to this event.

If a threat filter rule has been triggered, we may see the message ‘Invalid character in request’.

The next log entry log entry looks very similar, but it is a rule that causes the traffic to be blocked
based on the threat filter rules that have been triggered. Skipping this rule will bypass a wide
range of protection. You can identify these rules as they refer to anomaly scores being exceeded.

To resolve the issue the threat filter rule can be added to the skip filter rules in the protection
policy. You may need to check which protection policy is being used by the firewall rule.
GeoIP Blocking
When blocked by GeoIP, users will see a 403 forbidden error.

In the log viewer you will see an entry that shows the request and the 403 status code.

In the reverseproxy.log you will see an ‘authz_core’ error with the message ‘client denied by
server configuration’.

Note that this is the same message that you see for the ‘Allowed client networks’ configuration
and will need to be verified manually when troubleshooting.

To help troubleshoot access issues related to GeoIP blocking, you can lookup what country the
IP address is resolving to from the console using the command show country-host ip2country
ipaddress.

You will need to check the firewall rule configuration in the ‘Access permission’ section. In this
case the ‘Blocked countries’ field was misconfigured.

Web Server Authentication

Web server authentication can be enabled to force users to authenticate before they can even
reach the web server. This provides great protection for login forms and so forth.

When a user connects to a website the Sophos firewall prompts for the user’s credentials and
authenticates them.

If they are authenticated successfully the Sophos firewall, then requests the page from the web
server.

The Sophos firewall can also pass through the login credentials from the user so that they can be
logged into the web server.

Credentials can only be passed through to the web server using basic auth.

Web Server Authentication Configuration

Authentication policies control how the Sophos firewall implements web server authentication.

You can choose between presenting a form, which can be customized using a template, or using
a basic authentication prompt.

It is in the authentication policy that you define which users and groups can authenticate.
Choose whether to pass through the credentials, this can only be done using basic auth. You can
also optionally add a prefix or suffix to the username when it is passed through to the web server.

The authentication policy is set either in the web server protection firewall rule, or if path-specific
routing is enabled, per path.

User Cannot Authenticate

When logging into a website that has web server authentication enabled the user is prompted to
login again or sees an unauthorized error.

If we look in the log viewer, we can see that the user has been successfully authenticated. This
indicates that the issue is most likely on the web server and not the Sophos firewall.

The next step is to check the logs on the web server. It could be for example, that the web server
is using a different authentication source to the Sophos firewall. Even if the two sources should
be the same, for example different domain controllers in the same domain, it could be that they
are out of sync.

Another cause could be that the site does not support basic authentication and it need to be
enabled.

High Availability
• Both appliances must be the same model
• Both appliances must have the same firmware version
• The dedicated HA port must be in a DMZ zone
• SSH access must be enabled for the zone
• The WAN links should not have DHCP or PPPoE configured
• No alias should be configured on the dedicated HA port
• Link speed/duplex and MTU/MSS should be default on the dedicated HA port
• For active-active deployments, both devices must have a license

Deployment Process
When HA is enabled, both devices will start the process to form the HA cluster. This process is
completed in 3 steps by the primary device.
 1. Sanity Check
First the Sophos firewall performs a sanity check, this is done by the CSC service.

• Check model number and firmware

• Check HA is disabled on auxiliary device
• Check that all monitoring ports are up
• No alias, VLAN or MAC address overrides on HA port
• SSH is enabled on the zone of the dedicated port and zone type is DMZ
• Connects to port 22 on auxiliary appliance with passphrase

2. Preparation
The second step is to prepare the system for HA.

• Synchronize configuration file with auxiliary device (msync.conf)

• Synchronize connection tracking file with auxiliary device (ctsync.conf)
• Generate virtual MAC addresses and synchronize them with the auxiliary device
• Synchronize the original MAC address for both devices

3. Configuration Synchronization
The final step is to synchronize the configuration.

• Soft boot of the auxiliary device

• Auxiliary device joins HA
• Auxiliary device synchronizes the configuration from the primary device

The primary device will remain in a frozen state until the configuration is complete.

Log Files
In the log viewer you will find the HA logs in the SYSTEM module. You can apply a filter to the log
component to match on HA.

The HA log files can be found in the /log directory via the advanced shell.

• /log/msync.log - HA Synchronization Service

• /log/ctsyncd.log - Conntrack Synchronization Service
• /log/applog.log - HA configuration and status update
• /log/csc.log - Central service which manages all services
Unable to Activate HA
You can see the error ‘HA could not be enabled’ when trying to activate high availability.

Start by checking the following first:

• Verify that all HA prerequisites have been met

• Verify the passphrase used in the HA configuration
• Check connectivity to the auxiliary appliance LAN IP and dedicated port IP address
• Check the applog.log contains the results of checks carried out when enabling

Verify that all cables are properly connected.

When enabling HA you see the following messages in the applog.log:

• fwm:enableha successfully done - means that the sanity check has completed on the
local appliance
• enableha on peer done - confirms that HA is enabled on the peer appliance
• enableha: HA is enabled now - confirms that the sanity check has completed on the
peer appliance

Once the sanity check is performed on the appliances, the primary appliance will change its
state to 5:2 meaning it will become a standalone device first before primary-auxiliary state.

If we review the configuration on the auxiliary device, we can see that SSH has not been enabled
for the HA zone.

HA Console Commands
Once enabled, the HA status can be checked using the console command system ha show
details.

The system ha command can be used to:

• Disabled HA: system ha disable

• Enable and disable load-balancing in active-active clusters: system ha load-balancing
[on/off]
• Show the HA details: system ha show details
• Tail the HA logs: system ha show logs
Both Devices Become Primary Device
Once enabled hardware issues with ports and cables can cause problems. If the dedicated port
or cable fails, both appliances will become standalone primary devices and send GARPs to the
network switch to take ownership of the virtual MAC. This will likely result in routing issues.

In this case you will likely want to shut down or disconnect one of the devices to restore
functionality.

• Both devices think the other is faulty

• It also indicates an issue with the dedicated HA port or cable
• You will likely see routing issues as both device ARD for ownership of the virtual MAC

The dedicated HA port failure causes both appliances to become standalone primary devices.
In this scenario, you will need to shut down one of the devices, repair the link (assuming it is not
the interface itself) and then start the appliance.

It will detect the primary and take on the role of the auxiliary.

In the case where the devices are not directly connected, this issue could also be caused by a
misconfigured switch, or virtual switch if you are using virtual Sophos firewalls.

Primary Device Fails Over to Auxiliary

You can connect to the WebAdmin of the faulty device using the peer administration IP address
defined in the HA configuration.

In the control centre you may see some indication of the cause of the problem. In this case the
interfaces widget is red. Clicking on this will display further information about the interfaces. We
can see that PortB is reported as being unplugged. This could be because it is unplugged or due
to a faulty interface or cable. To verify if a failover is being caused by a defective interface or
cable, complete the following steps:

• Review the port status using dmesg e.g. dmesg | grep PortE
• Check and correct the speed and duplex settings on both sides of the connection if the
port is going up and down.
• Check the packet drops, errors, and collisions on the interface using ifconfig on the
advanced shell or show network interfaces on the console.
• Try replacing the cable.
• If the problem persists, use SF Loader from GRUB menu to perform an Ethernet card test.
Reports
The reporting process on the Sophos firewall is made up of four stages.

1. Sophos firewall subsystems

• Produce data that needs to be fed into the reports.
• Authentication server
• Proxies
• Firewall
• Vpn
• IPS
• Application filter

2. Garner
• Is the logging application on Sophos firewall
• Validates event threshold to indicate if thresholds have been reached
• Forwards database entries

3. Database Entry
• Sophos firewall uses 2 database engines
• PostgreSQL is the primary database and it records data
• SQLite is used to store the event data.

4. Reports
• Repots are processed and divided into sections so that the administrator can use
them to determine specific information about Sophos firewall.
• Dashboards
• Applications
• Network & threats
• Email
• Compliance
• Customs & special
• Settings

Reports Are Not Being Generated

Depending on what the root cause turns out to be, you may also see a lack of reporting data in
the Control centre.

If the Services icon is orange, it indicates there is an issue with one or more services.
Clicking on the orange Service icon in the control centre shows that the ReportDB service is
stopped.

You can also check the status of the services required for reporting in the advanced shell using
the service command and grep to filter the output.

The three services to check are postgres, garner and reportdb.

You can start any stopped services using the command service <name of service>:start -ds
nosync

In the console you can check that on-box reporting is enabled. This enabled by default but
should be checked. The console command is show on-box reports.

To enable on-box reporting use the console command set on-box reports on.

If the reports that are not returning records are in the past, check the log retention report period
in Data management to ensure that the data is available and has not expired and been deleted.

Check that there is space on the report partition. You can do this on the console with the
command system diagnostics show disk.

This will output three values for:

• Configuration, this is stored on the /conf partition.

• Content, this is stored on the /content partition.
• Report, this is stored on the /var partition.

You can also check the disk usage from the advanced shell with the command df -h.

In this case we are interested in the /var partition as that is where the reporting data is stored.

If there is low or no disk space on the /var partition, you can check where the disk space is being
used on the advanced shell with command du -sh /var/*

In this example we can see that /var/tslog is taking up most of the space. This directory is where
the logs are stored that are accessed from /log.

With log rotation and data retention settings on the Sophos firewall, this type of issue is most
frequently caused by debug logs and packet captures. It is important to always debug logging
when you have finished troubleshooting a service.

If you need to remove old reporting data from the Sophos firewall this is done in the Report
settings on the Manual purge tab.

You can remove reporting data either by reporting module or for all reports, for a custom duration
(recommended), or purge all reporting data.
We do not recommend purging all reporting data, as recent data may be required for auditing and
troubleshooting purposes.

Additional Tools
You can also check the service status for subsystems from the console using the command
system diagnostics show subsystem-info

This can be useful so that you do not need to switch back and forth between the console and the
advanced shell.

How to Find Help from Sophos

Should you need support, navigate to https://www.sophos.com/support to access
documentation, downloads, training and support packages.

The overview page provides quick access to the Support portal, documentation, technical videos
and to chat with our support agents or to view Sophos central status and the latest malware
information.

Documentation
Documentation, including product user guides, release notes, pocket guides, and other useful
information.

https://www.sophos.com/support/documentation

Knowledge Base
The Sophos knowledgebase, for technical documents on specific configurations and issues.

https://support.sophos.com/
Sophos Community
Using the Sophos community, you can reach Sophos staff for help, as well as participating in
discussions, and receiving assistance. This forum allows you to raise questions, share
knowledge, and discuss your experiences with Sophos products.

https://community.sophos.com/

SophosLabs
Sophos labs provide access to Sophos reports, real time data and Sophos threat reports.

https://sophos.com/labs

Threat Information
SophosLabs keeps a library of all known threats. You can search for a threat and view important
information such as a threats characteristic, or how it spreads. The threat library also includes
suggested instructions on how to remove the threat.

https://sophos.com/en-us/threat-center/threat-analyses/viruses-and-spyware.aspx

Sophos TechVids
Sophos provides a series of technical videos that cover configuration tasks, self-help,
remediation and how-to videos for common issues.

https://techvids.sophos.com/

Sophos Support
Create a Customer Case for:
• Access and support portal issues
• Licensing and ordering
• Updating contact information
• MFA resets

Create a Technical Support Case for:

• Issues with a Sophos product that you are unable to resolve
 • Advanced hardware replacements for appliances
• Software downloads or updating issues

For critical cases, we recommend that you create a technical support case first. Once you have
received the automated case number, follow-up the case with a call to the technical support
team.

Include the following when creating your case:

• Include any errors and symptoms

• Include the steps to reproduce the issue
• Include all troubleshooting steps completed
• Include all logs and additional information gathered

Sophos News and Alerts

We want to make sure you are aware of everything we are doing with our products, from tips to
updates and improvements. You can keep up to date with the latest alerts and news by visiting
our blog sites for our Sophos community and Sophos news.

You can also subscribe to our Sophos central status page for email and SMS alerts, follow
Sophos on X, and subscribe to our RSS feed. If a high-profile incident occurs, we publish
advisory banners to our support and community pages linking to applicable documentation,
knowledgebase articles and additional information.

Sophos News
Sophos news publishes the latest news about Sophos, our products and the latest information
for reporters who want to write about Sophos.

https://news.sophos.com/

SMS Notification Service

The Sophos SMS notifications service is a free of charge service that provides proactive SMS
alerting for Sophos products and services.

You are immediately prompted in the event an issue arises, so you will know exactly what is
happening, what the impact is and how to fix it.
You can sign up for the service and select the products you would like to receive alerts for. Once
configured you will receive instant notifications on technical issues or product updates. The SMS
message will contain the product name and a link to a knowledgebase article on our support
page where you can find more details.

Sign up for SMS alerts on https://sms.sophos.com/

Really Simple Syndication (RSS) Feeds

Really simple syndication is a format for delivering regularly changing web content. We syndicate
content such as our latest news, product advisories and virus alerts as RSS feeds that you can
load into your news reader.

https://sophos.com/company/rss-feeds

X
At Sophos, we use the X platform formally known as Twitter to help educate and connect with
partners, customers and interested prospects.

When we send out alerts via social media, it allows channel followers and users to search for
#sophos to find out the latest information.

Follow Sophos to hear about community solutions, news, articles, the latest product releases
and the latest issues.

https://twitter.com/sophossupport