Solution ID: sk33328 7/15/2016

How to clear $FWDIR/state/ directory to resolve policy corruption issues

Product: Security Management, Multi-Domain Management / Provider-1, Security Gateway

Version: All
Last Modified: 20-Jun-2016

Solution

DO NOT IMPLEMENT THESE STEPS for VSX Gateways - you will lose configuration scripts
that reside in the $FWDIR/state/ directories, causing more problems to the environment.

DO NOT clear the $FWDIR/state directory on 41000 or 61000 appliances. This will result
in deleting configuration files that are unrecoverable.

Perform the below steps on the Security Gateway and on Security Management Server / Multi-
Domain Security Management Server / Provider-1

Important Note: The following procedure does not apply to StandAlone or VSX Gateways.

Table of Contents:

• For Security Gateway managed by Security Management server

• For Security Gateway managed by Provider-1 CMA / Domain Management Server

For Security Gateway managed by Security

Management server
Note: Local console access to servers during this procedure is required.

1. Close all SmartConsole windows (SmartDashboard, SmartView Tracker, etc.).

2. Stop Check Point Services on the Security Gateway and on Security Management Server:

[Expert@HostName]# cpstop

3. Backup the current $FWDIR/state/ and $FWDIR/database/ directories on the Security

Gateway and on Security Management Server:

[Expert@HostName]# mkdir /var/tmp/statebackup

[Expert@HostName]# cp -rf $FWDIR/state/* /var/tmp/statebackup
[Expert@HostName]# mkdir /var/tmp/databasebackup
[Expert@HostName]# cp -rf $FWDIR/database/* /var/tmp/databasebackup

4. Delete the current content of the $FWDIR/database/ directory on the Security Gateway.
(Do not remove the $FWDIR/database/ directory itself)

[Expert@HostName]# cd $FWDIR/database/
[Expert@HostName]# pwd
[Expert@HostName]# rm -rf *

Be very careful executing these commands because in case you miss the / between FWDIR
and "database" you will delete all the file system of the appliance.

5. Delete the current content of the $FWDIR/state/ directory on the Security Gateway and
Security Management Server.

(Do not remove the $FWDIR/state/ directory itself.)

Be very careful executing these commands because in case you miss the / between FWDIR
and "state" you will delete all the file system of the appliance.

[Expert@HostName]# cd $FWDIR/state/
[Expert@HostName]# rm -rf *

Note: in a cluster environment, both cluster members should have these directories cleared
simultaneously to avoid potential corruption in policy and in configuration.

6. Start Check Point Services on the Security Gateway and on Security Management Server:

[Expert@HostName]# cpstart
 Notes:

• A message stating that the Security Gateway cannot get/fetch a Security Policy will be
displayed.
It can be ignored - this is a indication that the contents of the $FWDIR/state/ directory
on the Security Gateway and on Security Management Server have been cleared
properly.

When starting Check Point Services, the Security Gateway will install Security Policy in
the following sequence:
i. Install from $FWDIR/state/local/ directory on Security Gateway.
ii. If in cluster, then check for updated policy on peer member(s).
iii. Install from $FWDIR/state/ directory on SmartCenter / Security Management
Server.

• Since on Security Gateway the files have been deleted from local $FWDIR/state/
directory, Security Gateway will install 'Default Policy', that blocks all traffic from
passing through (/proc/sys/net/ipv4/ip_forward = 0).

7. Connect with SmartDashboard to Security Management Server.

8. Install the Security Policy onto this Security Gateway.

9. You will see fwm process on management consume high CPU, that its ok, please monitor it
through top command until the management server will build the policy again.

If you experience problems after this, use your backup to restore the previous
configuration:

1. Run 'cpstop' command.

2. Delete the content of '$FWDIR/state/' and '$FWDIR/database/' directories on the Security

Gateway. (Do not remove these directories).

3. Copy the contents of your backup, 'state' and 'database' directories on the Security Gateway
and on the SmartCenter / Security Management Server to their original locations.

4. Run 'cpstart' command.

For Security Gateway managed by

Provider-1 CMA / Domain Management
Server
Note:
When clearing the 'state' directory on Provider-1 CMA / Domain Management Server, it must be
cleared simultaneously on all Primary and Secondary CMAs/Domains, which manage the Security
Gateway. Otherwise, the contents of the 'state' directory may be repopulated from the Active
CMA/Domain.

Note: Local console access to servers during this procedure is required.

1. Close all SmartConsole windows (SmartDashboard, SmartView Tracker, etc.).

2. Stop Check Point Services on the Security Gateway:

[Expert@HostName]# cpstop

3. Stop Check Point Services on both the Primary and Backup CMAs/Domains:

[Expert@HostName]# mdsstat
[Expert@HostName]# mdsstop_customer <CMA/Domain_Name>
[Expert@HostName]# mdsstat

4. On Primary and Backup CMAs/Domains, switch to the context of the CMA/Domain, where the
'state' directory will be cleared:

[Expert@HostName]# mdsstat
[Expert@HostName]# mdsenv <CMA/Domain_Name>
 [Expert@HostName]# mcd

5. Backup the current $FWDIR/state/ and $FWDIR/database/ directories on the Security

Gateway and on the Primary and Backup CMAs/Domains:

[Expert@HostName]# mkdir /var/tmp/statebackup

[Expert@HostName]# cp -rf $FWDIR/state/* /var/tmp/statebackup
[Expert@HostName]# mkdir /var/tmp/databasebackup
[Expert@HostName]# cp -rf $FWDIR/database/* /var/tmp/databasebackup

6. Delete the current content of the $FWDIR/database/ directory on the Security Gateway.
(Do not remove the $FWDIR/database/ directory itself).

[Expert@HostName]# cd $FWDIR/database/
[Expert@HostName]# pwd
[Expert@HostName]# rm -rf *

Be very careful executing these commands because in case you miss the / between FWDIR
and "database" you will delete all the file system of the appliance.

7. Delete the current content of the $FWDIR/state/ directory on the Security Gateway.
(Do not remove the $FWDIR/state/ directory itself).

[Expert@HostName]# cd $FWDIR/state/
[Expert@HostName]# rm -rf *

Be very careful executing these commands because in case you miss the / between FWDIR
and "state" you will delete all the file system of the appliance.

Note:
In a cluster environment, both cluster members should have these directories cleared
simultaneously to avoid potential corruption in policy and in configuration.

8. Delete the content of the $FWDIR/state/Name_of_Gateway_or_Cluser_Object/ directory on

the Primary and Backup CMAs/Domains.
(Do not remove the $FWDIR/state/ directory itself).

[Expert@HostName]# cd $FWDIR/state/Name_of_Gateway_or_Cluser_Object/
[Expert@HostName]# pwd
[Expert@HostName]# rm -rf *

Be very careful executing these commands because in case you miss the / between FWDIR
and "database" you will delete all the file system of the appliance.

9. Start Check Point Services on both the Primary and Backup CMAs/Domains:

[Expert@HostName]# mdsstat
[Expert@HostName]# mdsstart_customer <CMA/Domain_Name>
[Expert@HostName]# mdsstat

10. Start Check Point Services on the Security Gateway:

[Expert@HostName]# cpstart

Notes:

• A message stating that the Security Gateway cannot get/fetch a Security Policy will be
displayed.
It can be ignored - this is a indication that the contents of the $FWDIR/state/ directory
on the Security Gateway and on Provider-1 CMA / Domain Management Server have
been cleared properly.

When starting Check Point Services, the Security Gateway will install Security Policy in
the following sequence:
i. Install from $FWDIR/state/local/ directory on Security Gateway.
ii. If in cluster, then check for updated policy on peer member(s).
iii. Install from $FWDIR/state/ directory on Provider-1 CMA / Domain Management
Server.

• Since on Security Gateway the files have been deleted from local $FWDIR/state/
directory, Security Gateway will install 'Default Policy', that blocks all traffic from
passing through (/proc/sys/net/ipv4/ip_forward = 0).

11. Connect with SmartDashboard to Provider-1 CMA / Domain Management Server.

12. Install the Security Policy onto this Gateway.

If you experience problems after this, use your backup to restore the previous
configuration.

1. Run 'cpstop' command.

2. Delete the content of '$FWDIR/state/' and '$FWDIR/database/' directories on the Security

Gateway. (Do not remove these directories).

3. Copy the contents of your backup, 'state' and 'database' directories on the Security Gateway
and on the Provider-1 CMA / Domain Management Server to their original locations.

4. Run 'cpstart' command.