SANDBLAST TRAINING

Troubleshooting and debugging

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 1
 01
TROUBLESHOOTING

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​
 Cloud emulation troubleshooting
• Error about connectivity to the cloud
̶ Check proxy settings:
̶ Global Properties

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 3
 Cloud emulation troubleshooting
• Error about connectivity to the cloud
̶ Check proxy settings:
̶ Under GW object

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 4
 Cloud emulation troubleshooting
• Error about connectivity to the cloud
̶ Check DNS settings:
̶ GAIA UI

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 5
 Cloud emulation troubleshooting
• Error about connectivity to the cloud
̶ Check DNS settings:
̶ nslookup –query SRV te.checkpoint.com

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 6
 Cloud emulation troubleshooting
• Everything was working ok, now all file emulations are ending with errors
̶ Quota expired ?
̶ tecli show cloud quota

̶ Renew your subscription!

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 7
 Cloud emulation troubleshooting
• Everything was working ok, now all files are ending with error.
̶ Hourly/ Monthly quota exceeded?
̶ Also shown in GUI too:

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 8
 Local emulation Troubleshooting
• tecli show statistics
̶ Do emulations work?
̶ How many files do you see?
̶ Do you have any hits on cache?
̶ What is you average processing time?
• tecli cache dump all
̶ Do you have files with verdicts?
• tecli show emulator emulations
̶ Are emulations happening at this moment?

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 9
 Local emulation Troubleshooting
• tecli show downloads all
̶ Are your images in ready state?
̶ Shows you revision number of images
̶ Shows you available detection rules revisions
̶ Compare to: https
://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolu
tiondetails=&solutionid=sk92509
̶ If images are wrong or not in ready state
̶ Delete old images:
̶ # rm –rf /var/log/files_repository/images
̶ Kill Threat Emulation daemon and rerun update
̶ # fw kill ted
̶ # tecli a d u a
©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 10
 Local emulation Troubleshooting
• tecli advanced engine version
̶ Make sure you have the correct engine version
̶ Compare to:
https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails
=&solutionid=sk95235

• Curl_cli –vk https://te.checkpoint.com

̶ Make sure you get feedback from update servers

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 11
 Where are the logs?
Problem Logfile Comment
Ted.elg is the logfile of the threat
Emulation fails $FWDIR/log/ted.elg
emulation daemon
/var/log/maillog
Maillog is the Postfix mail transport log:
Mail is not delivered emaild.mta.log is the internal MTA log
$FWDIR/log/emaild.mta.elg connected to TED

$FWDIR/log/te_engine_log_file.elg

TE engine update fails $FWDIR/log/te_file_downloader.elg

$FWDIR/log/ted.elg

$FWDIR/log/te_file_downloader.elg
TE image update fails
$FWDIR/log/ted.elg

TE image initialization fails $FWDIR/log/te_image_prep_util.elg

$FWDIR/log/scrubd.elg
Threat Extraction fails
$FWDIR/log/scrub_cp_file_convertd.elg
Scrubd.elg is the general logfile of the
Threat Extraction daemon;
scrub_cp_file_convertd.elg is the log
for the file conversion process

DLPU is the process responsible for

File aggregation from stream does not work $FWDIR/log/dlpu.elg aggregating files from the network
stream

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 12
 03
DEBUGGING

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​
 Inspection Flow – Streaming - Debug
Logfile Comments

$FWDIR/log/ Start/Set debug level with:

ted.elg # tecli debug set <...>
TED
Restore default debug level:
# tecli debug defaults
AV

Deep scan / Archive

Temporary file

File hash
$FWDIR/log/ Start debug:
dlpu.elg # fw_debug dlpu on
TDERROR_ALL_ALL=5
DLPU DLPU DLPU
Stop debug:
# fw_debug dlpu on
TDERROR_ALL_ALL=0

Note! Be aware that it is not „fw debug“

but „fw_debug“

CoreXL CoreXL CoreXL

parser parser parser Note
If file is malicious
in AV with PREVENT
-> file will always get a
TE DETECT log

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 14
 Streaming – Debug – log examples
$FWDIR/log/ted.elg TED unique event ID

TED

$FWDIR/log/dlpu.elg

Temporary file

$FWDIR/log/dlpu.elg
DLPU DLPU

CoreXL CoreXL
parser parser

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 15
 Inspection Flow – MTA - Debug
Postfix

Logfile Comments

$FWDIR/log/ted.elg Start/Set debug level with:

# tecli debug set <...>
emaild
Restore default debug level: localhost:10026
# tecli debug defaults

/opt/CPsuite-R77/fw1/tmp/
email_tmp/ TED
$FWDIR/log/ Start debug:
emaild.elg # fw debug in.emaild.mta on
TDERROR_ALL_ALL=5
Temporary file
Stop debug:
# fw debug in.emaild.mta on
TDERROR_ALL_ALL=0 emaild
/var/log/maillog localhost:10025

Postfix

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 16
 MTA – Debug – log examples
/var/log/maillog

Postfix
$FWDIR/log/ted.elg

TED unique event ID

emaild
localhost:10026
$FWDIR/log/emaild.elg

TED

Temporary file

emaild
localhost:10025
/var/log/maillog

Postfix

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 17
 Inspection Flow – TX - Debug
Logfile Comments
Postfix
$FWDIR/log/ Start debug:
scrubd.elg # scrub debug on
# scrub debug set all all
/var/log/jail/$FWDIR/log/ # for PROC in $(pgrep
scrub_cp_file_convertd.elg cp_file_convert) ; do fw debug
$PROC on TDERROR_ALL_ALL=5 ; done emaild
Stop debug:
# fw debug cp_file_convert off
TDERROR_ALL_ALL=0
# scrub debug off Note:
# scrub debug reset
If file is malicious and was TXed -> SCRUBD
file will always TED
get a TE DETECT log
$FWDIR/log/emaild.elg Start debug:
# fw debug in.emaild.mta on
TDERROR_ALL_ALL=5

Stop debug: Temporary file

# fw debug in.emaild.mta on
TDERROR_ALL_ALL=0
emaild
/var/log/maillog

Postfix

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 18
 TX – Debug – log examples
/var/log/maillog

Postfix
/var/log/jail/$FWDIR/log/scrub_cp_file_convertd.elg

$FWDIR/log/scrubd.elg
emaild

SCRUBD TED
$FWDIR/log/emaild.elg

Temporary file

emaild
/var/log/maillog

Postfix

©2017 Check Point Software Technologies Ltd. [Internal

[Restricted] ONLYUse]for
fordesignated
Check Pointgroups
employees​
and individuals​ 19
 Further debugging info

• Check Point Processes and Daemons

• https://supportcenter.checkpoint.com/supportcenter/portal?eventSubmit_doGoviewsolutiondetails=&solutionid=sk97638

• Short link - http://tiny.cc/sk97638

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 20
 QUESTIONS?

Next – Threat Extraction

©2017 Check Point Software Technologies Ltd. [Internal Use] for Check Point employees​ 21