Trend Micro Apex One Troubleshooting Guidev1.5

TREND MICRO™Apex One
AMEA Partner
Case Submission Handbook
TREND MICRO™Apex One AMEA Partner Case Submission Handbook

Document Version 1.5
Prepared by: Alghie Garcia, Jessie Menil, Wilson Salvador
Contributors: Jean Luces, Michelle Ramos, Nickel Xu, Raymond Villafania, Regidor De Guzman
Copyright © April 2020 by Trend Micro Inc. All Rights Reserved.

Table of contents
Introduction ...................................................................................................................... 6
What's new .................................................................................................................... 7
I. Reviewing System Requirements ...................................................................................... 8
Pre-deployment ............................................................................................................. 9
Collecting Basic Information .......................................................................................... 10
II. Policy Deployment Process ............................................................................................ 13
What happens after a policy is deployed from Apex Central to Apex One Server? ................ 13
Policy Deployment Triggers ........................................................................................... 14
Time needed for policy deployment status to reflect on Apex Central ................................. 14
Apex One Policy vs. Integrated Features ............................................................................. 15
Scenario 1: Default iProduct policy settings ..................................................................... 15
Scenario 2: Apex One server does not have a valid iProduct license ................................... 15
Agent Optimization .......................................................................................................... 16
General Problem Isolation Testing ...................................................................................... 21
III. Apex One Common Issues .......................................................................................... 24
A. Server Installation/Upgrade Issues ............................................................................. 25
Troubleshooting Tips ................................................................................................ 25
Fresh installation of Server .................................................................................... 25
Upgrade from OfficeScan to Apex One Server ......................................................... 25
Critical Patch/Hotfix Installation ............................................................................. 27
Logs to collect .......................................................................................................... 29
Useful links ............................................................................................................. 31
B. Agent Installation Issues ........................................................................................... 32
Remnants of old installation .................................................................................. 32
3rd-party AV is installed ........................................................................................ 33
Logs to collect .......................................................................................................... 36
C. Offline Issues ........................................................................................................... 37
Check Server/Agent communication ....................................................................... 37
Identify IIS Issues ................................................................................................ 44
TLS Issue ........................................................................................................... 47
Check License and Configuration ............................................................................ 49
Licensing ......................................................................................................... 49
Check DB Connection ........................................................................................ 49
NAT agents ..................................................................................................... 51
Logs to collect .......................................................................................................... 53
D. Agent Upgrade Issues .............................................................................................. 55
How to check for Server/Agent Communication? ..................................................... 55
How to review the agent update configuration? ....................................................... 57
How to check for Mismatched Certificate? ................................................................ 57
Upgrade File Issue ................................................................................................ 58
Review Update Agent Configuration ....................................................................... 58
Unable to upgrade Windows 10 ............................................................................. 60
Logs to collect .......................................................................................................... 61
E. Performance Issues .................................................................................................. 63
2 / 206
Optimization of System Performance ...................................................................... 63
Disable Windows Defender .................................................................................... 65
Battery Configuration ............................................................................................ 68
Logs to collect .......................................................................................................... 69
F. Web Console Issues .................................................................................................. 70
Apex One Master Service was stopped .................................................................... 70
Logs to collect .......................................................................................................... 79
G. Smart Protection Server (SPS) Issues ......................................................................... 80
Unable to Login to SPS console .............................................................................. 80
Unable to Login using Root Password ..................................................................... 80
Changing SPS IP Address ...................................................................................... 83
Web Reputation Service (WRS) and File Reputation Service (FRS) shows Unavailable ... 84
Best Practice Configuration ................................................................................ 87
Logs to collect .......................................................................................................... 88
IV. Apex One iProduct Common Issues .............................................................................. 89
iProduct Activation Code (AC) Guide .............................................................................. 89
A. Apex One Endpoint Sensor (iES) ................................................................................ 91
Installation of Apex One Endpoint Sensor ................................................................... 91
iES Installation Verification .................................................................................... 91
iES Installation failed ............................................................................................ 93
Activating Apex One Endpoint Sensor (iES) ................................................................. 94
Apex One Endpoint Sensor (iES) Policy Deployment Issue ............................................ 95
Apex Central Issue ............................................................................................... 95
Apex One Issue ................................................................................................... 95
Apex One agent Issue ........................................................................................... 96
Useful links .......................................................................................................... 97
Log Collection per Issue ............................................................................................ 98
B. Apex One Application Control (iAC) .......................................................................... 99
Policy Deployment Flow for iAC ................................................................................. 99
Check Apex One Server status in Apex Central ............................................................ 99
Verify iAC Service Status ......................................................................................... 100
How to Verify iAC Service Status in Apex One Server ............................................. 100
Apex One Server Certificates ............................................................................ 103
How to Verify iAC Service Status in Apex One Agent .............................................. 105
Troubleshooting iAC Policy Deployment .................................................................... 107
Policy Error “Product Communication Error” .......................................................... 107
Policy Error “Application Control Service: Unactivated licenses” ................................ 108
Policy Error “Pending: Waiting for product agent” .................................................. 110
Log Collection ........................................................................................................ 112
C. Apex One Vulnerability Protection (iVP) .................................................................... 113
iVP Licensing Issue ................................................................................................. 113
Review Command Tracking/IIS/Services Status ...................................................... 113
Troubleshooting "iProduct Service not Starting" ..................................................... 115
Troubleshooting Certificate Issue "License Deployment was Unsuccessful" ................ 117
Policy Deployment Issue ......................................................................................... 121
Policy status “Pending: Apex Central deploying” ..................................................... 124
Policy status “System error. Error ID: 5” ............................................................... 124
3 / 206
Policy status shows "Unable to logon Product" ....................................................... 126
Policy status “Pending: Waiting for product agent” ................................................. 127
Log Collection ........................................................................................................ 132
Apex Central ...................................................................................................... 132
Apex One Server ................................................................................................ 132
Apex One Agent ................................................................................................. 133
Enabling Manual Debug ....................................................................................... 134
D. Apex One Data Loss Prevention (iDLP) ..................................................................... 139
Pre-requisites when deploying Data Loss Prevention ................................................... 139
Apex One Data Loss Prevention (iDLP) Installation ................................................. 139
Apex One Data Loss Prevention (iDLP) License Activation ....................................... 140
Enabling and Verifying the Data Loss Prevention (iDLP) Module ................................... 142
Enabling iDLP via Apex Central ............................................................................ 142
Enabling iDLP via Apex One ................................................................................ 144
Verifying if iDLP policy is deployed ....................................................................... 144
Verifying if iDLP is installed properly ..................................................................... 144
Blocking USB using Device Control .......................................................................... 146
Adding USB device to Approved List ..................................................................... 147
Deploying Data Loss Prevention Policy ...................................................................... 148
Deploying iDLP via Apex Central .......................................................................... 148
Deploying iDLP via Apex One .............................................................................. 152
Troubleshooting iDLP Common Issues ...................................................................... 153
Data Protection Status is showing “Not Installed” ................................................... 153
Data Protection Status is showing “Stopped” .......................................................... 155
Unable to install Data Protection plug-in ................................................................ 155
USB Exception is not working .............................................................................. 157
USB Blocking is not working ................................................................................ 159
DLP Blocking is not working in browser ................................................................ 160
Some devices are being blocked by DLP (e.g. Scanner) ........................................... 160
Log Collection ........................................................................................................ 161
Collect CDT on the Server ................................................................................... 161
Collect CDT on the Agent .................................................................................... 161
Collect Device Control information ........................................................................ 161
Collect dsagent crash dump file ............................................................................ 162
Isolation if issue is caused by DLP ........................................................................ 162
Collect Full HTTP Dump ...................................................................................... 162
E. Apex One (Mac) ..................................................................................................... 163
Apex One (Mac) Server Requirements ....................................................................... 163
Apex One (Mac) Server Installation and Activation ..................................................... 163
Installation Verification ........................................................................................ 163
Apex One (Mac) agent Installation ............................................................................ 166
Deploying Apex One (Mac) Policy from Apex Central .................................................. 170
Apex One (Mac) Common Issues ............................................................................. 172
Blank page when accessing console ...................................................................... 172
Logs to be collected ........................................................................................ 176
Getting error "Format of the initialization string does not conform to specification..." on
TMSM_DBTool.log when installing Apex One (Mac) plug-in ..................................... 177
Plugin will not start after installing (upgrade) Apex One patch ................................. 179
4 / 206
Apex One (Mac) agent is unable to start after upgrading to macOS 10.15 (Catalina) .. 182
iProduct System Requirements ..................................................................................... 183
V. How to enable debug? ................................................................................................ 185
How to debug the Apex One server? ............................................................................ 186
How to debug Widget Framework? ............................................................................... 187
How to debug CM Agent Issues? .................................................................................. 187
How to manually debug the agent? .............................................................................. 188
How to debug Scan Engine? ........................................................................................ 188
How to enable Apex One Diagnostic Log? ..................................................................... 189
How to debug SPS Server using CLI? ............................................................................ 190
Indexes ......................................................................................................................... 195
How to collect Windows Performance Recorder (WPR)? .................................................. 195
How to collect Windows Dump Files? ............................................................................ 197
How to collect Procdump Logs ..................................................................................... 198
How to collect ProcMon logs? ...................................................................................... 199
How to collect UI Network Traffic Log? ......................................................................... 200
How to replicate issue for Offline agents? ...................................................................... 201
How to replicate issue for Outdated agents? .................................................................. 202
How to check if Apex One Server is using 3rd-party certificate? ....................................... 202
Feedback ....................................................................................................................... 206
Useful links .................................................................................................................... 206
5 / 206
TREND MICRO™Apex One
AMEA Partner Case Submission Handbook

This document serves as a manual for troubleshooting common issues. It provides in-depth troubleshooting
guidelines about configuration, components, and functionality of Apex One on-premise.
By following this document, we can ensure that submitted cases are already isolated and verified from the given
troubleshooting guidelines.
Overview
Figure below shows an Apex One Sample setup.
6 / 206
What's New in Apex One!
This guide will help partners/customers to know the common issues on Apex One and how to troubleshoot it. It contains
step-by-step procedure, Apex One commands, and useful tools.
The following tables outlines the new features and enhancements in this version of Trend Micro Apex One™ .
Item Description
Offline Predictive Machine Learning Predictive Machine Learning has been upgraded to provide
offline protection against portable executable files. The
lightweight, offline model helps protect all endpoints against
unknown threats when a functional Internet connection is
unavailable
Fileless Attack Protection Security Agent policies provide increased real-time protection
against the latest fileless attack methods through enhanced
memory scanning for suspicious process behaviors. Security
Agents can terminate suspicious processes before any
damage can be done.
Off-premises Security Agent Protection Enhanced Edge Relay Server support allows for increased
communication between the Apex One server and off-
premises Security Agents. Security Agents can receive
updated policy settings from the Apex One server even when
a direct connection to the server is unavailable.
Rebranded Console The OfficeScan server and OfficeScan agent programs have
been rebranded to the Apex One server and Security Agent
respectively. The new Apex One server integrates with Apex
Central (formerly Trend Micro Control Manager) to provide
increased protection against security risks. The all-in-one
Security Agent program continues to provide superior
protection against malware and data loss but also allows you
implement Application Control, Endpoint Sensor, and
Vulnerability Protection policies without having to install and
maintain multiple agent programs.
URL: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-2019-server-online-help/introduction-and-
get/introducing-product_/whats-new.aspx
7 / 206
I. Reviewing System Requirements
On this section, you will see the requirements for Pre-deployment and Collecting Basic Information.
1. Pre-deployment will discuss the following:

1. Apex One System Requirements
2. URLs used by Apex One
3. Ports and Protocols used by Apex One
2. Collect Basic Information will discuss items that are needed when submitting a case to Trend Micro Support:
1. Case Description
2. Server Information
3. Agent Information
4. Network Layout
8 / 206
System Requirements
Server and client have met minimum system requirements.
Verify Apex One System Check the System Requirements:

Requirements https://docs.trendmicro.com/all/ent/apex-one/2019/en-us/apexOne_2019_req.pdf
Supported IP address Pure IPv4 and Dual IP Stacks is supported, but pure IPv6 is not s upporte d
Product Limitation on IIS · Apex One is a 32-bit program.
· Apex One installs under WOW on 64-bit computers (Standard and Enterprise editions).
What URLs are used by Apex One?
Here are the URLs used by Apex One:
1 http://osce14-p.activeupdate.trendmicro.com/activeupdate
2 http://osce14-ilspn30-p.activeupdate.trendmicro.com/activeupdate
3 http://osce14-ilspn30w r-p.activeupdate.trendmicro.com/activeupdate
4 http://osce14.icrc.trendmicro.com/
5 http://osce14-0-en.url.trendmicro.com
6 http://oscecmp140-de-f.trx.trendmicro.com/
7 http://osce140-en.fbs25.trendmicro.com/
8 http://osce14-en-census.trendmicro.com/
9 http://osce14-en.gfrbridge.trendmicro.com/
10 http://licenseupdate.trendmicro.com/
Ports and protocols used by OfficeScan/Apex One that should be allowed through a
firewall or router
Here are the different ports and protocols used in OfficeScan/Apex One which should be allowed to communicate
via firewall or router. This is typically the scenario in case the customer deployed either an OfficeScan/Apex One
server or a client/agent in a DMZ or they have segmented their network into multiple subnets.
Age nt/Se rve r com m unication It is a random 5-digit port number set during installation. To determine this port number, check the
port "Client_LocalServer_Port" parameter in the \PCCSRV\ofcscan.ini file.
Ne tBIOS ports This uses TCP/UDP port 137, TCP port 139, and TCP port 445. These ports are used w hen
installing clients/agents via Remote Install and w hen clients/agents send quarantined files to the
server using the UNC path.
Com m unication w ith Control MCP agent uses TCP port 80 on HTTP or TCP port 443 on HTTPS to communicate w ith Control
M anage r/Ape x Ce ntral Manager/Apex Central.
Lice ns e ports These allow access to the Trend Micro License Server via TCP port 443.
Standalone Sm art Prote ction If Standalone Smart Protection Server is used in the environment, File Reputation Service for
Se rve r smart scan uses port 80 for HTTP and port 443 for HTTPS. Web Repuation Service uses port
5274. The w eb console uses port 4343 for HTTPS.
Unm anage d e ndpoints This port (TCP 135 by default) is used by the OfficeScan/Apex One server to check w ith those
9 / 206
che ck ing unreachable and determine w hether itʼs managed by another OfficeScan/Apex One server. This
port can be configured through the follow ing menu path: OfficeScan/Apex One w eb
console > Assessment > Unmanaged Endpoints > Define scope.
Collect Basic Information
Case Description
When submitting case, it is important to have clear and complete information on the case.
1. Provide a short description of the problem.

2. Provide the step–by-step process to reproduce the problem
3. Screenshot of the problem/error
4. Provide information if there are any changes on the system or the netw ork before problem happened
5. What is the expected result?
Server Information
1. Product ve rs ion and build
Using Apex One w eb console, go to He lp > About
2. Product re gis try inform ation
Registry export of HKLM \SOFTWARE\WOW6432Node\TrendM icro\OfficeSca n\service\Informa tion
3. Bas ic Sys te m Inform ation
Run m s info32 to open Window s Sys te m inform ation. Click File > Export to a text file or .nfo file
10 / 206
4. Eve nt Logs
o Run e ve ntvw r and then expand Window s Logs :

o Right-click Application > Save All Eve nts As ... > Specify the file name then click Save .
o Do the same for "Se curity", "Se tup" and "Sys te m ".
5. Database Server Information
o Using Apex One Console, go to He lp > About
o Database Server Type and information (e.g. MSDE/SQLExpress/SQL):

1. Open PCCSRV\Private\ofcs e rve r.ini
2. Look for the entry SQL Server: DBE_ENGINE=1002
Note: The Apex One server uses SQL Server b y default.
o Service Pack installed

1. Using any DB brow ser tool (e.g. Microsoft SQL Server Management Studio)
- Go to Run > Type: s s m s > Type SQL Query: s e le ct @@ve rs ion > Press F5 to execute the commands.
11 / 206
6. IIS re late d applications
o List dow n other Applications (e.g. Control Manager/Apex Central, 3rd party applications) using IIS.
o Identify the w ebsite security level (Low /Medium/High)
- Low = HTTP only
- Medium = SSL primary and HTTP secondary
- High = SSL Only
7. Tim e Ele m e nt
o Take note of the system time of the server (relative to time on the agent)
o Take note of the system timezone
Basic Agent Information
Product version and build · Identify the Apex One agent version and build number
o Right-click on the system tray icon, then click on Com pone nt Ve rs ion
· Collect ofcscan.ini in the product agent directory

Basic System Information Run m s info32 and export system information to a text file
Time Element · Take note of the system time of the agent (relative to time on the server)
· Take note of the system timezone
Network Layout
Check Netw ork Layout Diagrams/draw ings of netw ork layout how agents are connected to the Apex One
Server
Identify firew all, VPN, NAT and other netw orking services in use
12 / 206
II. Policy Deployment Process
What happens after a policy is deployed from Apex Central to Apex One Server?
1. Apex Central deploys policy to Apex One server.
2. Apex One server dispatches policies to iProduct Servers.
3. For Saas, Apex One server now waits for SaaS agents to poll (default every 10 min).
§ On-premise agent will receive server notification immediately.
4. After Apex One agents get policy tasks/commands, Apex One agents also notify the iProduct agents.
5. Apex One server marks agent as “deployed successfully” once Agent One agents get the policies from server.
§ For iProduct agents, after the policies are applied, iProduct agents report policy status to
corresponding iProduct servers accordingly.
6. iProduct servers write iProduct agentsʼ policy status to database & Apex One server consolidates all status
result from iProduct servers.
7. Apex One server then sends consolidated policy status to Apex Central.
13 / 206
Policy Deployment Triggers
SCENARIO USE CASE AFFECTED ENDPOINTS AFFECTED POLICIES DEPLOY TIMING
All endpoints without policy and match the new

New filtered policy Only this policy Immediate
criteria
CREATE POLICY
New specified policy The specified endpoints Only this policy Immediate
Edit targets (criteria) for filtered All endpoints as long as they are not in specified
All filtered policies Immediate
policy polices
Endpoints in this policy

(If endpoints are removed from polices,
Edit targets for specified policy Only this policy Immediate
they will be regarded as “new” endpoints
EDIT POLICY by policy deployment flow)
Edit policy setti ngs only The endpoints in the policy Only this policy Immediate
Reorder policies All endpoints as long as

All filter policies Immediate
(including policy removal) they are not in specified polices
New endpoint reported to Apex 120 sec after endpoints are reported
The new endpoints Policies applicable to these new endpoints
Central to Apex Central
NEW OR CHANGED
ENDPOINTS
Endpoint property changes
The changed endpoints All policies Every 24 hours
(which also causes policy changes)
Apex Central default mechanism On premise: Every 24 hours

POLICY ENFORCEMENT All endpoints All policies
to ensure all endpoints get policies
SaaS: Every 10 minutes
Time needed for policy deployment status to reflect on Apex Central
· Within 20 minutes
o Creating new policies for the 1st time, or new registered agents that never had a policy applied (Apex
Central checks every 120 seconds to see if there are new agents)
o Admin reorders policies
o Admin edit policy settings or targets (either specified or filtered)
· Wait for next policy enforcement

o New agents that passed Apex Centralʼs new agent check (every 120 seconds), but didnʼt get an
applicable policy (becomes “without policies”)
o Agents that received polices & need to be moved to another policy due to agent property changes (e.g.
location in AU, IP address, etc)
AD-based filtered policies always need to have Apex Central sync the latest AD info first in order to trigger policy
changes.
14 / 206
Apex One Policy vs. Integrated Features
Scenario 1: Default iProduct policy settings

By default, iProduct settings are set to “disabled”, this implies iProduct agents are not installed. Under this
situation, after Apex One server dispatches policies to iProduct servers, iProduct servers will directly respond
“successfully deployed” to Apex One server.
The very first policy deployment that enables iProducts settings will trigger iProduct agent installation.
Once iProduct agents are installed, policy setting changes to iProducts will just fall into the normal policy
deployment flow
Scenario 2: Apex One server does not have a valid iProduct license
When there is a policy containing settings to enable iProduct settings, before dispatching the policies to iProduct
servers, Apex One server will first check if there are valid licenses; if there is no valid license, Apex One server will
respond “unactivated licenses” error code to Apex Central directly.
15 / 206
Agent Optimization
How to optimize Apex One agent?
1. Install the latest patch for Apex One

https://downloadcenter.trendmicro.com/index.php?
regs=ph&prodid=1745&_ga=2.65440174.1208411755.1586855937-175934259.1554708004
2. Minimize Behavior Monitoring's functionality without sacrificing the security of Apex One
If process SYSTEM has high CPU, do the following:
Note: Unload the Apex One agent first. Always back up the whole registry before making any modifications.
Incorrect changes to the registry can cause serious system problems.
a. Skip System File Event Scan:
[HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\AEGIS]
"SkipSystemFileEvent"=dword:00000001
b. Skip scan when opening process from system:
[HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\AEGIS]
“SkipOpenProcessFromSystem” =dword:00000001
If process TMBMSRV.exe, NtRtScan.exe, TmCCSF.exe and LogServer.exe have high CPU, do the following:
a. Disable activity monitor to stop sending event to product processes:
[HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\Misc.]
"EnableAegisActivityMonitor"=dword:00000000
3. Exclude the application on Real-time scan, Behavior Monitoring and Trusted Program List
16 / 206
a. Real-time scan
b. Behavior Monitoring
c. Trusted Program List
4. Enhance Application Control feature (applicable to those agents with Application Control enabled)
a. Delayed Application Control's startup process during boot-up.
Note: To prevent CPU high utilization / high disk consumption for Application Control Agent when
machine boots up.
i. Make sure the iAC agent build is at least "TMiACAgentSvc.exe" >= 3.0.0.2003. To verify you
may check the following file:
C:\Program Files (x86)\Trend Micro\iService\iAC > right click and select Properties > go to
Details tab and check the File version or right-click from Agent Tray icon and click "Component
Versions".
17 / 206
ii. Unload Apex One Security Agent
iii. Set the registry with value below
Key : DelayLoadAC
HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\iACAgent\DelayLoadAC
Type : DWORD
Valid Range : 0-10 (min)
b. Increased the LRU Cache 2000 (default)

i. Unload Apex One Security Agent
ii. Stop iAC agent service (TMiACAgent service)
iii. Set the registry with value below
Key : LRUCacheSize
HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\iACAgent\LRUCacheSize
Type : DWORD
Value : 5000 (Default = 2000)
Note: The iAC service may consume disk when opening VB or other application since the Application
Control Agent will evaluate the PE files, it will try to calculate the hash value(SHA1 and SHA2) and the
digital signatures information. Those information help iAC Agent to make the decision when a process
needs to be allowed or blocked. When iAC Agent try to evaluate PE files, it will need CPU and I/O
loading. To resolve this kind of issue, we have an LRU cache mechanism which keep those PE file's
hash values and digital signatures information when the PE file has been evaluated once. The LRU
18 / 206
cache mechanism will speed up when the process/image launching. However, it still need to spend the
cost in the first time to calculate those information.
5. Change the interval of Endpoint Sensor's data forwarding from 15 minutes (default) to 3 hours (applicable to those
agents with Endpoint Sensor enabled)
6. Enable the deferred scan.

Defer Scan can postpone the timing of scanning and for VSAPI engine not to perform file-lock while waiting
7. Make sure that the debug module has been disabled.

VSAPI:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TmFilter\Parameters] DebugLogFlags=0
BM:
HKLM\SOFTWARE\TrendMicro\Aegis\DebugLogFlags = dword:00000000
HKLM\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\Real Time Scan
Configuration\DACPolicyDump = dword:00000000
AEGIS:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\tmevtmgr\Parameters]
"DebugLogFlags"=dword:00000000
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\tmcomm\Parameters]
19 / 206
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\tmactmon\Parameters]
DLP: (remove the keys)

HKLM\Software\Trend Micro\PC-cillinNTCorp\DlpLite\debugcfg
HKLM\Wow6432Node\Software\Trend Micro\PC-cillinNTCorp\DlpLite\debugcfg
20 / 206
General Problem Isolation Testing
Summary
When there is an issue on an endpoint with the OfficeScan/Apex One Security Agent installed, isolation
testing is a recommended preliminary step to help determine where the issue is.
Once the issue has been isolated and you have an idea on the service (e.g. Realtime scan, WRS, behavior
Monitoring) causing the issue you can start debugging the specific service causing the issue.
Where to start isolating the issue?
Using windows services turn each service off one at a time until the issue is gone. Take note of the
suspected service and turn the suspected service back on to confirm. As components can interact with each
other, it is possible that disabling different services could potentially resolve the issue. If any other service also
corrects the issue, please note those as well.
How to turn off the following services using Apex One web console?
Turn-off each service from the web console, do a manual update on client. Test if the issue persists.
1. Real Time Scan (VSAPI) Proce dure : Go to Agents -> Agent Management -> select 1 machine -> Settings ->
Scan Settings -> Real-time Scan Settings -> untick "Enab le virus/malware scan" -> Save
Note : If this action solves the issue, please enable this setting and do action 3, 4, 8, 10,
and 12 to confirm the problematic service further.
2. Web Reputation Service (WRS) Proce dure : Go to -> Agents -> Agent Management -> Click 1 machine -> Settings ->
Web Reputation Settings -> untick "Enab le Web reputation policy on the following
operating systems" -> Save
Note : If this action solves the issue, please enable this setting and do action 8, 10, and
13 to confirm the problematic service further.
3. Predictive Machine Learning Service Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Predictive
Machine Learning Settings -> untick "Enab le Predictive Machine Learning" -> Save
(PML)
Note : If this action solves the issue, please enable this setting and further test File and
Process types, separately.
· Agents -> Agent Management -> Click 1 machine -> Settings -> Predictive Machine
Learning Settings -> Unclick "File" -> Save
· Agents -> Agent Management -> Click 1 machine -> Settings -> Predictive Machine
Learning Settings -> Unclick "Process" -> Save
4. Behavior Monitor Service (AEGIS) Proce dure :
21 / 206
· Agents -> Agent Management -> Click 1 machine -> Settings -> Behavior Monitor
Settings -> untick "Enab le Malware Behavior Blocking" -> Save
· Agents -> Agent Management -> Click 1 machine -> Settings -> Behavior Monitor
Settings -> untick "Enab le Event Monitoring" -> Save
and 11 to confirm the problematic service further.
5. Unauthorized Change Prevention Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Additional
Service Settings -> Unauthorized Change Prevention Service -> untick -> Save
Service (AEGIS)
10, and 11 to confirm the problematic service further.
6. Firew all Service (NSC) Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Additional
Service Settings -> Firew all Service -> untick -> Save
7. Suspicious Connection Service Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Additional
Service Settings -> Suspicious Connection Service -> Unclick -> Save
8. Advanced Protection Service (TMCCSF) Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Additional
Service Settings -> Advanced Protection Service -> Unclick -> Save
Note : If this action solves the issue, please enable this setting and do action 3, 10, 11,
12, and 13 to confirm the problematic service further.
9. Ransomw are Protection

A. Acce s s Docum e nt Control
Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings ->
Behavior Monitor Settings -> Unclick "Protect documents against unauthorized
encryption or modification" -> Save
B. Softw are Re s tricte d Policy

Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings ->
Behavior Monitor Settings -> untick "Block processes commonly associated with
ransomware" -> Save
10. Program Inspection (TMUMH) Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings ->
Behavior Monitor Settings -> untick "Enab le program inspection to detect and b lock
compromised executab le files" -> Save
Note : You must see the stop of the tm um h by command "s c que ry tm um h". If tmumh
is still running, run command "s c s top tm um h" to stop it. A reboot might be needed
because tmmon has hooked to the processes.
11. New ly Encountered Programs Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Behavior
Monitor Settings -> untick "Monitor newly encountered programs downloaded through
(Meerkat)
web or email application channels" -> Save
12. Scan Memory (Ravage Scan) Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Scan
Settings -> Real-time Scan Settings -> untick "Quarantine malware variants detected in
memory" -> Save
22 / 206
13. Brow ser Exploit Prevention Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Web
Reputation Settings -> untick "Block pages containing malicious script " -> Save
14. Data Protection Service Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Additional
Service Settings -> Data Protection Service -> untick -> Save
Note : If this action solves the issue, please enable this setting and do action 15 and 16
to confirm the problematic service further.
15. Device Control Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> Device
Control Settings -> untick "Enab le Device Control" -> Save
Note : If this action solves the issue, please enable this setting and do action 16 to
confirm the problematic service further.
16. DLP Settings Proce dure : Agents -> Agent Management -> Click 1 machine -> Settings -> DLP
Settings -> untick "Enab le Data Loss Prevention" -> Save
NOTE: For isolating on Apex One as a Service, see KB 1123591
23 / 206
III. Apex One Common Issues
On this section, you will see Troubleshooting Tips and Logs to be Collected for the Top Common Cases:
1. Server Installation / Server Upgrade Issues

a. Fresh Server Installation Issue
b. Upgrade Issue from OfficeScan to Apex One
c. Critical Patch / Hotfix Installation Issue
2. Agent Installation Issues
a. Remnants of old installation
b. 3rd-party AV is installed
3. Offline Issues
a. Checking of Server/Agent Communication
b. Identifying IIS Issues
c. Checking of License and Configuration
d. TLS Issue
4. Agent Upgrade Issues
a. Checking of Server/Agent Communication
b. Reviewing Update Configuration
c. Checking for Mismatched Certificate
d. Upgrade File Issue
e. Checking for Update Agent Configuration
24 / 206
A. Server Installation/Upgrade Issues
On this section, we will be discussing common issues when installing, upgrading, or patching Apex One Server.
Troubleshooting Tips
Listed are the consolidated troubleshooting steps per issue:
1. Fresh Server Installation Issue

2. Upgrade Issue from OfficeScan to Apex One
3. Critical Patch / Hotfix Installation Issue
If issues are not resolved after performing the provided troubleshooting tips, collect the recommended logs and file
a case to Trend MIcro Support.
Fresh installation of Server

System Requirements
If the target device does not meet the system requirements then the software may not work correctly after
installation. You may also experience performance issues and other problem related to resources.
See System Requirements
Upgrade from OfficeScan to Apex One Server
A. Quick migration guide for Trend Micro Apex One
Summary:
Upgrading to Trend Micro Apex One™ allows you to enable extended endpoint features like Application
Control, Endpoint Sensor, and Vulnerability Protection — all within one product.
It redefines endpoint security with its breadth of capabilities delivered as a single agent, with consistency
across SaaS and on-premises deployments. This offers enhanced automated detection and response and
actionable insights that maximize security for customers.
This article provides an overview of multiple scenarios and recommended upgrade plans. For a detailed
guide, please refer to the Install and Upgrade Guide in the Deployment Suggestions Based on Product
Features section below
The following topics are discussed on this KB
· Pre-Upgrade Checklist for Apex One Server

· Pre-Upgrade Checklist for Apex One Agent
· Sizing Considerations
· Deployment Suggestions Based on Product Features
25 / 206
See KB 1122308 for more details
B. During Server Upgrade, the installer detected that there are unsupported Agent Operating
Systems.
1. Access Officescan Server web console.

2. Go to Agents > Agent Management.
3. Export Client Listing.
4. Check the exported Client Listing for any unsupported OS
5. If there are no unsupported OS on the agent listing, export the information from Apex One database
a. Access SQL Server
b. Access Apex One DB
c. Export the data from dbo.TBL_CLIENT_INFO
6. Check the exported file and filter the OS_MAJOR, OS_MINOR

7. There should be no machines on the following:
a. 6.0 = Windows XP and Windows Server 2008
b. 6.2 = Windows 8
c. 5.2 = Windows Server 2003
8. If there are any machines that are those version, delete it from the database by using this SQL
Command:
DELETE FROM [DBname].[dbo].[TBL_CLIENT_INFO] Where UID ='GUID of the unsupported machines'
26 / 206
How to troubleshoot of Critical Patch / Hotfix Installation Issues?
If encountered an issue when installing a Critical Patch/Hotfix, check tmpatch.log on C:\
1. Look for this keyword: failed.

Sample log file:
[2019-09-25:09:58:41][perfLWCSPerfMonMgr.dll : C:\Program Files (x86)\Trend

Micro\OfficeScan\PCCSRV\LWCS\perfLWCSPerfMonMgr.dll[3.1.0.1009]->C:\Program Files (x86)\Trend
Micro\OfficeScan\PCCSRV\Backup\CriticalPatch_B2012\LWCS\perfLWCSPerfMonMgr.dll[3.1.0.1009]]
[2019-09-25:09:58:41][perfLWCSPerfMonMgr.dll : C:
\Users\santosh.z\AppData\Local\Temp\3\7ZipSfx.000\FileGroup180\perfLWCSPerfMonMgr.dll[3.1.0.2023]->C:\Program
Files (x86)\Trend Micro\OfficeScan\PCCSRV\LWCS\perfLWCSPerfMonMgr.dll[3.1.0.1009]]
[2019-09-25:09:58:41]Create new File Failed,last error:[32]
[2019-09-25:09:58:41]Rollback the file because the file copy fail.
[2019-09-25:09:58:41][perfLWCSPerfMonMgr.dll : C:
\Users\santosh.z\AppData\Local\Temp\3\7ZipSfx.000\FileGroup180\perfLWCSPerfMonMgr.dll->C:\Program Files (x86)
\Trend Micro\OfficeScan\PCCSRV\LWCS\perfLWCSPerfMonMgr.dll fail]
Failed.
[2019-09-25:09:58:41]Create new File Failed,last error:[32]

- This error means file is used by another process and not allowed to access.
2. In Apex One Server, perform the following action.
o Unload Apex One Agent

o Stop Apex One Master Services
o Stop any SQL Services
o Ensure no more TM related running processes from Task Manager
3. Based on the example above the hotfix/patch failed to replace the file perfLWCSPerfMonMgr.dll
If the hotfix/patch failed to replace a file/folder the logs will indicate the location in the logs and located
in C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\LWCS\perfLWCSPerfMonMgr.dll fail
4. Since the the patch failed to replaceperfLWCSPerfMonMgr.dll, manually rename this file (e.g.
perfLWCSPerfMonMgr.dll.backup )
On the example TmPatch.log, it failed on perfLWCSPerfMonMgr.dll. Rename the file from
27 / 206
perfLWCSPerfMonMgr.dll to perfLWCSPerfMonMgr.dll.bak
5. Reinstall the Critical Patch/Hotfix. (Run as Administrator)
28 / 206
Information and logs to Collect:
Collect Relevant Information
Get Server Information Verify OS Type, ServicePack, and Microsoft Hotfixes installed
Get SQL Information Check the SQL Server version and authentication used
Get Apex One Information Check the current version and build number:
A. Through UI:
1. Access web console > Help > About
B. Through registry:
HKLM\SOFTWARE\TrendMicro\OfficeScan\service\Information
Logs to be collected
New Installation Fresh Installation log file: C:\Windows\ofcmas.log
Collect CDT debug logs
What to check when running CDT Tool?

o Basic Information
o Installation & Uninstallation
Request for a copy of the Database
For steps on how to create backup DB, check this Microsoft link:
Take a screenshot of the error
Patch Installation Log File: C:\tmpatch.log
Request for a copy of the Database
For steps on how to create backup DB, check this Microsoft link:
Upgrade fail due to unsupported agent Log File: C:\tmpatch.log

OS
Copy of exported data from dbo.TBL_CLIENT_INFO
Steps in collecting the exported data:
29 / 206
1. Access SQL Server
2. Access Apex One DB
3. Export the data from dbo.TBL_CLIENT_INFO
30 / 206
Useful links
Know le dge Bas e Title Sum m ary

Article
KB 152876 Supported upgrade path to Apex This article lists the OfficeScan versions that can be upgraded to Apex One
One 2019 2019.
KB 1122308 Quick migration guide for Trend Upgrading to Trend Micro Apex One™ allow s you to enable extended
Micro Apex One™ endpoint features like Application Control, Endpoint Sensor, and
Vulnerability Protection — all w ithin one product.
It redefines endpoint security w ith its breadth of capabilities delivered

as a single agent, w ith consistency across SaaS and on-premises
deployments. This offers enhanced automated detection and
response and actionable insights that maximize security for
customers.
This article provides an overview of multiple scenarios and

recommended upgrade plans. For a detailed guide, please refer to the
Install and Upgrade Guide in the Deployment Suggestions Based on
Product Features section below .
31 / 206
B. Agent Installation Issues
On this section, we will be discussing common issues when installing Apex One agents. Troubleshooting steps for
the common issues are provided.
Listed are the consolidated troubleshooting steps per issue:
1. Remnants of old agent installation
2. 3rd-party AV is detected
a case to Trend Micro Support.
How to remove remnants of old installation?
1. You can use the Common Uninstall Tool:
· Available on Business Support Portal: https://success.trendmicro.com/diagnostic-tools

· Login on https://success.trendmicro.com/sign-in and navigate to My Support > Diagnostic Tools.
2. You can manually remove the remnants by following the steps on this KB:
https://success.trendmicro.com/solution/1039283-uninstalling-clients-or-agents-in-officescan#collapseOne
32 / 206
How to install Apex One agent on a machine with 3rd-party AV?
Here are troubleshooting steps when 3rd party antivirus programs unable to automatically uninstalled from the
computer before installing the Apex One agent.
1. Verify first whether 3rd party antivirus program are included already from the list of competitor products that
Apex One can automatically remove:
KB reference: https://success.trendmicro.com/solution/1105236-list-of-competitor-products-that-officescan-can-
automatically-remove
Note: If the uninstall password protection of 3rd party software is enabled, it is recommended that you need to
disable it first.
· You can also verify it from the tmuninst.ptn and tmuninst_as.ptn files under the \PCCSRV\Admin.
You can open these files using a text editor such as Notepad.
· You can also verify it from a certain Patch/HF installer, see example below:
a. Right click and Extract HF installer (apex_one_2019_win_en_hfbnnnn.u.exe).
b. Look for the tmuninst.ptn file and open it using a text editor such as Notepad.
33 / 206
2. If the 3rd party software is confirmed in the lists that can be detected and uninstalled, ensure you run the
updated installer such as MSI as follows:
· In the affected machine, right click CMD > select Run as administrator > Type "cd" with your MSI
installer Location path > Type your “MSI installerʼs name” > Press “Enter” and wait until finish.
· If it works and need to apply on mass deployment, you may deploy it via SCCM or GPO and it should
be done by the customerʼs System Administrator.
· Depending on the uninstallation process of the software, the endpoint may or may not need to restart
after uninstallation.
· If automatic agent migration is successful but a user encounters problems with the Security Agent right
after installation, restart the endpoint.
· If the Apex One installation program proceeded to install the Security Agent but was unable to uninstall
the other security software, there will be conflicts between the two software. Uninstall both software, and
then install the Security Agent using any of the installation methods discussed in Deployment
Considerations (Online Document: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-
2019-server-online-help/protecting-trend_cli/installing-the-trend/deployment-considera.aspx#GUID-
31C5ACC3-3D4B-4ADE-98FB-C145FE418573
3. If the 3rd party software on the target computer cannot be found in the list, Trend Micro Technical Support
can assist you to include it in the Apex One agent installer with coordination with our DEV Team to detect these
34 / 206
antivirus programs. Before contacting Trend Micro Technical Support:
· Prepare the following information below for our further checking:
1. What is the version and build number of the Apex One Server?
2. What is the version and build number of the 3rd party AV to be removed?
3. What type of Security Agent installation method will the customer use?
4. What is the client machine's operating system?
5. Kindly provide the copy and installation guide of the 3rd party installer [32 and 64 bit].
6. In the computer that 3rd party AV installed, kindly provide the following:
A. Screenshot of the "Program and Features".
B. Screenshot of the "About" status from 3rd party AV icon.
C. Kindly export and send to us the Registry entries from this path:
- Go to HKEY_LOCAL_MACHINE\Software\....
[32-Bit] = Microsoft\Windows\CurrentVersion\Uninstall\
[64-Bit] = Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\
· If the installer can no longer be retrieved, you can uninstall the third-party software using Add/Remove
Programs under Control Panel.
· If you encounter any problems uninstalling the 3rd party software, you need to contact the vendor of the
3rd party software.
4. If you want to prevent Apex One from uninstalling 3rd party security products during agent installation.
You may refer on this link for your further information.
KB reference: https://success.trendmicro.com/solution/1123821-prevent-apex-one-from-uninstalling-3rd-party-
security-products-during-agent-installation
35 / 206
Get the Operating · Verify if if issue affects specific version of Operating System (e.g. Windows 10)
System of the affected
machines
MSI package File name: OFCNT.LOG

installations
Location: In a temporary system file, for example in Windows 7:

C:\Users\Administrator\AppData\Local\Trend Micro\Security Agent\OFCNT.LOG
Web installations File name: WebInstall.log

Location: C:\
Remote Installations File name: RemoteInstall.LOG

Location: C:\
Autopcc and EXE File name: OFCNT.LOG

package installations
Location: %windir%\
36 / 206
C. Offline Issues
On this section, we will be discussing troubleshooting steps when encountering offline agents.
Listed are the consolidated troubleshooting steps:
1. Checking Server-Agent Communicaton
2. Identifying IIS Issues
3. TLS Issue
4. Checking License and Configuration
How to check network communication between Apex One Server and agent?
A. Check Apex One Server to Agent communication
1. Ping Offline_Agent_ address/FQDN

o Apex One server should be able to ping the agent
2. Telnet Offline_Agent_ address/FQDN through 5-digit listening port
a. Open ofcscan.ini on <installation path>\PCCSRV\
b. Check the value for Client_LocalServer_Port
c. Open cmd and run this command:
telnet OfflineAgent_IP_FQDN Client_LocalServerPort
d. If the Client_LocalServer_Port is open, you should get the following results:
37 / 206
B. Check Agent to Apex One Server communication
1. Ping ApexOneServer_IP/FQDN
o Agent should be able to ping the server
2. Check if the client is using the correct Client LocalServerPort
Client LocalServerPort is a random 5-digit port number port set during installation and used for
Server/Agent communication
a. Open \PCCSRV\ofcscan.ini, search and take note of the Client_LocalServer_Port
b. Check the Client Listening Port in Registry key:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\TrendMicro\PC-
cillinNTCorp\CurrentVersion\LocalServerPort
Important: Client LocalServerPort from agent's registry should match to Apex One Server
\PCCSRV\Ofcscan.ini (Client_LocalServer_Port).
From serverʼs ofcscan.ini From agentʼs registry

Client_LocalServer_Port LocalServerPort
c. Check the Client Listening Port from Agent Icon
1. Right-click on the agent icon in the system tray and choose "Component Versions".
2. At the top of the window, it will display the listening port.
38 / 206
3. Check Apex One Master_DomainName, Server Port, and Server SSLport
a. Open \PCCSRV\ofcscan.ini, search and take note of the following:
Master_DomainName = xxxx
Master_DomainPort = xxxx
Master_SSLPort = xxxx
b. Check the Client Listening Port in Registry key
Important: The following entries should match
From server's ofcscan.ini From agent's registry

Master_DomainName Server
Master_DomainPort ServerPort
Master_SSLPort ServerSSLPort
4.Telnet ApexOneServer_IP/FQDN through MasterSSLPort
a. Open ofcscan.ini on <installation path>\PCCSRV\

b. Check the value for Master_DomainPort and Master_SSLPort
c. Open cmd and run this command:
telnet ApexOneServer_IP/FQDN Master_DomainPort
39 / 206
telnet ApexOneServer_IP/FQDN Master_SSLPort
d. If the Master_DomainPort and Master_SSLPort are open, you should get the following results:
C. What to do when there is a port mismatch between server and agent?
If the following ports are not the same between server and agent, this will result to agent OFFLINE issues.
From server's ofcscan.ini From agent's registry

Master_DomainName Server
Master_DomainPort ServerPort
Master_SSLPort ServerSSLPort
40 / 206
What is the possible reasons why there is a port mismatch?
· Agent migration failed
· Client used an old installation package using a different port
· The server configuration have changed (e.g. Hostname, IP address)
· The agent is reporting to a different server.
To resolve this issue use ipxfer utility tool to transfer or re-establish communication between
OfficeScan/Apex One agents and server.
See KB 0127004 for more details on how to use the tool.
D. How to check if Apex One Server is able to communicate with the agents?
Note: The following procedures are only done on the Apex One Server
1. Access this URL using Internet Explorer:

https://<IP_Hostname_ApexOneAgent>:Client_LocalServer_Port/?CAVIT
· Expected result: !CRYPT!
2. Check verconn.log on <installation path>\PCCSRV\Log:
· Look for the target IP address

· Sample of verconn.log:
E. How to check if agent is able to communicate with the OfficeScan server?
Note: The following procedures are only done on the OFFLINE Apex One agents
Access the following links using Internet Explorer:
1. https://IP_FQDN_ApexOneServer:Master_SSLPort/officeScan/download/server.ini
41 / 206
o Expected result: see server.ini or download the file
2. https://IP_FQDN_ApexOneServer:Master_SSLPort/officeScan/cgi/cgionstart.exe
o Expected result: -2
3. https://IP_FQDN_ApexOneServer:Master_SSLPort/officeScan/cgi/isapiclient.dll
o Expected result: -1
F. How to check if FQDN is working?
Note: The following procedures are only done on the OFFLINE Apex One agents
Sometimes, if we only use Apex One server IP address to Telnet, it may accessible but via FQDN, it will
fail. Thus, agent using FQDN to contact Apex One server might encounter DNS problem.
To verify this:
1. In CMD, try to run: nslookup <ApexOneServerFQDN>.
2. It should display DNS resolution of Apex One Server IP Address.
3. You may try to download server.ini (See How to check if a gent is a ble to communica te with the
OfficeSca n server?) via FQDN and check whatʼs being used by the Agent from C:\Program Files (x86)
\Trend Micro\OfficeScan Client\AU_Data\AU_Log\Tmudump log.
Sample tmudump log where agents are accessing the Apex One server via FQDN:
42 / 206
Inf 20200319 12:10:23 6896 28972 Downloading [https://apex-one-
server.com:4343/officescan/download/server.
ini] to [C:\Program Files (x86)\Trend Micro\Security Agent\AU_Data\AU_Temp\
6896_28972\server.ini]...
4. Another option is to get the agents registry info Under

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\TrendMicro\PC-
cillinNTCorp\CurrentVersion\Misc.\UpdateFrom and append server.ini
Download server.ini by accessing the link via browser

e.g. https://apex-one-server.com:4343/officescan/download/server.ini
43 / 206
How identify Internet Information Services (IIS) Issues?
A. Check if OfficeScan IIS Web Site is running.
1. Web server status should be on Running state
In Apex One Server, go to Run > Type: inetmgr > Expand localhost > Site > OfficeScan
2. Ensure IIS Admin Service and World Wide Web Publishing Service are on Running Status
2.1 In Apex One, go to Run > Type: services.msc
B. How to verify the isapiClient.dll version used on IIS?
How to verify the isapiClient.dll version:
1. Open Run window and type inetmgr
2. Go to Application Pools
3. Right-click on OfficeScan AppPool then go to Advance Settings
4. Check the value of Enable 32-bit Application
o If it is set to FALSE, you should be using isapiClientx64.dll
o If it is set to TRUE, you should be using isapiClientx86.dll
To counter-check the file:
44 / 206
1. Go to ...\Apex One\PCCSRV\Web_OSCE\Web\CGI\
2. Look for isapiClient.dll
3. Compare the size of the file with the following:
o If the size is same as isapiClientx64.dll, you are using 64-bit isapiClient.dll
o If the size is same as isapiClientx32.dll, you are using 32-bit isapiClient.dll
Sample screenshot for 64-bit isapiClient.dll
C. Check if OfficeScan/Apex One Server and Database services are running
In Apex One/SQL Server, go to Run > Type: services.msc
The following services should be on Running state
1. Apex One Server
· Apex One Master Service
· Apex One Active Directory Integration Service
· Apex One Apex Central Agent
· Apex One Common Client Solution Framework
45 / 206
· Apex One Deep Discovery Service
2. SQL Server
· SQL Full-test Filter Daemon Launcher (MSSQLSERVER)
· SQL Server (MSSQLSERVER)
46 / 206
How to check if there is TLS issue?
If the Server-Agent communication are established but still agent shows an Offline status from Agent Management
console, kindly check also the machineʼs TLS supported version.
There is a known issue arises after upgrading to XG SP1 due to advancements in secure communications
(HTTPS protocol using TLS). Older operating systems do not natively support TLS 1.2 as their default secure
protocol.
A. To verify if your agent has incompatible protocol issue
1. In Agentʼs ofcdebug.log, you can see these error lines:
o Windows Error Code: 12030
o nError = -27 means LOADHTTP_ERROR_FAIL_SEND_HTTP_REQUEST

2. Check Windows Event Logs, there are several Schannel errors (Event ID 36871):
"A fatal error occurred while creating a TLS client credential. The internal error state is 10013."
3. In Wireshark logs, please follow the TLS Steam of the Client Hello TLS handshake.
The client initiated a Client Hello to the server with Version: TLS 1.0.
The server sent a Reset packet [RST, ACK] indicating that the connection has been terminated.
47 / 206
B. To address this issue:
1. Ensure Windows will negotiate the highest mutual supported version of TLS by the server and client.
Older operating systems may require specific patches to support newer protocols. Please refer on this
article for further information on TLS 1.1 and 1.2 for your reference.
KB reference: https://success.trendmicro.com/solution/1119045
2. You can also use IISCrypto.exe (Download Link: https://www.nartac.com/Products/IISCrypto/Download)

a. Run it as Administrator from the machine.
b.Compare the protocols between the server and client then enable the highest mutual supported version of
TLS.
c. Please reboot the machine to fully take effect the changes.
This is an example of successful TLS Protocol communication.
48 / 206
Check License and Configuration
On this section, license and configuration that can affect the agent status will be discussed:
1. Licensing
2. Checking DB Connection
3. NATed agents
How to check Apex One License?
Ensure license is not expired and it should be on Activated Status. Verify as well that is has still enough Seat
counts to properly accommodate your registered Agents.
In Apex One Server, go to Administration > Settings > Product License:
How to check if Apex One and SQL Server can establish connection?
A. Check connection between Apex One and SQL Server
1.1 In Apex One server, navigate to \PCCSRV\Admin\Utility\SQL

1.2 Double click the SQLTxfr.exe to run the tool
1.3 Input necessary credentials > Click Test Connection.
49 / 206
2. Ensure credential inputted from SQLTxfr.exe Tool with connection successful should be identical from Apex
One Server.
B. Steps on fixing DB issues
1.1 If there's no connection between, perform the following to reconnect Apex One Server to its SQL Server
using SQLTxfr.exe Tool with its necessary credentials. See link below for your further reference:
Online Documents: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-2019-server-online-
help/managing-the-product/managing-the-product_001/sql-server-migration/sql_tool_use.aspx
1.2 If DB seems to be corrupted with table missing or manual removed by mistake, perform backing up and
restoring the Apex One SQL Server database with its last known good configuration. See link below for your
further reference:
KB: https://success.trendmicro.com/solution/1113252-backing-up-and-restoring-the-officescan-sql-server-
database
50 / 206
2. Ensure credential inputted from SQLTxfr.exe Tool with connection successful should be identical from Apex
One Server.
How to check if heartbeat is enabled on NAT Agents?

NATed agent is offline when Heartbeat is not enabled
Condition: Apex One server is published on the internet. All agents are based in office LAN.
Configuring the Heartbeat and Server Polling Features:

1. Go to Agents > Global Agent Settings.
2. Click the Network tab.
3. Go to the Unreachable Network section.
4. Configure server polling settings.
For details about server polling, see Server Polling.
a. If the Apex One server has both an IPv4 and IPv6 address, you can type an IPv4 address range and IPv6
prefix and length.
Type an IPv4 address range if the server is pure IPv4, or an IPv6 prefix and length if the server is pure IPv6.
When any agent's IP address matches an IP address in the range, the agent applies the heartbeat and
server polling settings and the server treats the agent as part of the unreachable network.
Note:
o Agents with an IPv4 address can connect to a pure IPv4 or dual-stack Apex One server.
o Agents with an IPv6 address can connect to a pure IPv6 or dual-stack Apex One server.
o Dual-stack agents can connect to dual-stack, pure IPv4, or pure IPv6 Apex One server.
b. In Agents poll the server for updated components and settings every __ minute(s), specify the server polling
frequency. Type a value between 1 and 129600 minutes.
Tip:
Trend Micro recommends that the server polling frequency be at least three times the heartbeat sending
frequency.
5. Configure heartbeat settings.

For details about the heartbeat feature, see Heartbeat.
a. Select Allow agents to send heartbeat to the server.
b. Select All agents or Only agents in the unreachable network.
c. In Agents send heartbeat every __ minute(s), specify how often agents send heartbeat. Type a value between
51 / 206
1 and 129600 minutes.
d. In An agent is offline if there is no heartbeat after __ minute(s), specify how much time without a heartbeat
must elapse before the Apex One server treats the agent as offline. Type a value between 1 and 129600
minutes.
6 Click Save.
Reference: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-2019-server-online-
help/managing-the-product/managing-the-trend_c/client_computer_sing_006/unreachable-client_c/configuring-
the-hear.aspx
52 / 206
Get the "number of Select from the list below:

agents" affected · ALL agents affected.
· only ONE agent is affected
· few or some agents are affected. How many?
Discussion:
· When all agents are offline, this may indicate that the issue is at server side, or
global network issue in customer's environment.
· If only one or a few is affected its possible that the server has no issues and the
issue is localized on the agent side.
Get the Operating · Verify if issue affects specific version of Operating System (e.g. Windows 10)
machines
Get Apex One Check the current version and build number:
Information
A. Through UI:
Get the "latest changes Check what are the recent changes done prior to the issue:
done" on the · Applied a Critical Patch/Hotfix
environment
· Change in TLS configuration
· Change network configuration
Get the Firewall/Proxy Check with the Network Team for any firewall/proxy configuration between the server
Configuration and agents
From Apex One - CDT Logs

Server · What to check when running CDT Tool?
§ Basic Information
§ Functionality
§ Update & Deployment
§ Enterprise Firewall
53 / 206
· How to replicate issue for Offline agents?
- If CDT is not working:

· Manual debug log
· Application and System Event Logs

· Latest Verconn.log (…\PCCSRV\Log)
· Backup copy of Registry
- Collect Wireshark logs

For steps in gathering Wireshark logs:
https://www.howtogeek.com/104278/how-to-use-wireshark-to-capture-filter-and-
inspect-packets/
From the affected - CDT Logs

machine
· What to check when running CDT Tool?
§ Connectivity Issue
§ Enterprise Firewall Issue
§ Update/Deployment Issue
- If CDT is not working, collect the following:

· Manual debug
· Latest Connection logs (…\Security Agent\ConnLog)
· Latest Verconn.log (…\PCCSRV\Log)
- Collect Wireshark logs

For steps in gathering Wireshark logs:
https://www.howtogeek.com/104278/how-to-use-wireshark-to-capture-filter-and-
inspect-packets/
54 / 206
D. Agent Upgrade Issues
On this section, we will be discussing troubleshooting steps when encountering outdated agents.
1. Checking of Server/Agent Communication

2. Reviewing Update Configuration
3. Checking for Mismatched Certificate
4. Upgrade File Issue
5. Checking for Update Agent Configuration
6. Unable to upgrade Windows 10
If issues are not resolved after performing the provided troubleshooting tips, collect the recommended logs and file a
case to Trend MIcro Support.
How to check for Server/Agent communication?
1. Test if server is reachable from the client and vice versa
Access the following URL from the outdated agent using Internet Explorer:
URL Expected Result ( This means OSCE server is

reachable)
https://<OSCE_Server>:<Master_SSLPort>/officesc
an/cgi/isapiClient.dll Expected feedback from browser: -1
ex:
https://10.205.0.20:4343/>/officescan/cgi/isapiCl
ient.dll
https://<OSCE_Server>:<Master_SSLPort>/officesc Expected feedback from browser:

an/download/server.ini
display server.ini content or pop-up file save
notification
https://<OSCE_Server>:<Server_Port>/officescan/cg Expected feedback from browser is: -2

i/cgionstart.exe
ex:
https://10.205.0.20:4343/>/officescan/cgi/cgionstart.
exe
55 / 206
Access the following URL from the Apex One Server using Internet Explorer
URL Expected Result ( This means OSCE server is reachable)
https://<agentʼs IP address>:<local
server port>/?CAVIT Expected feedback from browser: a page with a string of text
starting with !CRYPT! should appear.
ex: https://10.205.0.20:12345/?
CAVIT
2. Check the status of the agent: online/offline and internal/external
Make sure that the machines are showing as online and internal
1. To verify the agent status: Open web console go to Agents > Agent management and search for the target
agent > check the connection status column
2. To verify the agent location:

§ Open the agent console
· Right-click the agent icon on the system tray and click Open Security Agent
Console
§ Click the lower right icon as shown below
3. If agent is offline, see Offline Troubleshooting.
56 / 206
How to review the agent update configuration?
To upgrade the endpoint, ensure that you configure the following setting.
1. Go to Agents > Agent Management.

2. Click the Settings > Privileges and Other Settings > Other Settings tab.
3. Go to the Update Settings section.
4. In the Security Agents only update the following components dropdown, select "All components (including
hotfixes and the agent program)".
5. Click Apply to All Agents or target group of Agents
6. Check the agent registry to verify if the settings are applied:

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\TrendMicro\PC-cillinNTCorp\CurrentVersion\Misc.]
"NoProgramUpgrade"=dword:00000000
How to check for mismatched certificate?
To check if the server and agent have mismatched certificate:
A. On Apex One Server:
1. Go to <installation folder>\PCCSRV\Pccnt\Common\
2. Look for OfcNTCer.dat
3. Create a copy and change file extension to .cer
4. Click on the file then go to the Details tab.
5. Check Serial Number/Thumbprint
B. On affected agent:
1. Go to <installation>\Trend Micro\Security Agent\
2. Look for OfcNTcer.dat
3. Create a copy and change file extension to .cer
57 / 206
4. Click on the file then go to the Details tab.
5. Check Serial Number/Thumbprint
C. If the certificates are mismatched, you can copy the OfcNTcer.dat from the Apex One server to the affected
machine
1. After doing so, you can try to upgrade the agents to see if it will be successful
To further troubleshoot certificate issue see link below:
Title: Troubleshooting certificate-related issues in OfficeScan (OSCE)
Summary: This article provides information about common certificate-related issues that occur on either the
OSCE agent or server
See KB1117028 for further details
How to check for agent program upgrade file issue (newpnt.zip/newpx64.zip)?
This issue occurs when the files newpnt.zip and newpx64.zip which are for "main program upgrade" on the "update
agent” contains some legacy files.
A. On the Apex One server, download newpnt.zip and newpx64.zip under "C:\Program Files (x86)\Trend Micro\Apex
One\PCCSRV\Download\".
B. On each "update agent", please perform the actions below.
1. Check if the following files are included in the newpnt.zip and newpx64.zip.
bspatch.exe
bzip2.exe
libMsgUtilExt.mt.dll
msvcm80.dll
msvcp80.dll
msvcr80.dll
2. If yes, then unload the agent
3. Replace newpnt.zip and newpx64.zip with the files that you download from server (step a).
4. Reload the agent
How to check if customer is using Update Agent? How to check Update Agent
Configuration?
To check for Update Agents and its configuration:
58 / 206
a. Go to Updates > Agents > Update Source
b. Check if the Update Agent Settings are correctly configured
c. Check if the Update Agents are using HTTPS connection as well
d. Make sure that the Update Agents are updated

i. Check the activeupdate folder of the specific Update Agent
ii. Update Agents are online and communication to and from the normal
e. Check if the Update Agent is allowed to deploy components. Check registry to verify privilege of Update Agent
Location: HKLM\SOFTWARE\WOW6432Node\TrendMicro\PC-cillinNTCorp\CurrentVersion\Misc.
\UpdateAgent
1: Component Update
2: Domain Settings
3. Component Update and Domain Settings
4. Client Program and Hotfixes
5. Agent Program and Hotfixes and Component Updates
6. Domain settings, and Client Program and Hotfixes
7. All Privileges
f. Check where the agent is downloading the hotfix:

Location: [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\TrendMicro\PC-
cillinNTCorp\CurrentVersion\Misc.]
"RelayClientGetHotfixFrom"="https://server:port"
59 / 206
How to check issue on upgrading Windows 10 due to unsupported version of Apex One
Agent?
Microsoft has changed the upgrade process for Windows with its Windows 10 OS. Instead of a new version of
Windows every few years, they now provide a full feature upgrade approximately every 6 months.
We recommend to hold off on updating Windows to the new release until after the Apex One agents have applied the
appropriate patch, as doing so beforehand may result in incompatibilities. Incompatibilities may include performance
issues, program crashes, and even system BSoDs.
Please refer to the table below for the list of Apex One compatible version:
Window s 10 ve rs ion Ape x One Ape x One as a Se rvice

Initial Window s 10 Any Apex One version Any Apex One Saas version
Window s 10 RS1 (1607) - Anniversary Update Any Apex One version Any Apex One Saas version
Window s 10 RS2 (1703) - Creators Update Any Apex One version Any Apex One Saas version
Window s 10 RS3 (1709) - Fall Creators Any Apex One version Any Apex One Saas version
Update
Window s 10 RS4 (1803) - April 2018 Update Any Apex One version Any Apex One Saas version
Window s 10 RS5 (1809) - October 2018 Any Apex One version Any Apex One Saas version
Update
Window s 10 RS6 (1903) - May 2019 Update Apex One CP 1132 or higher Any Apex One Saas version
Window s 10 (19H2/1909) - November 2019 Apex One Patch 1 Build 2087 or higher Any Apex One Saas version
Update
60 / 206

Get the Operating System · Verify if if issue affects specific version of Operating System (e.g. Windows
of the affected machines 10)
A. Through UI:
done" on the environment · Applied a Critical Patch/Hotfix
· Change Update Agent Settings
From Apex One Server - CDT Logs

§ Upgrade & Patch & Hotfix
§ Functionality
· How to replicate issue for outdated agents?

· Manual debug log
· Ous.ini (….\PCCSRV)
61 / 206
From the affected machine - CDT Logs

· Manual debug
· Tmudump.txt (…\Security Agent\AU_Data\AU_Log)
· Upgrade log (…\Security Agent\Temp)
If customer is using Update - CDT Logs

Agents · What to check when running CDT Tool?

· Manual debug
· Tmudump.txt (…\Security Agent\AU_Data\AU_Log)
· Upgrade log (…\Security Agent\Temp)
62 / 206
E. Performance Issues
On this section, we will be discussing troubleshooting steps when encountering performance related issues.
1. Optimization of System Performance

2. Disable Windows Defender
3. Battery Configuration
4. Optimization of Apex One agent
How to optimize the system performance?
This section provides information on the number of supported agents depending on enabled features.
· The sizing data below is for reference only. It is possible for Apex One to manage more than the upper bound
recommendation below if using higher spec machines. Customers can gradually increase number of
endpoints while observing the server performance data. Actual sizing limit can vary depending on product
configurations and customer environment factors.
· Sizing data below takes into considerations that both Vulnerability Protection and Application Control
features are enabled.
· Apex One is expected to provide a comparable experience running on the same hardware as OfficeScan XG
if the new advanced features (i.e. Vulnerability Protection, Endpoint Sensor, Application Control) are not
enabled.
· Gigabit Network Interface Card (NIC) required
63 / 206
64 / 206
How to disable to Window Defender?
Running Apex One and Windows defender on the same machine can lead to the following effects:
• Slow login
• Application lockup
• Machine unresponsiveness/hang
Using the Security Center will disable Windows Defender temporarily. This means that if your computer appears
to be at risk, Windows Defender can turn itself back on automatically. Hence, please edit using the registry.
This will turn off Windows Defender for good until you manually turn it back on again.
Note: Always back up the whole registry before making any modifications. Incorrect changes to the registry can
cause serious system problems.
1. Open the registry.
65 / 206
2. Browse to below path.
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender
3. Right-click on Windows Defender folder, select 'New' on the drop-down menu and choose 'DWORD (32-bit)
Value'
Only do this if you do not see DisableAntiSpyware in the folder. If you do see it, you can skip to step 5
4. Name it DisableAntiSpyware and hit Enter

A new item will appear in the folder, with the text highlighted. Delete out the current text and type in
'DisableAntiSpyware.' Then press Enter. The item should now be saved in the folder
66 / 206
5. Double click DisableAntiSpyware and change '0' to '1'
Double-click on the new DisableAntiSpyware item. A window will pop-up to edit the DWORD. In the 'Value data'
field, enter '1.' Click 'OK.'
6. Restart your machine.

Restart your Windows device to install the new edits. Your Windows Defender should now be permanently
disabled
Note: If you do want to turn on Windows Defender in the future, follow steps 1-2, and then right-click on
'DeleteAntiSpyware' and select 'Delete.' A warning will appear — click 'Yes.' Restart your computer. Windows
Defender should now be turned back on.
67 / 206
How to configure battery high performance?
Steps on how to configure battery high performance:
1. Press the Windows + R keys to open the Run dialog box.

2. Type in the following text, and then press Enter : powercfg.cpl
3. In the Power Options window, under Select a power plan, choose High Performance
Note: If you do not see the High Performance option, click the down arrow next to Show additional plans.
On Windows XP: In the Power Options Properties dialog box, under Power Schemes tab, choose the power
scheme as Always On. If available, change the System standby and System hibernates settings to Never.
4. Click Save changes or click OK
68 / 206

Get the Operating · Verify if if issue affects specific version of Operating System (e.g. Windows 10)
machines
Information
A. Through UI:
done" on the · Applied a Critical Patch/Hotfix
environment
· Change Update Agent Settings
Check what are the recent changes done prior to the issue:
o Collect Windows Performance Recorder (WPR)
o Collect Windows Dump Files
o Collect Procdump logs
69 / 206
F. Web Console Issues
On this section, we will be discussing common issues regarding Apex One web console.
1. Apex One Master Service was stopped
How to troubleshoot when web console is showing this error "Apex One Master
Service was stopped because SQL Server is unavailable"?
A sample error you might encounter in accessing your web console is regarding the SQL Server being
unavailable:
A. Check Apex One server's connectivity to the SQL database
1. Verify if the Apex One Server can connect to the SQL database by creating a data link (UDL) file :
a. Open Notepad.
70 / 206
b. Click File > ‘Save Asʼ.
c. Select ‘Desktopʼ as the location.

d. Enter File Name ‘SQL Test.udlʼ.
e. Select ‘All Filesʼ as the ‘Save as typeʼ.
f. Click Save.
71 / 206
g. Go to Desktop and right-click the file ‘SQL Test.udlʼ, then select ‘Propertiesʼ.
h. Go to ‘Connectionʼ tab.
i. Under ‘Select or enter a server nameʼ, type the SQL Database server which hosts your Apex One
Database
72 / 206
Note : If you donʼt know what is the server name of the SQL database used by the apex one
server, open the ofcserver.ini from apex one server folder : ..Trend Micro\Apex
One\PCCSRV\Private. Search for ‘[DBServer]ʼ and the server name of the SQL database is
the value of ‘Server=ʼ :
j. Enter the username and password for the SQL account. Afterwards, select the database name of
the Apex One server, and click ‘Test Connectionʼ.
k. If the Result = ‘Test connection succeededʼ, it means that the Apex One Server can successfully
connect to the SQL database. If you are still unable to login to the Apex One console, proceed to step # 2
73 / 206
l. If the Result = ‘Login failed for user xxxxxʼ, This means that the SQL credentials you entered is
incorrect. Check with the SQL admins for the correct username/password.
m. If the Result =ʼ[DBNETLIB][ConnectionOpen (Connect()).]SQL Server does not exist or access

deniedʼ, this means that the Apex One server cannot connect to the SQL server or the SQL server is
down. Check with the network team for the network connection and/or check with the SQL Database
admins if the SQL services are running.
2. If there are some changes to the SQL account used by the Apex One server to connect to the SQL
database, update the account information by using the ‘SqlTxfrʼ Tool:
a. Go to Apex One folder ..Trend Micro\Apex One\PCCSRV\Admin\Utility\SQL
b. Right click ‘SqlTxfr.exeʼ and select ‘Run as Administratorʼ
74 / 206
c. Enter the ‘Server Nameʼ, Correct SQL Username/Password and the Database Name.
d. Click ‘Test Connectionʼ before proceeding
75 / 206
e. If there are no errors encountered, click ‘Startʼ and select ‘Yesʼ on the prompt that will appear.
76 / 206
f. Select ‘Yesʼ to confirm application of new connection settings
g. Exit the program once done
77 / 206
3. Restart the Apex One Master Service and try to access the Apex One web console again.
78 / 206
Get Server Information Verify OS Type, ServicePack, and Microsoft Hotfixes installed
Get SQL Information Check the SQL Server version and authentication used
Information
A. Through UI:
From Apex One - CDT Logs

Server · What to check when running CDT Tool?
§ Functionality

· Manual debug log
· Diagnostic Log
· UI Network Traffic Log
· Ofcserver.ini (PCCSRV\Private)
· IIS Logs
· Folder C:\Windows\System32\inetsrv\config\
79 / 206
G. Smart Protection Server (SPS) Issues
On this section, we will be discussing common issues regarding Apex One's Smart Protection Sources.
1. Unable to Login to SPS Unable to Login to SPS console

2. Unable to Login using Root Password
3. Changing SPS IP Address
4. Web Reputation Service (WRS) and File Reputation Service (FRS) shows Unavailable
Troubleshooting unable to login to SPS console
You are unable to log in to the SPS console and you get the error "Insufficient free disk space".
The issue occurs because the SPS Web services keeps on crashing and it generates too many core
dumps when Predictive Machine Learning (PML) service requests are heavy.
To resolve this issue, do the following:
Important: Open SPS CLI to issue the commands in steps 1 to 3.
1. Execute the following command to stop the lighttpd service:

#service lighttpd stop
2. Execute the following command to clear the crush dump:

#rm -f /var/coredumps/*
3. Execute the following command to start the lighttpd service:

service lighttpd start
4. Verify if the SPS Web console is now accessible.
5. Apply the SPS Critical Patch based on your SPS version:
For SPS 3.1 - SPS 3.1 Critical Patch Build 1064

For SPS 3.2 - SPS 3.2 - Critical Patch Build 1090
For SPS 3.3 - SPS 3.3 Critical Patch Build 1076
80 / 206
Troubleshooting unable to login using "root' password
To reset the root password in SPS server.
1. Restart the server.
2. Interrupt the boot process by pressing the Space Bar when the Grub menu appears.
3. Press ‘eʼ to edit the selected item (i.e. Trend Micro Smart Protection Server (3.10.0-693.2.2.e17.x86_64) 3ʼ.
4. Scroll down and delete the line“ ro crashkernel=auto rd.lvm. lv=sps/root rd.lvm.lv=sps/swap rhgb quie.
5. Delete “rhgb quiet” and type in “rw init=/sysroot/bin/sh”.
Note: The key to this step is to not remove the LVM/DISK LABELS or the boot will fail.
81 / 206
6. Press Ctrl-X to start.
7. Access the system with the command: chroot /sysroot and then press Enter.
8. Type passwd and create a new password for your root account.
9. Execute “exit” to terminate chroot state which started in step 7 or the reboot commands will not work.
Note: Both “init 6” or reboot” commands work after “exit”, but “shutdown –r now” will not work in this
mode.
10. Reboot the server
82 / 206
How to change SPS IP address?
At SPS3.3, you must change "/etc/issue" also to have the IP shown on CLI changed.
Here are the complete steps to change IP address:
1. Logon SPS via CLI with "root" account.
2. Type the below command to change SPS IP address.

/etc/trend/svanetwork set ethernet static "<new IP address>" "<subnet mask>" "<gateway IP
address>" "<vlan ID>"
Note:
The parameters of svanetwork after "ethernet":
"static": To set static IP
"<new IP address>": The static IP address for this TMSPS server.
"<subnet mask>": Subnet mask
"<gateway IP address>": Gateway route IP address
"<vlan ID>": The ID of VLan. Default set to "0".
Example:
/etc/trend/svanetwork set ethernet static "192.168.0.1" "255.255.255.0" "192.168.0.254" "0"
3. Run command to change IP in "etc/issue". Ignore this action before 3.3.

sed -i 's/<old IP address>/<new IP address>/g' /etc/issue
Example:
sed -i 's/192.168.0.224/192.168.0.1/g' /etc/issue
4. Reboot SPS
5. Verify the IP on the CLI welcome page and the connection
83 / 206
Web Reputation and File Reputation Services
The Standalone SPS Console shows an X mark in both File Reputation and Web Reputation Services. The
following error also appears in the Reputation Service Log:
Cannot read monitor.ini configuration file. Verify the file exists or check the permissions.
This issue causes the Smart Scan agents to get a "Smart Scan Unavailable" error or a "Connecting" status since
the Apex One serverʼs update source is the Standalone SPS.
1. Log on to SPS Server and go to /var/tmcss/conf directory using the following command:
cd /var/tmcss/conf
2. Check if monitor.ini file exist using ls command.
The following shows the monitor.ini file does not exist:
If the file does not exist, there are 2 options to resolve it.
Option 1: Recreate the monitor.ini
Option 2: Copy the monitor.ini from a working SPS Server with the same version. (If no other SPS server is
available, it can be requested from Technical Support)
84 / 206
Option 1: Recreate the monitor.ini file
1. Log on to the SPS server as Administrator.
2. Stop the SPS service – lighttpd
3. Using cd, run the following command then hit Enter.

cd /var/tmcss/conf
4. Create the monitor.ini file using touch command then hit Enter:
touch monitor.ini
5.Using the ls command, verify if the file has been created then hit Enter.
ls –lrt monitor.ini
Note: The monitor.ini should have 0 file size.
6.Change the ownership of the file to webserv using the following command then hit Enter.
chown webserverv:webserv monitor.ini
7. Using ls, execute the following command then hit Enter. Verify the ownership and file size.
Notice that the file size is now at 107 and the owner is webserv.
8.Start the lighttpd service under /var/tmcss directory then hit Enter.
85 / 206
Option 2: Copy the monitor.ini file from a working SPS Server.
Important: The Source SPS Server Version should be the same as the affected SPS Server.
1.At the Source SPS Server, stop the lighttpd service using the following command.
2. Log in again to the SPS console. File Reputation and Web Reputation should now have check
marks next to them.
86 / 206
Best Practice Configuration
Ensure all SPS URLs are allowed in firewall:
Pattern Update https://slspn30-p.activeupdate.trendmicro.com/activeupdate/

https://slspn30wr-p.activeupdate.trendmicro.com/activeupdate/
https://slspn30wrcom-p.activeupdate.trendmicro.com/activeupdate/
https://slspn30wrnewd-p.activeupdate.trendmicro.com/activeupdate/
Smart Feedback https://tmsps300-en.fbs20.trendmicro.com:443/
https://tmsps30p2-en-wis.trendmicro.com
Smart Protection Proxy http://tmsps300-en.census.trendmicro.com
http://tmsps330-en-domaincensus.trendmicro.com
https://grid-global.trendmicro.com
https://rest.mars.trendmicro.com
http://tmsps30-en.grid-gfr.trendmicro.com
How to enable TLS 1.2 support in Smart Protection Server
Enabling TLS 1.2 on SPS 3.3 This would disable SSL 2.0 and SSL 3.0
Important: TLS 1.2 can only be enabled by turning on supported ciphers. Instructions below provide
information of TLS 1.2 supported ciphers only.
Customers who adopted this instruction were advised to test compatibility with browsers and applications in
staging environment first.
Important: SPS version 3.1 or later is required.
1. Log in to command shell.
2. Execute the following command:
vi /etc/lighttpd/lighttpd.conf
3. Replace "var.ssl-cipher-list" with var.ssl-cipher-list = "TLSv1.2:!eNULL:!aNULL”.
4. Save and exit vi interface.
5. Execute the following command:
service lighttpd restart
After applying the changes, SPS web console and Smart Scan will be limited to use TLS 1.2 only.
87 / 206
Get Server Information Verify OS Type, SPS Version and Build Version
Through UI:
1Access SPS web console > Help > About
A. Through UI:
done" on the environment · Applied a Critical Patch/Hotfix
· Change in TLS configuration
Get the Firewall/Proxy Check with the Network Team for any firewall/proxy configuration between the
Configuration server and agents
From iSPS Server Collect CDT on Apex One Server.

§ Functionality
§ Update & Deployment (if involving updating the server, agent)
From SPS Server To collect CDT from SPS Server:
A. Through Web Console:

o Access Web Console > Administration > Support > Click "Start". Upload
the .tar.gz file to Technical Support for further analysis.
B. Through CLI
o If unable to login to console and can't collect CDT from console, follow
the instructions on How to debug from SPS Server CLI?
88 / 206
IV. Apex One iProduct Common Issues
This section discusses troubleshooting common issues on Apex One Integrated Products (iProducts):
o Apex One Endpoint Sensor (iES)

o Apex One Application Control (iAC)
o Apex One Vulnerability Protection (iVP)
o Apex One Data Loss Prevention (iDLP)
o Apex One (Mac)
iProduct Action Code (AC) guide
Ape x One Ape x One (M ac)

AC Ke y Type s Ape x Ce ntral
AV,iDLP,VDI iES iAC iVP Ape x One iES
(M ac)
o o o o o
Ne w Ke y Apex One Full Feature
(Window s & Mac)
o o
Apex One Endpoint
Sensor
o
Le gacy Ke ys TMCM Advanced
(Stand alone
products )
o
OSCE
o
TMVP
o
TMEAC
o o
TMES
o
TMSM
Activation Key Types Entitlement Scope

Trend Micro Control Manager (TMCM) AC will still work on Apex Central
Apex Once Full Feature Covers all Apex One 2019 features except for Apex One
Endpoint Sensor (iES) & Apex One Sandbox as a Service.
Please contact TM Sales to purchase add-on features.
Apex One Endpoint Sensory Covers Apex One Endpoint Sensor feature for both Apex One
& Apex One (Mac)
Trend Micro Endpoint Application Control AC will work on Apex One to activate Application Control
(TMEAC) Integration (iAC) feature but must be deployed via Apex Central
Trend Micro Vulnerability Protection (TMVP) AC will work on Apex One to activate Vulnerability Protection
integration (iVP) feature but must be deployed via Apex Central
Trend Micro Endpoint Sensor (TMES) AC will work on Apex One to activate Endpoint Sensor (iES)
feature but must be deployed via Apex Central
89 / 206
90 / 206
A. Apex One Endpoint Sensor (iES)
Installation of Apex One Endpoint Sensor

a. It can be installed during the installation of Apex One Server.
b. If user opted to skip the process of installing Endpoint Sensor during the installation of Apex One server,
iES can be installed through Maintenance mode: https://success.trendmicro.com/intkb/solution/1123009
How to verify if Endpoint Sensor (iES) is installed correctly?
Installation logs
o C:\w indow s\TMESSetupDebug.log
o C:\w indow s\iATASSetupDebug.log
o C:\w indow s\ OFCMAS.log
Endpoint Sensor Files

o <installation path>\Trend Micro\Apex One\iServiceSrv\iES
o <installation path>\Trend Micro\Apex One\iServiceSrv\iATAS
Review iES related Services

· Tre nd M icro Endpoint Se ns or Se rvice : Service Status: stopped (not activated yet)
· Tre nd M icro Advance d Thre at As s e s s m e nt Se rvice : AtasService status: stopped (not activated yet)
Endpoint Sensor Application Pool

o OfficeScan_iATAS_AppPool
o OfficeScan_iESAgent_AppPool
o OfficeScan_iESConsole_AppPool
Endpoint Sensor IIS Sites

o OfficeScan > officescan_iesagent
o OfficeScan > officescan_iesconsole
o OfficeScan > officescan_iatas
91 / 206
92 / 206
Below are common reasons why iES installation fails:
A. Installation Failed due to iES database:
· Check the installation logs C:\Windows\TMESSetupDebug.log

log snippet:
Initializing deployment (Start)
Intializing deployment (Failed)
, StdErr=**** Could not deploy package. Unab le to connect to master or target server 'OSCE-ApexOne-iES'. You myst have a
user with the
05-02 17:54:57 [1] ERROR - [UpgradeDB] [Agent Storage] Setup DB failed. [SqlComponent.cs - (89)]
05-02 17:54:57 [1] DEBUG - after install -1
05-02 17:54:57 [1] ERROR - Install::InstallPlugins() - Failed to install plugin
05-02 17:54:57 [1] INFO - 801
05-02 17:54:57 [1] DEBUG - -------Done-------
· Send the installation log to support
B. Installation Failed due to FIPS enabled:
· It is a known issue that iES cannot be installed if FIPS is enabled

· Check the installation log C:\Windows\TMESSetupDEbug.log
log snippet:
ERROR - System. Invalid OperationException: This implementation is not part of the Windows Platform FIPS validated
cryptographic algorithms. at System.Security.Cryptograpy.RijndaelManaged..ctor() at
Cryptography.AesProvider.AesEnryptTransforms(String key, String iv) at Cryptography.AesProvider.EncryptAES256(String
srouce) at Setup.Helper.Installation.Install.ConvertToXmlDataPair(Dictionary'2& inputPair, Dictionary'2&dataPair at
Setup.Helper.Installation.Install.FreshInstallFlow(String[]&msgBody)at Setup.Helper.Installation.Install.FreshInstall(String[]
&msgBody) at Setup.Helper.Installation.Install.Upgrade(String[]&msgBody) at
Setup.Helper.Installation.InstallationHelper.ProcMessage(String MsgId, String[]MsgBody) at Setup.Program.Main(String[]args)
· Solution: File case to Support and request for Apex One Hot Fix 2121
93 / 206
Activating Apex One Endpoint Sensor (iES)
Endpoint Sensor Service: Unknown Error
Status Description
Unsuccessful License deployment was unsuccessful
Endpoint Sensor Service: Unknown Error
Troubleshooting steps:
A. Check if all iES and iATAS components are complete:
1. Check if iES and iATAS services are existing but not running
2. Check if iES and iATAS AppPools are existing
3. Check if iES and iATAS IIS Sites are complete
4. If any above are incomplete, reinstall iES Server and iATAS server.
a. Remove iES and iATAS: https://success.trendmicro.com/solution/1122946
b. Reinstall iES and iATAS: https://success.trendmicro.com/solution/1123009
B. Check if Apex One Server is using 3rd-party certificate:

See How to check if Apex One Server is using 3rd-party certificate?
C. iES has incorrect DBName:
1. Check SQL server and compare the DB name of Apex One and iES
2. The iES DB name should be the same as Apex One with -iES appended to it.
3. If the DB names are different, check the config.xml on <installation path>\Apex One\iServiceSrv\iES for
the DB Name
4. Stop the iES Services and open config.xml

5. Rename the iES database with the database name based on the config.xml
6. Restart the iES Services.

7. Try again to deploy the license.
94 / 206
Apex One Endpoint Sensor (iES) Policy Deployment Issue
Apex Central Issue
1. Test connectivity between Apex Central and Apex One Server. From Apex Central "ping IP/FQDN of Apex
One Server" and verify if the server is reachable.
2. Make sure that SSO from Apex Central to Apex One is working properly
1. Products SSO
a. Access Apex Central console.
b. Go to Directories > Products.
c. Go to Local Folder > <Apex One Folder> > Apex One Server
d. Click on Apex One Entity > Configure > Apex One Single Sign-On
e. SSO should be working.
2. Managed Servers SSO
a. Access Apex Central console.
b. Go to Administration > Managed Servers > Server Registration.
c. Change Server Type to Apex One.
d. Click on the URL for Apex One.
e. SSO should be working.
Apex One Issue
1. Policy status “Pending: Managed server deploying”
§ Check if Apex One Server is using 3rd-party certificate:

See How to check if Apex One Server is using 3rd-party certificate?
2. Endpoint Sensor Server: System Error: Error ID: 420
An "Error ID: 420" occurs while the Apex One Endpoint Sensor policy is deployed and the "Unable to get the
registered server list. There are no registered servers." error appears on the Apex Central "Preliminary
Investigation" page.
95 / 206
Symptoms
o From diagnostic.log, iATAS is not started so parent proxy will not call execute function to iESProxy
o From iATASSetupDebug.log, you may find "access denied" errors during ATAS upgrade
1. Check if Trend Micro Advanced Threat Assessment Service (iATAS service) is running
2. If it is stopped or cannot be started, reinstall iATAS Service.
3. Uninstall iATAS:
a) launch a command prompt with administrator privilege and navigate to

...\Trend Micro\OfficeScan\PCCSRV\Admin\Utility\iServicePackage\iATAS\Setup\.
b) Run the following command: iATASSetup.exe -uninstallation
4. Reinstall iATAS using Maintenance Mode: https://success.trendmicro.com/solution/1123009
Apex One agent Issue
o Endpoint Sensor Service: 201509003:

§ The error means Installation failed
What to check?
1. Check if the agents are getting the update from Apex One server or an Update Agent
2. If the agent is getting an update from Update Agent, make sure that complete Update Agent
files
o Endpoint Sensor Service: 201504423:

§ This is a generic timeout error
§ Please try to reboot those affected machines then try to redeploy the policy.
96 / 206
Useful Links
Title KB
Error ID Mapping for policy deployment status of Apex Central: See KB 1122453
Removable of standalone plug-in products: See KB 1122946
97 / 206
Installation Issue 1. CDT Logs from Apex One Server

a. Basic Information
2. Installation Logs from Apex One Server
a. C:\windows\TMESSetupDebug.log
b. C:\windows\iATASSetupDebug.log
c. C:\windows\ OFCMAS.log
License Issue 1. CDT logs from Apex Central Server
a. Update or Deployment Issues
b. General Issues
2. CDT logs from Apex One Server
b. Installation
c. Functionality
d. Update & Deployment
Policy Deployment 1. CDT logs from Apex Central Server
a. Web User Interface
b. Update or Deployment Issues
c. General Issues
2. CDT logs from Apex One Server
b. Functionality
c. Update or Deployment
3. If error is on Agent, CDT logs from affected agent:
b. Connectivity Issue
c. Update/Deployment Issue
d. Endpoint Sensor
98 / 206
B. Apex One Application Control (iAC)
Policy Deployment Flow for iAC
NOTE:
o Application Control Server and Apex One Server are two components in one server
o Application Control Agent and Apex One Security Agent are two component in one client.
How to check Apex One Server status in Apex Central?
1. Logon to Apex Central Management Console.

2. Go to Directories > Products tab.
3. Expand Local Folder and look for the Apex One Server.
4. Verify that it has a green check beside the Apex One Server Name.
99 / 206
IMPORTANT: Make sure that the Apex One Server is NOT in the “New Entity” folder. Else you will not be able to
deploy policy to it.
How to verify iAC service status in Apex One Server?
iAC Services
1. Logon to the Apex One Server machine.
2. Open Services Console (services.msc).
3. Look for the Trend Micro Application Control Service and verify the status is Running.
iAC Folders

2. Go to %PROGRAMFILES%\Trend Micro\iService and make sure iAC folder exists.
iAC Registry Keys

2. Open Registry Editor (regedit.exe)
3. Go to HKLM\SOFTWARE\WOW6432Node\TrendMicro\iAC and make sure the following registries
exist.
iAC Database
100 / 206
1. Open SQL Management Studio.
2. Connect to the SQL Server where Apex One Database is created. (You may need assistance from a
DB Admin who have administrative access to SQL Server Database.)
NOTE: To know the SQL Server and Database Name, login to the Apex One Web Management console
and go to Help > About.
3. Expand the Apex One Database tables and make sure that you see all the iac.* tables.
iAC in IIS Manager

2. Click Start > Run and type inetmgr.exe. Then hit enter to open IIS Manager
3. Go to Application Pools and verify that the OfficeScan_iAC_AppPool is started.
101 / 206
4. Go to Sites > OfficeScan and verify that the OfficeScan_iAC virtual website and sub-folders exist.
102 / 206
Apex One Server Certificates
1. IIS Certificate:
2. Open IIS Manager.
3. Go to Sites > OfficeScan.
4. Under Action, click Bindings… to open Site Bindings dialog box.
5. In the Site Bindings dialog box, select https and click Edit to open Edit Site Bindings dialog box.
6. In the Edit Site Bindings dialog box, take note of the SSL certificate.
7. Verify Installed Certificates in the Local Machine Certificate Store.

8. Click Start > Run and type “certlm.msc” to open Local Machine Certificate Store Management
Console.
103 / 206
9. Go to Trust People > Certificates and make sure that the following certificates exist:
NOTE: The apexone.trend.local should be the same as the SSL Certificate found in the IIS Manager.
10. Go to Personal > Certificates and make sure that the follow certificate exists:
NOTE: The apexone.trend.local should be the same as the SSL Certificate found in the IIS Manager.
104 / 206
How to verify iAC service status in Apex One Agent?
1. Logon to the Apex One Security Agent machine.

2. Open the Services Console (services.msc).
3. Make sure that the following service exists and started.
iAC Folders
2. Go to %PROGRAMFILES%\Trend Micro\iService\iAC and make sure the following sub-folders exist.
iAC Registry Keys

2. Open Registry Editor (regedit.exe).
3. Go to HKLM\Software\TrendMicro\iACAgent and verify the following registry keys exist.
4. Go to HKLM\System\CurrentControlSet\services\AcDriver and make sure the following registry keys

exist.
105 / 206
Agent Console iAC “Enabled” status
2. Right-click the agent icon on the system tray and select Security Agent Console.
3. Go to Apex One Security Agent and make sure that the Application Control is green.
106 / 206
Troubleshooting iAC Policy Deployment
Policy Error “Product Communication Error”
This error can happen when Apex One and Apex Central are installed on the same server.
To resolve this, follow the steps below:
1. Logon to the Apex One-Central Server.

2. Click Start > Run and type inetmgr.exe. Then hit enter to open IIS Manager.
3. Go to Application Pools and verify if the OfficeScan_iAC_AppPool is started. Otherwise, right-click the select
Start.
4. Restart the Apex One IIS Website.
5. Redeploy the Policy.
107 / 206
Policy Error “Application Control Service: Unactivated licenses”
A. Verify iAC has valid license.
1. Login to the Apex Central Web Management Console.

2. Go to Administration > License Management > Managed Products.
3. Verify that all the licenses are valid.
4. If any of the above license is expired, verify if it is for iAC. If this is the case, kindly contact your Trend
Micro Sales to help in re-activating the license.
B. Unable to deploy iAC Activation Code.
You get the following error when deploying iProduct valid licenses.
The issue can happen if the the Apex One SQL Database is assigned a Windows Account to manage. It may not
have sufficient web service framework access permissions. Fix this by adding the Windows Account to Apex One
Serverʼs IIS_IUSRS Local Groups.

2. Open Computer Management Console.
3. Go to Local Users and Groups > Groups.
4. Configure the IIS_IUSRS group, add the Windows Account.
108 / 206
5. Re-deploy the Policy.
C. Disable "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing".
The error appears in the C:\Windows\OFCSVR.log.MM_DD_HH_MM_SS.log.
1. Login to the Apex One Server.

2. Open Local Security Policy console (secpol.msc).
3. Go to Local Policies > Security Options.
4. Change the System cryptography: Use FIPS compliant algorithms for encryption, hashing, and
signing to Disabled.
109 / 206
Policy Error “Pending: Waiting for product agent”
Policy to enable Application Control will always show "status pending" on the Apex Central console.
Application Control module cannot download policy setting because of the certificate verification failure. The
following Apex One Agent debug log can be seen.
From OFCDEBUG.log
For this, verify properties of the Apex One Server SSL Certificate.
A. Verify if the certificate is not expired and it is allowed to issue policy for all.
1. Open Local Computer Certificate Store and go to Trusted People > Certificates.
2. Double-click the Apex One Server SSL Certificate and make sure that All issuance policies exists
and the validity is not expired.
110 / 206
B. If using a 3rd Party or Corporate Certificate Authority (CA)
Follow the KB Article below to properly configure it with Apex One Serverʼs SSL Certificate.
Configuring Apex One to use a certificate signed by corporate Certificate Authority

https://success.trendmicro.com/intkb/solution/1122205
111 / 206
I. Using Case Diagnostic Tool
Use the article below for steps in how to use Trend Micro Case Diagnostic Tool to collect needed logs for
troubleshooting purposes.
Using the Case Diagnostic Tool (CDT) to collect the information needed by Technical Support
II. Manually Collecting iAC-related logs files.
iAC Server Installation Logs

C:\Windows\OFCSVR.log
C:\windows\iATASSetupDebug.log
C:\windows\ OFCMAS.log
C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iAC\config.xml
C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\OfUninst.ini
IIS Logs
C:\inetpub\logs\LogFiles\W3SVC1\u_exYYMMDD.log
C:\inetpub\logs\LogFiles\W3SVC3\u_exYYMMDD.log
MCP Agent Logs

C:\CMAgent_debug.log
C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\\CMAgent\Agent.ini
C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\\CMAgent\Product.ini
C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\\CMAgent\cmagentdebug.log
Apex One Server Debug Log

C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\\Log\ofcdebug.log
iAC Agent Installation Log
C:\Windows\TMiACAgentSetup.log
Connectivity
C:\Program Files (x86)\Trend Micro\Security Agent\\ConnLog\Conn_YYYYMMDD.log
Apex One Agent Debug Log

C:\OfcDebug.log
Apex Central Registration
Server C:\Program Files (x86)\Trend Micro\Control
Manager\DebugLog\CMEFScheduler_OSCE_iAC.log
C:\Program Files (x86)\Trend Micro\Control
Manager\DebugLog\TMCM_CascadingMCPAgentSDK.log
C:\Program Files (x86)\Trend Micro\Control Manager\DebugLog\WebUI_OSCE_iAC.log
C:\Program Files (x86)\Trend Micro\Control
Manager\WebUI\WebApp\widget\repository\log\diagnostic.log
112 / 206
C. Apex One Vulnerability Protection (iVP)
iVP Licensing Common Issues
Review Command Tracking Status
o Hereʼs the sample screenshot for successful deployment of iVP license profile from Apex Central to Apex
One server:
o After you click Deployed, wait for until the license has been activated properly.
o For additional checking, check Command Tracking. Look for Command: Deploy License Profiles
and it should have status of Successful: 1.
Review IIS and Services Status
o Check the status of iVP web service if itʼs running in IIS Manager:
Web service display name: OfficeScan_iVP_AppPool
113 / 206
o Check if iVP service on Apex One Server is healthy or not:
Server service display name: Trend Micro Vulnerability Protection Service
If the above-mentioned requirements werenʼt able to satisfy due to an error, proceed on the next steps for further
troubleshooting.
114 / 206
How to troubleshoot"iProduct Service not Starting"
Issue: iVP service on Apex One wasnʼt able to start properly.
Description: When you try to deploy iVP license from Apex Central, it fails as its iVP server service on Apex One
wasn't able to start properly.
Additional Information: When you start manually Trend Micro Vulnerability Protection, you encounter the
following error message:
Error Message: ”Windows could not start the Trend Micro Vulnerability Protection Service on Local Computer.
Error 1067: The process terminated unexpectedly”
Symptoms
· Verify System Event logs for an error, in this issue it shows:
Event ID: 7034

Source: Service Control Manager
Level: Error
General: “The Trend Micro Vulnerability Protection Service service terminated unexpectedly. It has
done this 10 time(s).”
· Based on ivp_server0.log (C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP\):
SEVERE: Unab le to send log to OSCE.

com.trendmicro.ivp.integration.osce.osf.web service.OSFWeb ServiceException: OSF SystemCall result code: 10006
at
com.trendmicro.ivp.integration.osce.osf.web service.ob ject.OSFWeb Request.getResultData(OSFWeb Request.java:120)
at com.trendmicro.ivp.core.command.osf.OSFOnLogCommand.run(OSFOnLogCommand.java:512)
at java.util.concurrent.Executors$Runnab leAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Troubleshooting
Check the Java Version in Windows:
1. Open the Start menu and click Control Panel.

2. Type "Java" into the search field and double-click the Java icon.
The Java Control Panel appears.
3. Click the General tab if it is not already open.
115 / 206
4. Click the About button. It shows:
e.g. Java File version: Version 8 Update 221 (build 1.8.0_221-b11)
Action Plan
1. Check the iVP server version:
a. Log in to the Apex One server computer.

b. Go to the iVP server installation folder (C:\Program Files (x86)\Trend Micro\Apex
One\iServiceSrv\iVP).
c. Right-click the Properties for iVPServer.exe.
d. Check the Details tab.
You can find the iVP server version.
e.g. iVPServer.exe File version: 3.0.0.2041
2. Check the iVP server installation source file version.

e.g. C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP
a. Go to C:\Program Files (x86)\Trend Micro\Apex One\PCCSRV\Admin\Utility\iServicePackage\iVP.

b. Find iVPServerInstaller.exe and right-click its Properties.
c. Check the Details tab.
You can find the iVPServerInstaller.exe version there.
e.g. iVPServerInstaller.exe File version: 3.0.0.2041
d. Check the version for iVPServer.exe and iVPserverInstaller.exe to see whether it is the same or not.
For example, if the version for iVPserverInstaller.exe is 3.0.0.2055 and the iVPserver.exe version is
3.0.0.2041, then the version is not the same. This means that maybe the upgrade failed for the iVP server.
The version should be the same for iVPserver.exe and iVPserverInstaller.exe.
3. Check the BundledJava version:

a. Log in to Apex One server computer.
b. Go to C:\Program Files (x86)\Trend Micro\Apex One\BundledJava\Bin.
c. Check the java.exe version and right-click its Properties.
d. Check the Details tab.
You can find the java.exe version there.
e.g. java.exe File version: 8.33.0.1
If you see that the bundleJava version is 8.x.x.x (not 11.31.0.11) and the JRE version is 8.x.x.x., the iVP
server upgrade will fail. It needs to download JRE 11.31. For example:
116 / 206
e. Stop Apex One Master Service.
f. Back up and delete the files in C:\Program Files (x86)\Trend Micro\Apex One\BundledJava\.
g. Unzip the downloaded JRE files and put all of the files in C:\Program Files (x86)\Trend Micro\Apex
One\BundledJava\.
4. Upgrade iVP server manually:

a. Open a command line with administrator privilege and cd to C:\Program Files (x86)\Trend Micro\Apex
One\PCCSRV\Admin\Utility\iServicePackage\iVP.
b. Type the following command:
start /wait iVPServerInstaller.exe -q -dir "C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP"

-VskipOSCEIntegration="true" -VskipPrepareConfig="true" -Dinstall4j.keepLog=true -
Dinstall4j.alternativeLogfile="C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP\install.log""
c. Wait for a while then go to C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP and check
ivp_server0.log to see whether it has an error log or not.
d. Manually start the Trend Micro Vulnerability Service (iVPServer.exe).
It should run properly now.
Note: If the steps above donʼt work, please collect the iVP_server0.log file as well as the screenshot for
the version of java.exe, iVPserver.exe, and iVPserverInstaller.exe for reference.
How to troubleshoot Certificate Issue "License Deployment was Unsuccessful"
Issue: “License deployment was unsuccessful”
Description: License deployment fails when deploying iVP license from Apex Central.
Error Message: “License deployment was unsuccessful. Vulnerability Protection Service: Unknown Error”
Based from the Command Tracking:
Symptoms
1. Check ofcdebug.log, the following error can be seen:
Log Information:
117 / 206
[ofcservice.exe]OSFSvcClient::setProductServiceInfo - failed to get iService info -
[libosfsvcclient.cpp(73)]
2. Check ivp_server0.log, location C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP\
Log Information:
Apr 07, 2019 1:33:32 PM com.trendmicro.ivp.core.Core main SEVERE: Failed to start iVP server.
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building
failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid
certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
Analysis: The SSL handshake error indicates that iVP cannot find Apex One's console certificate. This
issue usually happened when customer used 3rd party signed certificate on Apex One.
How to troubleshoot?
The following initial steps can be done to review customer certificate.
1. The Apex One server private key must be exportable.
How to import certificate and allow private key to be exported?
1. On the certificate console of Local Computer, choose Personal > Certificates.

2. Right click on the right panel and choose All Tasks > Import...
3. Check the checkbox of 'Mark this key as exportable...' in the import wizard.
118 / 206
How to verify if private key is allowed to be exported?

2. To view the certificate, double click target certificate.
Go to Details tab, click Copy to File to open certificate export wizard
119 / 206
3. Click Next. Export Private Key ("Yes, Export the private key") option should be available.
2. The certificate must be generated with a valid Common Name ( 'CN=<HOSTNAME>'.) .

2. To view the certificate, double click target certificate.
Go to Details tab, then view Subject details. The certificate must have a valid subject.
IMPORTANT: Follow KB1122205 if customer is using a 3rd party CA signed certificate.
120 / 206
Troubleshooting Policy Deployment Issue
How to check command tracking status?
The screenshot below shows a successful deployment of iVP policy from Apex Central Server.
For additional checking, check Command Tracking.
Look for recent Apply Policy under Command column > Click the Successful results to verify if itʼs already
deployed on Agentʼs Apex One Server.
When deployment is finished, connect to the endpoint, open the Apex One Security Agent Console via system
121 / 206
tray icon and verify if Vulnerability Protection is now Enabled with its running Trend Micro Vulnerability Protection
Service (Agent).
Confirm it has identical Policy Version that was recently deployed from Apex Central.
122 / 206
If the above-mentioned requirements werenʼt able to satisfy due to an error, proceed on the next steps for further
troubleshooting.
123 / 206
Policy status “Pending: Apex Central deploying”
Problem: Communication error occurs when Apex One and Apex Central are installed on the same server.
Error message: Policy status “Pending: Apex Central deploying”
Details: This issue occurs when Apex One is installed first before Apex Central is installed
Root Cause: The installation of Apex Central will stop the IIS Application Pool for Application Control and
Vulnerability Protection.
In order to prevent this error, here are the manual steps you need to follow:
1. Run IIS (Internet Information Services) Manager and go to Application Pools.

2. Start OfficeScan_iAC_AppPool and OfficeScan_iVP_AppPool.
3. Select the IIS site and click Restart.
4. Re-deploy the policy on Apex Central.
124 / 206
Policy status “System error. Error ID: 5”
Problem: Failed to deploy iVP policy
Error Message: “System Error. Error ID: 5” status with Description: “Vulnerability Protection Service: Disabled
product services”
Symptoms
Log Snippet:
Log File: ivp_server0.log (Location: C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP\)
com.microsoft.sqlserver.jdb c.SQLServerException: The TCP/IP connection to the host localhost,

port 1433 has failed. Error: "Connection refused: connect. Verify the connection properties.
Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections
at the port. Make sure that TCP connections to the port are not b locked b y a firewall.".
Troubleshooting
How to check if Apex One Server can connect to SQL Database Server using port 1433?
1. Log into your SQL server through Remote Desktop Connection.

2. Click Start > Expand your Microsoft SQL Server folder > select SQL Server Configuration Manager.
3. Expand SQL Server Network Configuration > Click the Protocols for MSSQLSERVER.
4. Right click TCP/IP > Select Properties > Click IP Addresses tab > Scroll down to IPAll > Ensure TCP
Dynamic Ports is blank and TCP Port is set to 1433 > Click Apply > OK.
125 / 206
5. Perform to restart SQL Server (MSSQLSERVER) service.
6. Test connection from Apex One Server to SQL Server on port 1433 via PowerShell.
Success Result:
NOTE: Ensure that port 1433 is allowed on your firewall.

Mixed mode authentication should be enabled as well for remote connections.
7. Log-in to manage Apex Central and deploy iVP license again and check the results.
Policy status shows "Unable to logon Product"
Problem: Failed to deploy iVP Policy
Error Message: "Unable to automatically logon to product".
Symptoms
Log Snippet:
Log File: ivp_server0.log (Location: C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP\)
126 / 206
SEVERE: Unab le to update policy tracking records.
javax.net.ssl.SSLHandshakeException: PKIX path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to
requested target
This issue happens when the server certificate is changed. See How to troubleshoot?
Policy status “Pending: Waiting for product agent”
Problem: Failed to deploy iVP Policy when deploying from Apex Central
Error Message: “Pending: Waiting for product agent”
Symptoms
Log Snippet:
Log File: ofcdebug log
2019 09/18 12:30:25 [2154 : 201c] (00) (E) [][tmlisten.exe]VerifyServerCert - Failed to verify the SSL
certificate - [olh_winhttpclient.cpp(820)]
2019 09/18 12:30:25 [2154 : 201c] (00) (D) [][tmlisten.exe]VerifyServerCert - << 0 -
[olh_winhttpclient.cpp(827)]
2019 09/18 12:30:25 [2154 : 201c] (00) (E) [][tmlisten.exe]winHttpStatusCallb ack - Close connection due to
certificate verification failure - [olh_winhttpclient.cpp(78)]
To address this issue, ensure that thereʼs no OfcIPCer.dat mismatch between the server and agent. Compare
the certificate with server public key in Trusted People if itʼs the same. If not, export the server public key then
backup and replace it in the affected machine.
How to verify Officescan SSL certificate?

1. In IIS Manager, click OfficeScan Web Site > Click Bindings.. > Verify the current SSL certificate information being
127 / 206
used by port 4343 > Click Edit.
2. In Edit Site Bindings, click View > Go to certificate Details tab > Take note of its Serial Number.
3. Open mmc.exe and Run as administrator.

4. On the File menu > Click Add/Remove Snap-in.
5. Under Available snap-ins, select Certificates > Click Add.
6. Click Computer account > Next.
7. Click Local computer > Finish > Click OK.
8. Expand Certificates (Local Computer) > Expand “Trusted People” > click Certificates.
9. Double click certificates, search for the SSL certificate youʼve checked on Action 1 and verify if it has identical Serial
number.
128 / 206
10. Right click the SSL certificate, select All Task > Export... > Next > Next > Browse… input location path and file
name > Save > Next > Finish > OK.
11. Double click the exported certificate with file extension .cer. Take note of the certicate Serial number from the
Details Tab and compare it with the Server and Agent OfcIPCer.dat.
How to verify if the certificate of agent and server match?
The certificate's serial number from the server and agent should match.
1. Create a copy of OfclPCer.dat from the server and client.
FROM OSCE Server:

file Location: …PCCSRV\Pccnt\Common\OfcIPCer.dat
Example: To easily identify, name the copy to OfcPCer-SERVER.dat
FROM OSCE Agent

file Location: ...OfficeScan Client\OfcIPCer.dat
Example: To easily identify, name the copy to OfcPCer-AGENT.dat
2. To open the file, update the file extension from .dat to .cer
3. The serial number of the certificate from the server and agent should match
129 / 206
How to resolve certificate mismatch?
In this example, we have verified that the certificate of on Local Machine Certificate Store and
Certificate(OfcIPCer.dat) files on server and agent does not match.
Certificate Serial Number

Local Machine Certificate Store (MMC): 1a48 48 xx xx xx xx xx xx xx xx xx xx xx xx xx
Under Certifica tes (Loca l Computer) > Expand
“Trusted People” > click Certifica tes.
Server 41 33 c5 xx xx xx xx xx xx xx xx xx xx xx xx xx
Agent 41 33 c5 xx xx xx xx xx xx xx xx xx xx xx xx xx
How copy the correct certificate to the affected machine?
1. Rename the exported file as OfcIPCer.dat.

2. Copy the file to affected agent machine.
3. Unload Apex One agent.
4. Backup original OfcIPCer.dat on agent side (AGENT: ...OfficeScan Client\OfcIPCer.dat) then replace
it using the newly exported OfcIPCer.dat.
5. Load Apex One agent.
6. Re-deploy policy, check if everything is OK or not.
7. If everything is OK, please check the OfcIPCer.dat in server side (SERVER: …
PCCSRV\Pccnt\Common\OfcIPCer.dat), is it the same?
8. If not, please replace also in server side using the exported OfcIPCer.dat from Trusted People and
130 / 206
trigger update from Agent or Server console.
In Apex Central Policy Management, the list of Agents with Deployed status should now gradually
adding up since updated OfcIPCer.dat are now being deployed from Apex One Server to Security Agent.
131 / 206
How to collect CDT from Apex Central?
Run the CDT as Admin and select Update or Deployment Issues and General Issues.
How to collect CDT from Apex One Server?
Run the CDT as Admin and select Basic Information, Functionality, Update & Deployment, and Enterprise
Firewall.
132 / 206
How to collect CDT from Apex One Agent?
Run the CDT as Admin and select Basic Information, Connectivity Issue, Enterprise Firewall,
Update/Deployment Issue, and Vulnerability Protection.
133 / 206
How to manually debug iVP?
Follow this procedure if CDT fails.
§ Manual debug
§ Application and System Event Logs
§ msinfo32
§ Backup copy of Registry
· Collect Wireshark logs
For steps see How to use wireshar to capture filter and inspect packets?
§ iVP folder from Apex One server

(C:\Program Files (x86)\Trend Micro\Apex One\iServiceSrv\iVP).
Note: Verify if ivp_server0.log or ivp_server*.log is included on the folder.
§ To amend debug level, you can may check details below for iVP manual debug.
Debugging iVP service on Apex One Server?
How to manually debug IIS iVP Web Service?

1. Open file: C:\Program Files (x86)\Trend Micro\OfficeScan\iServiceSrv\iVP\Web\log4net.config
2. Open log4net.config using notepad and look for <level value="INFO"/>.

3. Update the value FROM: <level value="INFO" /> TO: <level value="DEBUG" />
134 / 206
4. Save the file
5. Replicate the issue
6. Collect iVPWebApp.log
file location: C:\Program Files (x86)\Trend Micro\OfficeScan\iServiceSrv\iVP\Web\iVPWebApp.log
Note: Revert the changes to disable debug
How to manually debug IIS iVP Server?
1. Open file: C:\Program Files (x86)\Trend Micro\OfficeScan\iServiceSrv\iVP\logging\logging.properties

using notepad
2. Select which feature of iVP you are trying to debug.
iVP server debug log settings: Enable debug based on the feature you want to check.
Fe ature s Log Se ttings
Command received by iVP service on Apex com.trendmicro.ivp.core.thread.CommandHandlerThread.level=ALL

One Server (general use for functions of iVP Service, alw ays turn on it for the iVP features
you w ant to troubleshoot)
Update iVP Pattern com.trendmicro.ivp.core.command.osf.OSFOnNotifyCommand.level=ALL
Deploy Security Agent Policy com.trendmicro.ivp.core.command.UpdateClientSettingsCommand.level=ALL

com.trendmicro.ivp.core.command.NotifyResultCommand.level=ALL
com.trendmicro.ivp.core.command.HeartBeatCommand.level=ALL
com.trendmicro.ivp.core.util.SecurityConfigurationUtilities.level=ALL
com.trendmicro.ivp.integration.osce.osf.w ebservice.level=ALL
IPS Logs sending com.trendmicro.ivp.core.command.osf.OSFOnLogCommand.level=ALL

com.trendmicro.ivp.integration.osce.osf.w ebservice.object.OSFWebRequest.leve
l=ALL
Move Security Agent to another Apex One com.trendmicro.ivp.core.command.osf.OSFOnCommandCommand.level=ALL
Server
135 / 206
3. Add debug log settings at the end of the file
4. Save the file

5. Replicate the issue
6. Collect the following logs:
Installation Logs: C:\Program Files (x86)\Trend Micro\OfficeScan\iServiceSrv\iVP\install.log
Debug log: C:\Program Files (x86)\Trend Micro\OfficeScan\iServiceSrv\iVP\ivp_server0.log or
ivp_server*.log.
Note: Revert the changes to disable debug
How to manually debug IIS iVP Database?
1. How to check iVP tables from Apex One Server Database?
iVP tableʼs name format ivp.xxxx
136 / 206
2. How to check iVP server activation code from ivp.activationcodes table?
The “ActivationCode” column shows the iVP server activation code
Note: The AC may not be the same as ofcserver.ini.

The ofcserver.ini only record the first AC used to activate iVP.
3. How to check VP agentʼs information from ivp.hosts table?
4. How to check IPS rulesʼ information from ivp.payloadfilter2s table?
137 / 206
The “Identifier” column shows ruleʼs ID and “Name” column shows ruleʼs name
5. How to check iVP server event from ivp.systemevents table?
• The “EventNumber” column shows iVP server event.

• The “PlainDescription” column shows details of event.
138 / 206
D. Apex One Data Loss Prevention (iDLP)
Pre-requisites when deploying Data Loss Prevention
o Make sure the Apex One Data Loss Prevention is installed in the Apex One server.
o Make sure the Apex One Data Loss Prevention license is activated.
How to install Apex One Data Loss Prevention (iDLP)?
1. Log in to Apex One web console.

2. Go to Plug-ins tab
3. Click Download
4. Click OK and wait for download to finish
5. Click Install Now
6. Click Agree to accept Apex One Data Protection License Agreement
7. Wait for Installation to finish.
139 / 206
How to activate Apex One Data Loss Prevention (iDLP)?
1. Log in to Apex One web console.

2. Go to Plug-ins tab
3. Click Manage Program
4. Enter Apex One Activation Code to activate DLP. Click Save
5. Click View the license information and status.
6. Click Update Information
140 / 206
141 / 206
Enabling and Verifying the Data Loss Prevention (iDLP) Module
How to enable iDLP via Apex Central?
1. Log in to the Apex Central web console and go to Policies > Policy Management.
2. From the Product drop-down menu select Apex One Security Agent and click Create.
3. In the Create Policy screen, type the Policy Name and Specify targets.
Apex Central provides several target selection methods that affect how a policy works.
The policy list arranges the policy targets in the following order:
Specify Targets: Use this option to select specific endpoints or managed products.
For details, see Specifying Policy Targets.

Filter by Criteria: Use this option to allocate endpoints automatically based on the filtering criteria.
For details, see Filtering by Criteria.

None (Draft only): Use this option to save the policy as a draft without choosing any targets.
4. Select Additional Service Settings from the policy page.
Enable Unauthorized Changed Prevention Service. Based on your company policy enable this feature
desktops and/or servers.
142 / 206
Enable Data Protection Service. Based on your company policy enable this feature desktops and/or
servers.
5. Click Deploy.
143 / 206
How to enable iDLP via Apex One?
1. Log in to the Apex One web console and go to Agents > Agent Management
2. Select the agent or group where you want to enable DLP.
3. Click Settings > Additional Service Settings. Make sure to enable Unauthorized Changed Prevention Service
and Data Protection Service on desktops or servers or both, depending on your preference.
4. Click Save or Apply to All agents.
How to verify if iDLP policy is deployed via Apex Central?
After deploying iDLP policy under >Policies>Policy Management a policy version will be generated. Wait for a
few minutes for the policy to be deployed to the agent/s
1. To verify the policy deployment status go to Administration > Command Tracking
How to verify if iDLP policy is deployed on the agents?
1. Right click on the agent icon and select Component Versions
2. Verify if the Policy name and Policy version is correct.
How to verify if iDLP is installed properly?
IMPORTANT: Users will be prompted to restart computer to complete iDLP driver installation.
144 / 206
1. Open Apex One Security Agent Console verify if Data Loss Prevention feature is turned on and with
green status.
2. Verify if the Trend Micro Apex One Data Protection Service and Trend Micro Unauthorized Change
Prevention Service are running.
3. Verify whether the following registry keys were created properly:
For 32 bit agent:

HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\DlpLite
For 64 bit agent:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\TrendMicro\PC-cillinNTCorp\CurrentVersion\DlpLite
145 / 206
How to block USB using Device Control?
1. Make sure the pre-requisites are met. Refer to Pre-requisites when deploying Data Loss Prevention
2. Make sure the Data Loss Prevention module is enabled. Refer to Enabling the Data Loss Prevention Module
3. In the policy, enable the Block function.
From Apex Central, you will see the option below under Device Control Settings. Put a check mark on the Block
(Data Protection) checkbox.
From Apex One, select the option Block on the drop down list.
146 / 206
Adding USB device to Approved List
First thing you need to do is to get the device information, refer to the steps below:
1. Copy C:\Windows\System32\dgagent\listDeviceInfo.exe into C:\temp

2. Plug the device into the computer
3. Run C:\temp\listDeviceInfo.exe
4. Take note of the device vendor, model, and serial ID.
Once you have the device information, you may add it on the Allowed USB Devices/Approved Devices
Via Apex Central:

o Go to Policies > Policy Management > Select the policy deployed on agent
o Go to Device Control Settings
o Click on All users (default)
o Click on Allowed USB Devices
Via Apex One:

o Go to Agents > Agent Management > Select the agent or group where you want to check the
settings
o Go to Settings > Device Control Settings
o On the USB storage devices, click on Approved Devices
147 / 206
How to Deploy Data Loss Prevention Policy?
How to deploy iDLP via Apex Central
1. Log in to the Apex Central web console and go to Policies > Policy Management
2 From the Product drop down menu select Apex One Data Loss Prevention
3. Click Create
4. Provide a Policy name. Policy and choose Targets agent/s . Enable the Data Loss Prevention and add Rule/s
5. Under Targets select target selection method
Apex Central provides several target selection methods that affect how a policy works.The policy list arranges
the policy targets in the following order:
Specify Targets: Use this option to select specific endpoints or managed products.
For details, see Specifying Policy Targets.

Filter by Criteria: Use this option to allocate endpoints automatically based on the filtering criteria.
For details, see Filtering by Criteria.

None (Draft only): Use this option to save the policy as a draft without choosing any targets.
6. Under Apex One Data Loss Prevention Settings verify if Enable Data Loss Prevention is ticked.
148 / 206
7. Click Add to start adding Rules.
8. Enable the rule and set the name. Select a policy template (e.g. all credit card number) add it to the right pane.
9. Click Channel and select the channels you require. In this sample, we choose Webmails and Windows
Clipboard.
149 / 206
10. Click Action and select the preferred action then Save.
In this sample, we selected Block and checked the Notify agents user and Record data option.
11. Click Save
12 Click Deploy.
Wait for some time to deploy. The rule must be Enabled.
To track the deployment process, see Verifying if the Data Loss Prevention Policy is Deployed.
150 / 206
151 / 206
How to deploy iDLP via Apex One?
1. Log in to the Apex One web console and go to Agents > Agent Management
2. Select the agent or group where you want to apply DLP policy.
3. Click Settings > Data Loss Prevention Settings
4. Name the Policy. Enable the Data Loss Prevention and add Rule/s
5. Enable the rule and set the name. Choose the template (e.g. all credit card number) add it to the right pane
6. Click Channel and select the channels you require.
7. Click Action and select the preferred action.
8. Click Save or Apply to All agents.
152 / 206
Troubleshooting iDLP Common Issues
Data Protection Status is showing “Not Installed”
1. Check if DLP license is activated. see Apex One Data Loss Prevention license activation.
2. Check if DLP module is enabled. see Enabling the Data Loss Prevention Module.
3. Check if DLP is installed properly. see Verifying if Data Loss Prevention was installed properly.
How to troubleshoot and further isolate the issue?
Option 1: Modify Registry keys
1. Unload the Apex One agent.
2. Remove the value of the following registry keys on the agent:
Important: Always back up the whole registry before making any modifications. Incorrect changes to
the registry can cause serious system problems.
For 32 bit agent:

HKEY_LOCAL_MACHINE\SOFTWARE\TrendMicro\PC-cillinNTCorp\CurrentVersion\DlpLite
For 64 bit agent:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\TrendMicro\PC-
cillinNTCorp\CurrentVersion\DlpLite
o "version_main"=""
o "version_3rd"=""
3. Click Update now on the agent UI.
4. If issue is not resolved, perform Option 2.
Option 2: Reinstall DLP Service/Drivers
1. Disable DLP:
· Select agent/domain where DLP needs to be disabled.
· Click Settings > DLP Settings.
· In the Data Loss Prevention Configurations page, click Policies.
· Uncheck the "Enable Data Loss Prevention" option.
· Click Save.
2. Open the Apex One server's ..\PCCSRV\ofcscan.ini file using Notepad.
3. Look for the [Global Setting] section.
4. Add the DlpSSUninst=1 parameter so that the section looks like this:
[Global Setting]
153 / 206
DlpSSUninst=1
5. Save the changes and close the file.
6. Log on to the Apex One server's web console.
7. In the agent tree, select the agent/domain where you want to uninstall the DLP service/driver.
8. Go to Settings > Additional Service Settings.
9. Under Data Protection Service, uncheck the "Enable service on the following operating systems"
checkbox.
10. Click Save. On the agent side, the agent will prompt a Restart Required window.
11. Reboot the selected agent to completely remove their DLP components.
NOTE: If same issue still occurs, collect CDT logs on the Server and Agent while replicating the issue. see
Collect CDT on the Server and Collect CDT on the Agent
154 / 206
Data Protection Status is showing “Stopped”
1. Check if DLP license is activated. Refer to Apex One Data Loss Prevention license activation
2. Check if DLP module is enabled. Refer to Enabling the Data Loss Prevention Module
3. Check if DLP is installed properly. Refer to Verifying if Data Loss Prevention was installed properly
4. Check if there is error when starting the Trend Micro Apex One Data Protection Service. If yes, proceed in
collecting dsagent crash dump file and collect CDT on agent as well. Refer to Collect dsagent crash dump file
& collect CDT on agent
5. If the DLP is corrupted, follow the steps on Data Protection Status is showing “Not Installed” Refer to Data
Protection Status is showing “Not Installed”
Unable to install Data Protection plug-in
1. Check if the Apex One server has internet connection.

2. Check if the Update Source is correct. Go to Updates > Server > Update Source
3. If using proxy to download updates, make sure to configure Administration > Proxy
How to create an offline DLP installation package?
This method is used when Apex One server has no internet connection.
a. Download the following DLP files:
https://osce14-p.activeupdate.trendmicro.com/activeupdate/server.ini
https://osce14-p.activeupdate.trendmicro.com/activeupdate/product/osce14/enu/AddonSvcDLP.zip
https://osce14-p.activeupdate.trendmicro.com/activeupdate/product/osce14/enu/DLPPatchAgent.zip
b. Create a folder on C drive. You may also create it on your preferred location. (e.g. C:\DLP)
c. Copy server.ini file to DLP folder
d. Inside iDLP folder, create product folder
e. Inside product folder, create osce14 folder
f. Inside osce14 folder, create enu folder
g. Inside enu folder, paste AddonSvcDLP.zip and DLPPatchAgent.zip
The path of files should look like this:

C:\DLP\server.ini
C:\DLP \product\osce14\enu\AddonSvcDLP.zip
C:\DLP \product\osce14\enu\DLPPatchAgent.zip
How to modify .....DLP\server.ini?
155 / 206
1. Modify the server.ini as following in order to comment out [Server] settings. You will notice that *;* has
been added.
FROM:
[Server]
Availab leServer=1
Server.1=http://osce14-p.activeupdate.trendmicro.co.jp/activeupdate/japan AltServer=[http://osce14-
p.activeupdate.trendmicro.co.jp/activeupdate/japan|https://osce14-p.activeupdate.trendmicro.com/activeupdate]
TO:
[Server]
*;*Availab leServer=1
*;*Server.1=http://osce14-p.activeupdate.trendmicro.co.jp/activeupdate/japan
*;*AltServer=http://osce14-p.activeupdate.trendmicro.co.jp/activeupdate/japan
2. Share the DLP folder over the network
3. Go to Security Tab. Set folder permission.

Permission setting: Everyone must have read & write capability.
4. Login to web console go to Updates > Server > Update Source > check Intranet.
5. Change the Update Source and set UNC path to the above sharing folder (e.g. \\HOSTNAME\DLP)
For the credentials in the Update source, please use any of the format below:
domain\username
hostname\administrator
6. Download the plug-in. Go to Plug-ins > Apex One Data Loss Prevention > Download
7. If still unable to install the plug-in, please collect CDT on the server Refer to collect CDT on serve
156 / 206
USB Exception is not Working
4. Check if the issue is happening on a specific device or on all USB devices.
5. Check in Device Manager if the device is being detected as USB device.
6. Check Allowed USB Devices/Approved Devices configuration if correct.

Via Apex Central:
§ Go to Policies > Policy Management > Select the policy deployed on agent
§ Go to Device Control Settings
§ Click on All users (default)
§ Click on Allowed USB Devices
Via Apex One:

§ Go to Agents > Agent Management > Select the agent or group where you want to
check the settings
§ Go to Settings > Device Control Settings
§ On the USB storage devices, click on Approved Devices
To get the device information, refer to the steps below:

· Copy C:\Windows\System32\dgagent\listDeviceInfo.exe into C:\temp
· Plug the device into the computer
· Run C:\temp\listDeviceInfo.exe
· Take note of the device vendor, model, and serial ID.
7. Check if the agent received the setting. Go to <Agent_Install_Folder>\dlplite\dc_in.xml (internal agent) or

dc_out.xml (external agent). Verify if the USB device is listed, refer to below sample:
157 / 206
8. If the agent did not receive the setting, please help check the communication between the server and agent.
9. If the agent received the setting but same issue occurs, please collect the Device Control Information. Refer to
Collect Device Control Information
158 / 206
USB Blocking is not Working
4. Check if the issue is happening on a specific device or on all USB devices.
5. Check in Device Manager if the device is being detected as USB device.

dc_out.xml (external agent). Verify if the permissions are correct, refer to below sample. In this sample, USB
permission is blocked.
8. If the agent received the setting but same issue occurs, please collect the Device Control Information. Refer to
159 / 206
DLP Blocking is not working in browser
4. Check if the issue is happening on a specific browser or on all browser.
5. You may go to https://dlptest.com/ for testing purposes.
6. Check if the agent received the setting. Go to <Agent_Install_Folder>\dlplite\clc_in.xml (internal agent) or
clc_out.xml (external agent). Verify if HTTP and HTTPS channel are selected.
8. If the agent received the setting but same issue occurs, please collect CDT logs on the agent. Refer to Collect
CDT on the Agent
Some Devices are being blocked by DLP (e.g. Scanner)
3. Check if DLP is installed properly. Verifying if Data Loss Prevention was installed properly
4. Check in Device Manager if the scanner is being detected as USB or Printer or other Device Type.
dc_out.xml (external agent). Verify if the permissions are correct.
7. If the agent received the setting but same issue occurs, please collect the Device Control Information. Refer
to Collect Device Control Information
160 / 206
Collect CDT on the Server
1. Download the latest CDT on this link.
2. Run the CDT as Admin and select Basic Information.
3. Replicate the issue.
4. Collect todayʼs log.
Collect CDT on the Agent
1. Download the latest CDT on this link.

2. Run the CDT as Admin and select Basic Information and Data Loss Prevention.

4. Collect todayʼs log.
1. Copy C:\Windows\System32\dgagent\listDeviceInfo.exe into C:\temp

2. Copy this logger.cfg into C:\
161 / 206
3. Download WinAudit from : http://www.parmavex.co.uk/winaudit.html
4. Turn on CDT tool and select [Basic Information & Data Loss Prevention]. Refer to Collect CDT on
the Agent
5. Plug the device into the computer
6. Run C:\temp\listDeviceInfo.exe
7. Run winaudit.exe
8. Wait couple minutes until the auditing is over and STOP icon grey out like the follows:
9. Select File > Save to save the report.

10. Unplug the device
11. Turn off CDT tool
12. Collect the report and debug logs:
§ C:\temp\devInfo.(hostname)_(3 digits).log
§ C:\temp\dlpDeviceReport.htm
§ Winaudit report
§ CDT logs
Collect dsagent Crash Dump File
If DLP service process dsagent.exe crashes, its dump will be automatically created in the following location: %
WINDIR%\dsacrash.dmp
How to Isolate if issue is caused by DLP?

2. Isolate DLP driver first. Rename the file:
3. %WINDIR%\System32\drivers\sakfile.sys to %WINDIR%\System32\drivers\sakfile.sys.bk
4. Reboot and check if the issue is gone.
5. If the issue persists, isolate DLP service. Rename the file:
6. %WINDIR%\System32\dgagent\dsagent.exe to %WINDIR%\System32\dgagent\dsagent.exe.bk
7. Reload Apex One agent.
8. Report the two isolation results.
How to Collect Full HTTP Dump?
162 / 206
2. Edit %windir%\system32\dgagent\dsa.pro
3. Add the lines below:
log_raw_data=true
keep_tmp_file=true
dump_all=true
dump_dir=dumpdir
4. Close all browser processes.

5. Restart DLP agent by reloading Apex One agent.
6. Browse some website and reproduce the issue. Check that HTTP data can be recorded in %windir%
\system32\dgagent\dumpdir
7. Collect the full folder after the issue is reproduced.
E. Apex One (Mac)
Apex One (Mac) Server Requirements
For full details, refer to this article: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-(mac)-

2019-server-online-help/installing-the-serve_001/system_requirements.aspx
Apex One (Mac) Server Installation and Activation
1. Apex One (Mac) server can be installed from Apex One or OfficeScan Plug-ins tab.
Refer for full details here: https://docs.trendmicro.com/en-us/enterprise/trend-micro-apex-one-(mac)-2019-
server-online-help/installing-the-serve_001/install_server.aspx
2. Apex One (Mac) SaaS Version-- If you are using Apex One full license key—it will automatically activate
Apex One (Mac). If you are using legacy license for Apex One (Mac) separately, license needs to be added
and activated on Apex Central first.
Installation Verification
1. Installation Logs
· c:\TMSM_PreInstall.log
· c:\TMSM_Insatll.log
· c:\TMSM_DBInstall.log
· c:\TMSM_serverInfoTool.log
2. Apex One (Mac) Services
Verify that the following services display on the Microsoft Management Console
o ActiveMQ for Apex One (Mac)
o Apex One (Mac) Main Service
163 / 206
3. Apex One (Mac) Process
Verify of process is running in Windows Task Manager:
o TMSMMainService.exe
4. Apex One (Mac) Registry Key location
Verify that the following registry key exists in Registry

Editor: HKEY_LOCAL_MACHINE\Software\TrendMicro\
OfficeScan\service\AoS\OSCE_ADDON_TMSM
5. Apex One (Mac) Server Installation Folder
If you accept the default settings during Apex One server installation, you will find the server installation
folder at any of the following locations:
· C:\Program Files\Trend Micro\OfficeScan\Addon\TMSM
· C:\Program Files\Trend Micro\Apex One\Addon\TMSM
· C:\Program Files (x86)\Trend Micro\OfficeScan\Addon\TMSM
· C:\Program Files (x86)\Trend Micro\Apex One\Addon\TMSM
6. IIS App Pool
164 / 206
165 / 206
Apex One (Mac) agent Installation
You may get installer file for Apex One (Mac) Security Agent either from Apex Central or Apex One (Mac) Plugin.
1. Log on to the Trend Micro Apex Central console.

2. Go to Administration > Security Agent Download.
3. Select the "Mac OS" operating system.
4. Click Download.
Expected Result: After step 4, the tmsminstall.zip file package downloads successfully.
Procedure:
1. On the target endpoint, unzip the tmsminstall.zip file package.
2. Go to the unzip folder and double click the tmsminstall.pkg file to install the Apex One (Mac) Security
Agent.
Expected Result: The Apex One (Mac) Security Agent successfully installs on the endpoint.
166 / 206
167 / 206
The results display as shown in the following figure.
1. Verify that the Security Agent tray icon is on the menu bar.
2. Click the Security Agent tray icon and verify that the agent status is "Protection Enabled".
3. Verify that the TrendMicro folder is available in /Library/Application Support/ directory.
168 / 206
4. Check server connection status. The icon on the Security Agent console from the system tray indicate the
parent server connection status.
169 / 206
Deploying Apex One (Mac) Policy from Apex Central
Overview: For this example, we try to deploy Apex One (Mac) policy with Endpoint Sensor (iES) enabled:
1. Login to the Apex Central Web Console.

2. Go to Policies > Policy Management.
3. Select Apex One (Mac) from Product drop-down menu and click Create.
4. In the Create Policy screen, type in the name of the policy as Deploy Apex One for Mac.
5. Select Specify Target(s) and do the following:
a. In the Search tab, select Operating Systems checkbox and type Windows 10. Click Search.
6. In the search result, select the Mac endpoint and click the Add Selected Targets button to add.
Click OK to go back to the Create Policy screen
7. Most of Apex One features is enabled by default. For this exercise, enabled Endpoint Sensor feature.
Scroll to the bottom and expand the Endpoint Sensor tab. Click the Enable Endpoint Sensor
checkbox to enable this feature.
8. Click Deploy to start deploying the policy to the Apex One for Mac Security Agent.
9. Go to Administration > Command Tracking > Look for recent Apply Policy under Command column >
Click the Successful results to verify if itʼs already deployed on Agentʼs Apex One Server.
170 / 206
10. Go to Policies > Policy Management and verify that Apex One (Mac) Policy is now on Deployed
status.
11. When deployment is finished, connect to the Mac endpoint, open the Apex One Security Agent Console
via system tray icon and verify if Endpoint Sensor is now enabled with its running Trend Micro Security for
Mac agent).
171 / 206
Apex One (Mac) Common Issues
In this section, we will discuss commonly encountered issues in Apex One (Mac) including console blank page,
plugin errors, and services stopping.
Issue 1: How to troubleshoot "Blank page when accessing console"?
Description: Upon installing a later build of Apex One for Mac, the user is unable to access their plugin and
instead displays with a blank screen when clicking on "Manage Program"
Troubleshooting Steps:
A. From sample ofcdebug.log file, you may find this error:
B. In sample debug.log, check the requested certificate name:
C. Check the certificates on the Apex One server. To do this:

Run the command to check if there is any none self-signed certificate in the root store.
In Powershell interface, copy and run:
Get-ChildItem Cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject}
D. Collect the information of client certificates:
1. Open MMC. Run "mmc" in "Start > Run"

2. Add certificates snap-in by click "File > Add/Remove Snap-in"
3. Select "Certificates" in left "Available snap-ins:", and then click "Add >" button to add it into "Selected
snap-ins"
4. Select "Computer account" in "Certificates snap=in" and click "Next>" to continue.
5. Select "Local computer" in "Select Computer" windows, and click Finish to reflect the operation result.
6. Make sure "Certificates > Trusted Root > Certificates" having a valid root certificate
172 / 206
"OfcOSFWebAppRootCA"
7. Make sure "Certificates > Trusted People > Certificates" having a valid item "OfcOSFWebApp"
8. Double click on the certificate "OfcOSFWebApp" click on "Certification Path" tab check the "Certificate
status" is OK or not.
E. Make sure the server's IIS component installation.
1. Open "Server Manager" and select "Local Server" at left pane

2. Click "Manage > Add Roldes and Features" at top-right side of "Server Manager"
3. Click "Next>"
4. Select "Role-based or feature-based installation" and click "Next>"
5. Leave as default settings and click "Next>"
6. In the Roles selection list, expanding "Web Server (IIS) > Web Server > Security" and make sure
173 / 206
"Client Certificate Mapping Authentication" is selected, if not please add this feature for IIS role.
If above item 3 performed adding "Client Certificate Mapping Authentication", please test reboot the
computer and test if the console can be connected or not.
However, if issue still persists, please go to this site and follow the answer provided:
https://stackoverflow.com/questions/26247462/http-error-403-16-client-certificate-trust-
issue/35001970
F. Restart all the TMSM related services, by running following commands in command prompt with admin
permission
. net stop ofcaosmgr

. net stop tmsmmainservice
. net stop activemq4tmsm
. net start activemq4tmsm
. net start tmsmmainservice
. net start ofcaosmgr
G. Please try to open the Apex One (Mac) or Security for Mac console to confirm if the console can be opened or
not.
1. From this point, please check if you're able to access the console, however if the issue persists,
please check debug log again if the same error code (403.16) is there or if it has changed.
2. If it has changed to error 404, please check if the port bindings by Apex One and TMSM (Apex One
Mac) are set accordingly (by default set at 4343).
3. If same issue persists, proceed to Collect the required logs.
174 / 206
175 / 206
1.
Logs Collection
Proceed to Collect debug logs and submit to Technical Support.

1. CDT log
Download the latest CDT on this link.
Run the CDT as Admin and select Basic Information, TMSM (Apex One for Mac)
Replicate the issue.
Collect todayʼs log.
2. debug.log of TMSM
3. Take screenshots as well of currently installed certificates in customer environment
4. IIS bindings
176 / 206
Issue 2: How to troubleshoot "Unable to install the Apex One (Mac) Server. The
product's database cannot be installed."?
Description: The error below is being encountered when trying to install the Apex One (Mac) plug-in.
Possible Cause:
The SQL account that Apex One/OfficeScan uses contains special characters in the password.
Sample logs:
C:\TMSM_DBTool.log
C:\TMSM_PreInstall.log
Error being stated above is related to a connection string error being used by SQL Server to connect with the
database. The password being used to connect with the database has some special characters (Ex. [] {}() , ; ? * !
@.) that is incompatible with the connection string.
To solve this issue:
1. Change password of the account being used to connect with the SQL Server and it should not contain special
characters. Make sure that the password being used does not contain any special characters. (Ex. [] {}() , ; ?' * !"
@.).
Reference Article: https://blogs.msdn.microsoft.com/spike/2009/10/30/format-of-the-initialization-string-does-not-

conform-to-specification-starting-at-index/
177 / 206
2. To verify if the issue is resolved:
Apex One (Mac) plug-in should be installed successfully.
Log Collection
If issue persists, please collect the following logs for further analysis:
· C:\TMSM_PreInstall.log
· C:\TMSM_Install.log
· C:\TMSM_DbInstall.log
· C:\TMSM_serverInfoTool.log
178 / 206
Plugin will not start after installing (upgrade) Apex One patch
Issue 3: How to troubleshoot "Plugin will not start after installing (upgrade) Apex One
patch"?
Description: The ActiveMQ for Apex One (Mac) was unable to start due to corrupted/missing files caused by
the Apex One patch when doing the upgrade/backup
Apex One (Mac) Main Service will not start (dependent on ActiveMQ for Apex One (Mac))
§ Customer might experience the issue when:

o Apex One patch was installed (upgrade)
o Apex One build version is lower than apex-one-2019-win-en-criticalpatch-b2012.exe
Solution:
§ This issue has been resolved on apex-one-2019-win-en-criticalpatch-b2012.exe
Troubleshooting steps:
1. Verify if some files are missing or some files should not be on that directory
For example: There should be no \Trend Micro\Apex One\BundledJava\BundledJava folder
The \Trend Micro\Apex One\BundledJava should only contain
2. Restore from BundledJava_backup_xxxxx
3. Restart Apex One Mac services, (run restart_TMSM.bat)

BundledJava_corrupted (missing files)
179 / 206
BundledJava_backup_xxxxx, (Correct files)
Renamed BundledJava (corrupted) and restored from backup
180 / 206
Log Collection
If issue persists, please collect the following required logs:
1. TMSM logs (<Apex One>\Addon\TMSM\apache-activemq\data)
2. activemq.log
3. wrapper.log
The logs would show that the ActiveMQ for Apex One (Mac) last running state and correlate with the timestamp
when the patch was installed Apex One (hotfix_history)
181 / 206
Issue 4: The Apex One (Mac) agent is unable to start the protection on a Mac upgraded
to macOS Catalina v10.15 or higher.
Compatibility
Apex One Mac supports MacOS Catalina 10.15.4 on the following agent version as of writing:
Apex One On-premise: 3.5.2100 or higher
Apex One SaaS: 3.5.3310 or higher
Issues that might be caused by MacOS Catalina build upgrade are:

o Unable to Start Protection
- after applying all pre-requisites (kext, Full Disk Access, reboot)
o Apex One Mac console not showing
- after performing a "Reboot"
o Apex One Mac console keeps on restarting
- approximately restarting every 30+ seconds, conflict with other modules
o Freezing login screen (sleep)
- approximately stuck by 15 seconds
o Unable to collect debug logs
- Unable to generate the TMSMLog.tar after number of hours (typically it should take around 15
- 30 minutes).
Starting from MacOS Catalina 10.15, Apple implements new driver and security enhancement. MacOS devices
that already upgraded to MacOS Catalina with Agent version lower than (3.5.2089) needs to Uninstall and re-
install the agent.
For full details, refer to this KB article: https://success.trendmicro.com/solution/000149499-Trend-Micro-Apex-
One-Mac-Support-for-macOS-1015-Catalina
How to effectively submit this issue to Technical Support:
1. Indicating the right behavior (Category) would be beneficial on the troubleshooting steps or next action plan
2. If possible (please), indicated the performance category on the case title or initial summary
3. Most of the performance issues have intermittent and indistinguishable behavior, please make some time to
describe (technical observation) in the case description for the overview of the case
4. Indicate the steps that already been taken on the case description.
Recommended Action Plan:
1. Upgrade the Apex One (Mac) server to build 3.5.2141 or higher.

2. Uninstall (tmuninstall.zip) and reinstall (tmsminstall.zip) Apex One Mac agent. For more details, please refer
to this document.
3. If the issue still persist, collect Agent CDT logs
182 / 206
iProducts System Requirements
Apex One Application Control System Requirements
Here are the pre-requisites for Apex One Application Control:
Ite m Re quire m e nt
System Requirements Same as Apex One Server and Security Agent
License · Included in the Apex One Full Feature for Window s and Mac license
· An existing Trend Micro Endpoint Application Control License (activated in Apex Central)
Apex Central registration Required for licensing and Security Agent policy deployment
Compatibility w ith Tend · For server: The Apex One server w ith Application Control can exist on the same server w ith Trend Micro
Micro Endpoint Application
Endpoint Application Control Server (not recommended)
Control
Note : Trend Micro Endpoint Application Control server settings are not compatible w ith Apex One Application
Control Feature. You must manually configure all policies using the Apex Central w eb console
· For agent: Once you deploy an Apex One Application Control policy to the Apex One Security Agent, the
Security Agent w ill automatically uninstalls any existing Trend Micro Endpoint Application Control agent before
applying the Apex One Application Control settings.
Server The Apex One Setup program installs the Application Control feature automatically during normal Apex One
server installation.
After verifying that the Activation Code includes Application Control, Apex One starts the Tre nd M icro
Application Control Se rvice on the Apex One server computer.
Apex One Endpoint Sensor Requirements
Here are the pre-requisites for Apex One Endpoint Sensor:
System Requirements For server: Same operating system requirements as Apex One Server
SQL Server requirements differ.
For agent: Same system requirements as the Security Agent
The feature are only officially supported on the follow ing platforms:
o Window s 7 SP1
o Window 8.1
o Window 10
License · Apex One Endpoint Sensor license (activated in Apex Central)

· An existing Trend Micro Endpoint Sensor license (activated in Apex Central)
Compatibility w ith Tend Micro · For server: The Apex One server w ith Apex One Endpoint Sensor feature on the same server w ith the
Endpoint Application Control
standalone Trend Micro Endpoint Sensor server (not recommended)
Note : Standalone Trend Micro Endpoint Sensor server settings are not compatible w ith Apex One Endpoint
Feature. You must manually configure all policies using the Apex Central w eb console
· For agent: Once you deploy an Apex One Endpoint Sensor policy to the Apex One Security Agent, the
Security Agent w ill automatically uninstalls any existing Trend Micro Endpoint Sensor agent before applying
183 / 206
the Apex One Endpoint Sensor settings.
Redis service The Apex One server computer cannot have an existing Redis service installed. You must uninstall any existing
Redis service and allow the Setup program to install a new service.
SQL Server version · SQL Server 2017
· SQL Server 2016 SP1
Note : This feature does not support SQL Server Express versions
Database configuration Full-Te xt and Se m antic Extractions for Se arch should be enabled
Apex One Vulnerability Protection System Requirements
Here are the pre-requisites for Apex One Vulnerability Protection:
System Requirements Same as Apex One Server and Security Agent
License · Included in the Apex One Full Feature for Window s and Mac license
· An existing Trend Micro Vulnerability Protection license (activated in Apex Central)
Compatibility w ith Tend Micro · For server: The Apex One server w ith Apex One Vulnerability Protection feature on the same server w ith
Endpoint Application Control
the standalone Trend Micro Vulnerability Protection (not recommended)
Note : Standalone Trend Micro Endpoint Sensor server settings are not compatible w ith Apex One Endpoint
Feature. You must manually configure all policies using the Apex Central w eb console
· For agent: Once you deploy an Apex One Vulnerability Protection policy to the Apex One Security Agent, the
Security Agent w ill automatically uninstalls any existing Trend Micro Vulnerability Protection agent before
applying the Apex One Vulnerability Protection settings.
Compatibility w ith other Trend The follow ing Trend Micro products are not compatible w ith the Apex One Vulnerability Protection feature:
Micro products · Deep Security Agent
· Intrusion Defense Firew all agent
You cannot activate the Apex One Vulnerability Protection feature on Security Agents installed on endpoints
w ith an incompatible agent program installed. You must uninstall the conflicting program before activating the
Apex One Vulnerability Protection feature.
184 / 206
How to enable debug?
185 / 206
How to debug Apex One Server?
1. Debugging the server using the web UI.
1. Hover the mouse over the “T” of Trend Micro on the banner after logging in.
2. Click the letter T and debugging window appears.

3. Enable the debug mode.
4. Select Error for the Debug Level
5. Click on Save. You can now replicate the issue.
6. After reproducing the case, click again on the “T” of Trend Micro. Before disabling the debug log, take note of the
location of the log file. Then, disable the debug mode.
2. Manually debugging the server.
1. Copy the contents of the \Program Files (x86)\Trend Micro\OfficeScan\PCCSRV\Private\LogServer to the

root of C:
2. Edit the ofcdebug.ini file now located in the root of C:
3. Change DebugLog= C:\Program Files (x86)\Trend Micro\OfficeScan\PCCSRV\Log\ofcdebug.log to

"DebugLog=.\ofcdebug.log".
4. Change debugLevel_new=I to "debugLevel_new=D".
5. Change ForceStopOtherLogserver=0 to "ForceStopOtherLogserver=1".
·
If larger logs are desired, you can edit the debugSplitSize line. Default is 10 MB before splitting and zipping
the old file.
· By default, DebugMaxSplit=500, this limits the total number of split logs to 500 files.
6. Save the file.
7. Run LogServer.exe as Admin.
· You will see the ofcdebug.log file created in the root of C:

· When the file rolls-over, it will compress the old file with a .7z and start a new ofcdebug.log.
8. Reproduce the issue.
9. Close the LogServer.exe window to stop the debug log.
10. Delete the files copied from \Program Files (x86)\Trend Micro\OfficeScan\PCCSRV\Private\LogServer.
186 / 206
How to debug Widget Framework?
1. Go to the C:\Program Files\Trend

Micro\OfficeScan\PCCSRV\Web_OSCE\Web_console\HTML\widget\repository\widgetPool\product\ directory
in the OfficeScan Server.
2. Open the config.php file and change the value of wfconf_debug lines as shown below:
$GLOBALS['wfconf_debug'] = true;
$GLOBALS['wfconf_client_debug_level'] = “DEBUG”;
3. Save and close the file.

Make sure the other debug tools mentioned in this article are running simultaneously. Replicate the issue.
5. Collect the following files from the ..\Trend

Micro\OfficeScan\PCCSRV\Web_OSCE\Web_console\HTML\widget\repository\log\ directory:
· diagnostic.log
· client_diagnostic.log
Important: Disable debug mode before collecting the widget debug log.
To disable the debug log, open the config.php file and set the values below according to the following:
o Set $GLOBALS['wfconf_debug'] = "null";
o $GLOBALS['wfconf_client_debug_level'] = "OFF";
How to debug CM Agent Issues?
1. On the Apex One server, open the \Apex One\PCCSRV\CmAgent folder.
2. Open the product.ini file in a text editor.
3. Add the following lines at the end of the file:

[debug]
debugmode=3
debuglevel=3
187 / 206
debugtype=0
debugsize=10000
debuglog=c:\CMAgent_debug.log
4. Save and close the file.
5. Replicate the issue you encountered.
6. Send the C:\CMAgent_debug.log to Trend Micro Technical Support.
To disable debug mode, open the product.ini file then remove the lines you added in Step 3.
How to manually debug the agent?
1. Copy the contents of the \Program Files (x86)\Trend Micro\OfficeScan Client\Temp\LogServer\ folder
(excluding the Log folder) to the root of C:
2. Edit the ofcdebug.ini file now located in the root of C:
Change DebugLog=.\Log\ofcdebug.log to "DebugLog=.\ofcdebug.log".
Change debugLevel_new=E to "debugLevel_new=D".
Change ForceStopOtherLogserver=0 to "ForceStopOtherLogserver=1".
· If larger logs are desired, you can edit the debugSplitSize line. Default is 10 MB before splitting and
zipping the old file.
· By default DebugMaxSplit=100, this limits the total number of split logs to 100 files.
6. Save the file.
7. Run LogServer.exe as Admin.
· You will see the ofcdebug.log file created in the root of C:.
· When the file rolls-over, it will compress the old file with a .7z and start a new ofcdebug.log.
9. Close the LogServer.exe window to stop the debug log.
10. Delete the files copied from \Program Files (x86)\Trend Micro\OfficeScan Client\Temp\LogServer\.
How to debug Scan Engine?
1. Open the Registry Editor.

Note: Always back up the whole registry before making any modifications. Incorrect changes to the
registry can cause serious system problems.
188 / 206
2. Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TMFilter\Parameters.
3. Change the value of the "DebugLogFlags" key to "00003eff".
5. Once done, disable the debug mode by restoring the "DebugLogFlags" key to "0".
6. Locate the TMFilter.log file in your %SystemRoot% folder and send it to Trend Micro Technical Support.
How to enable Apex One Diagnostic Log?
1. Backup the file : ..\PCCSRV\Web_OSCE\Web_console\HTML\widget\repository\widgetPool\product\config.php
2. Open the the ‘config.phpʼ in notepad and change the value of debug to ‘Trueʼ, then click save.
See below example:
$GLOBALS['wfconf_debug'] = true;
3. Restart the Apex One Master service and the log will be generated on below location :
.. \PCCSRV\Web_OSCE\Web_console\HTML\widget\repository\log\diagnostic.log
Note : To disable the diagnostic log debugging, revert back the original config.php or change the value of the
debug back to ‘nullʼ again.
189 / 206
How to debug SPS Server using CLI?
This method is useful when customer cannot collect CDT or login using SPS Web Console.
How to generate CDT via SSH?
1. Connect to SPS server using SSH. In this example we will use putty.
2. Login as root account
3.Perform the following command
“/usr/tmcss/bin/CDT_ICRC_Linux.sh”
190 / 206
4. Collect CDT File. The location of the file will be provided after the command completed to run.
In this example file is located in /var/tmcss/cdt/Info_20171110_031204.tar.gz
How to collect CDT using WinSCP?
1. Download and install WinSCP on a windows machine to collect the CDT data
2. Run WinSCP application and input the credentials needed for SPS server then click Login
3. You can see that we are now successfully connected to SPS server and we can see all the directories available
on the SPS server
191 / 206
4. Go to the directori where CDT data is saved.
/var/tmcss/cdt
192 / 206
5. Select the CDT data and click download
6. Browse the location where you want to copy the CDT data on your desktop.
7. It will start copying the data and after the download is complete. You can now see the CDT data on your Desktop
where you saved it.
193 / 206
8. You can now zip this file and send the data to Trend Micro Technical Support or you can now also try analyzing
the data.
194 / 206
Indexes
How to collect logs using Windows Performance Recorder (WPR)?
Windows Performance Recorder (WPR) is a tool that extends Event Tracing for Windows (ETW) and provides
detailed recordings of system and application behavior and resource usage. You can use WPR together with Windows
Performance Analyzer (WPA) to investigate particular areas of performance and to gain an overall understanding of
resource consumption. WPR and WPA enable development and IT professionals to proactively identify and resolve
performance issues. WPR requires Windows 8 or later version operating system.
How to Use?
1. Download and Install Windows Performance Recorder from Windows MSDN.
o Windows 8 and later => Use Win10 WPT

o Windows 7/2008R2 => Windows 8 WPT
o Windows Vista/2008 => User WPT 4.x, refer to WPT 4.x usage
2.Once installed, open cmd.exe with elevated privilege and launch WPRUI.exe to open Windows Performance
Recorder.
3.Select the following:

Logging Mode: File
Resource Analysis:
· CPU Usage
· Disk I/O Activity
· File I/O Activity
· Registry I/O Activity
195 / 206
NOTE: If this performance issue is about memory usage, you could also select the following:
· Heap Usage
· Pool Usage
4.Click the Start button to begin recording.
a. Select what resource you want to monitor.

Note: Select what is applicable based on the issue you are troubleshooting
b. Click “start” to run the tool

Note: Ensure that the issue is happening when collecting information. Keep the tool running for about 30-
60sec or up until the replication is done.
196 / 206
6. Save the .etl file when the high CPU issue occurs.
7. Compress the .etl file with zip format.
How to collect Windows Dump Files?
For BSOD or system hang issue, we need a full dump at least.
How to collect FULL memory dump ?
1. Download Microsoft free tool "DumpConfigurator.hta".

See link to download
2. Unzip the WinPlatTools.zip ,go to \WinPlatTools\sourceCode---> you will see DumpConfigurator.hta.
3. Run it with Administrator privilege.
197 / 206
4. All the settings can be edited and saved by clicking Save Settings. The system will have to be rebooted for the
settings to take effect.
5 Submit the C:\Windows\MEMORY.DMP to Trend Micro Support Team.
How to collect ProcDump logs?
We can use ADplus or ProcDump to collect the dump for the crashed process.
1. Download latest version of ProcDump here.
2. Extract the tool (procdump.exe) on a temporary folder like desktop on the target computer.
3. Open command prompt (run as the Administrator) and change the directory to where the procdump.exe was
extracted.
198 / 206
4. Run the following command: procdump -ma someprocess.exe -s 20 -p "\Processor(_Total)\% Processor Time" 80
5. Click the Agree button when the EULA dialog box shows up.
The switches are defined as follows:

-ma someprocess.exe - means generate full dump on ntrtscan.exe process
-s 20 - means 20 seconds before creating dump
-p "\Processor(_Total)\% Processor Time" - 80 means threshold of 80% CPU
When the above command is executed, ProcDump monitors someprocess.exe and only when it reaches 80%
CPU Utilization for 20 seconds that the tool starts creating the full memory dump. The tool terminates itself after
creating the process dump file found in the same file path as the procdump.exe.
How to collect ProcMon logs?
Process Monitor can also be useful for performance issues, although care needs to be taken as Process Monitor can
also have a performance impact on the machine.
1. Download the Process Monitor Utility from Microsoft and place it in the machine.
2. Extract the files.
3. Run ProcMon.exe and accept the EULA.

It will automatically begin collecting data.
4. Reproduce the performance issue on the machine.
5. After the issue has been reproduced, stop the collection by clicking the magnifying glass icon in
Process Monitor so that there is a red line through it.
199 / 206
6. Choose File > Save and then All events and Native Process Monitor Format (PML).
7. Zip the PML file, then upload it for review.
How to collect UI Network Traffic Log?
1. Open the Apex One web console on the internet explorer.

2. Press F12.
3. Go to Network Tab and make sure that the debug is recording :
200 / 206
4. Access the Apex One web console to replicate the issue.
5. Save the log in har file:
Note: To disable the recording, just close the F12 Developer Tools.
How to replicate issue for Offline agents?

Steps on how to replicate issue for offline agents:
1. Enable CDT/Manual debug on Apex One server

2. Enable CDT/Manual debug on Apex One agent
3. Start Wireshark on Apex One agent
4. Start Wireshark on Apex One server
5. Unload/Reload Apex One agent. Provide timestamps.
6. Wait for 10 minutes
7. Collect logs and submit to Trend Micro Support Team
There are instances that the machines cannot handle the load of running CDT and Wireshark at the same
time.
201 / 206
You can follow the steps below:

5. Collect CDT logs
6. Start Wireshark on Apex One agent
7. Start Wireshark on Apex One server
8 Unload/Reload Apex One agent. Provide timestamps.
10. Collect Wireshark logs and CDT logs and submit to Trend Micro Support Team
How to replicate issue for Outdated agents?
Steps for replicating issue for outdated agents:

3. Enable CDT/Manual debug on Update Agent if agent get updates from Update Agent
5. Wait for 10 minutes (Note: wait for all Apex One agent services and drivers to be completely loaded, wait for
Apex One server to notify agent to perform program upgrade)
6. Collect logs and submit to Trend Micro Support Team
How to check if Apex One Server is using 3rd-party certificate?
1. Access Apex One Server

2. Look for IIS logs and open the latest IIS logs
3. Look for this keyword: SystemCall and check if the HTTP result is 403.16
4. Open certlm.msc and check the following certificates
a. Personal > Certificate
202 / 206
b. Trusted People > Certificate
c. OfcOSF > Certificate
5. Open inetmgr and check the certificate being used in Apex One Site Bindings
a. Click on Sites > OfficeScan.
b. Click on Bindings.
203 / 206
c. Click on https > Edit.
d. Check if the certificate being is used the default certificate or a 3rd-party issued certificate.
6. If customer is using 3rd-party certificate, follow the steps on this KB:

https://success.trendmicro.com/solution/1122205-configuring-apex-one-to-use-a-certificate-signed-by-
corporate-certificate-authority
7. If customer is using the default certificate and you still see HTTP 403.16, add the following registry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\Schannel]
Name: ClientAuthTrustMode
Type: REG_DWORD
Value: 2
204 / 206
8. Try again to deploy the license.
205 / 206
Feedback
For comments and suggestions you can answer a quick survey below.
· Comments and Suggestions
Useful links
Description URL
Knowledge Base https://success.trendmicro.com/product-support/apex-one
Online documents https://docs.trendmicro.com/en-us/enterprise/apex-one.aspx
· Installation and Upgrade Guide
· Administration Guide
· System Requirements
· Online Help
CDT Tool How to use CDT Tool?

Download link
206 / 206

Trend Micro Apex One Troubleshooting Guidev1.5

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Trend Micro Apex One Troubleshooting Guidev1.5

Uploaded by

Copyright:

Available Formats

TREND MICRO™Apex One

TREND MICRO™Apex One AMEA Partner Case Submission Handbook

Copyright © April 2020 by Trend Micro Inc. All Rights Reserved.

AMEA Partner Case Submission Handbook

Figure below shows an Apex One Sample setup.

1. Pre-deployment will discuss the following:

Server and client have met minimum system requirements.

Verify Apex One System Check the System Requirements:

What URLs are used by Apex One?

Here are the URLs used by Apex One:

Collect Basic Information

1. Provide a short description of the problem.

1. Product ve rs ion and build

Using Apex One w eb console, go to He lp > About

2. Product re gis try inform ation

Registry export of HKLM \SOFTWARE\WOW6432Node\TrendM icro\OfficeSca n\service\Informa tion

3. Bas ic Sys te m Inform ation

o Run e ve ntvw r and then expand Window s Logs :

o Using Apex One Console, go to He lp > About

o Database Server Type and information (e.g. MSDE/SQLExpress/SQL):

Note: The Apex One server uses SQL Server b y default.

o Service Pack installed

Basic Agent Information

· Collect ofcscan.ini in the product agent directory

SCENARIO USE CASE AFFECTED ENDPOINTS AFFECTED POLICIES DEPLOY TIMING

All endpoints without policy and match the new

Endpoints in this policy

Reorder policies All endpoints as long as

Apex Central default mechanism On premise: Every 24 hours

SaaS: Every 10 minutes

Time needed for policy deployment status to reflect on Apex Central

· Wait for next policy enforcement

Scenario 1: Default iProduct policy settings

How to optimize Apex One agent?

1. Install the latest patch for Apex One

c. Trusted Program List

b. Increased the LRU Cache 2000 (default)

6. Enable the deferred scan.

7. Make sure that the debug module has been disabled.

DLP: (remove the keys)

Where to start isolating the issue?

Learning Settings -> Unclick "File" -> Save

Learning Settings -> Unclick "Process" -> Save

4. Behavior Monitor Service (AEGIS) Proce dure :

Settings -> untick "Enab le Event Monitoring" -> Save

9. Ransomw are Protection

B. Softw are Re s tricte d Policy

NOTE: For isolating on Apex One as a Service, see KB 1123591

1. Server Installation / Server Upgrade Issues

1. Fresh Server Installation Issue

Fresh installation of Server

See System Requirements

Upgrade from OfficeScan to Apex One Server

A. Quick migration guide for Trend Micro Apex One

The following topics are discussed on this KB

· Pre-Upgrade Checklist for Apex One Server

1. Access Officescan Server web console.

6. Check the exported file and filter the OS_MAJOR, OS_MINOR

If encountered an issue when installing a Critical Patch/Hotfix, check tmpatch.log on C:\

1. Look for this keyword: failed.

[2019-09-25:09:58:41][perfLWCSPerfMonMgr.dll : C:\Program Files (x86)\Trend

[2019-09-25:09:58:41]Create new File Failed,last error:[32]

2. In Apex One Server, perform the following action.

o Unload Apex One Agent