Installing and Configuring the Microsoft Debug Diag Tool

The Microsoft Debug Diag Tool is used to capture dump files for a process that is hanging or crashing. In
a situation where a process or processes are crashing, it can be used to monitor a list of specified list of
processes, and generate a dump file if any of those processes encounter an exception and crash. In a
situation where a process or processes are hanging, it can be used to manually generate a dump file for
those processes.

To download the Tool, use the following link:

** Note: If you are running a 64bit OS, then you will ned the 64bit version, and if you are running a
32bit OS then you need the 32bit version**

http://www.microsoft.com/en-us/download/details.aspx?id=40336

Upon accessing the link above, click on the “Download” button – you will then be able to choose the
32bit or 64bit version of the Tool to download.

Installing the Tool:

1. Copy the file over to the server

2. Run the installer, and use the settings as shown here:
 3. When the installer Completes, Click the “Finish” button and the installer will then close

Configure the Tool to Monitor a Process or Processes

In a situation where a process or processes crash, configure the Debug Diag Tool to monitor that process
or processes. When the problem occurs, the Tool will automatically generate a dump file based on the
specifications you configure.

Here are a list of the most common CA Service Desk Manager process names:
animator_nxd.exe domsrvr.exe krc_daemon.exe pdm_proctor_nxd.exe
arcpur_srvr.exe ech_nxd.exe kt_daemon.exe pdm_text_nxd.exe
bopauth_nxd.exe ehm_nxd.exe ldap_virtdb.exe pdm_tomcat_nxd.exe
boplgin.exe ehwriter.exe lexagent_nxd.exe pdm_ver_nxd.exe
bpebr_nxd.exe filter_nxd.exe mdb_registration_nxd.exe rep_daemon.exe
bpeid_nxd.exe javaw.exe pcrpt_nxd.exe spel_srvr.exe
bpnotify_nxd.exe keit_daemon.exe pdm_d_mgr.exe sql_agent_nxd.exe
bpvirtdb_srvr.exe key_prov_nxd.exe pdm_export_mgr.exe sslump_nxd.exe
bu_daemon.exe kpi_qry_daemon.exe pdm_mail_nxd.exe ttv_nxd.exe
dbmonitor_nxd.exe kpi_sys_daemon.exe pdm_maileater_nxd.exe webengine.exe
For our purposes here, we will use the example of the domsrvr.exe process crashing on a Service Desk
Manager installation.

Here are the steps to configure the Debug Diag Tool to monitor the domsrvr process and generate a full
crash dump when the process crashes:

1. Launch the Debug Diag Tool by going to START > Programs > Debug Diagnostics Tool 2.0 >
DebugDiag 2.0 Collection
2. You will get a pop-up box asking if you want to register the application:

3. Click “Yes” here to register the application

4. Click “OK” on the pop-up box confirming that the registration is complete

5. The Debug Diag Tool interface will now launch:

6. Select “Crash” as shown above, and click “Next”

7. Select the radio button for “A Specific Process” as shown here, and click “Next”:
 8. On the Select Target screen – it will present you with a list of all currently running processes on
the server. Here you will click on the process which you want the tool to monitor, again for our
example we will chose “domsrvr.exe” and then click “Next”:

**Note: The check box for “This process instance only” is only needed if there are multiple instances of
the same process name running, but you only want to monitor a specific instance or ProcessID (PID) that
has been identified as the problematic process. For our example we will leave that box unchecked, and
simply choose domsrvr.exe, which will tell the tool to monitor ALL instances of domsrvr.exe that are
running.**

9. Here, you will now choose the specifications for generating the dump file. Select “None” in the
first dropdown as shown here, leaving all the other fields set to their defaults as shown here:
IMPORTANT NOTE: When using the Debug Diag Tool for the Javaw.exe process, you must set the tool to
only generate dump files for “Access Violation” exceptions. If this is not done, the tool will repeatedly
generate dump files for the Javaw.exe process filling up the drive space on the server. Use the following
steps to set the tool to only generate dump files for “Access Violation” exceptions for the Javaw.exe
process: Click Exceptions button:
Click Add Exception in the First Chance Exception Configuration:

Select “Access Violation” as shown here, and click OK:

Next, click OK, then click Save and Close. You can now continue with the next steps to complete the
creation of the rule.

11. Next, specify a name for this “rule” and select a location to have the dump files stored when they
are generated by the tool. For our example, we will name the rule “Domsrvr Crash”, and set the
 location to C:\Dumps to store the dumps that are generated, and then click “Next”:

**Important Note: We recommend creating a directory called C:\Dumps, or D:\Dumps (use a drive in
which you have a decent amount of available storage space)

12. Select the radio button for “Activate the rule now”, and click on “Finish”:
 13. You will get a pop-up box asking if you want to use the Microsoft Symbol server path for the
debugging symbols used to debug the dump files. Click “Yes” on this window.

14. You will now see the “Domsrvr Crash” rule in the Rule List within the Tool – you may close out
of the tool at this point as the Tool will continue to run in the background running the rules that
are currently activated, and thus monitoring the selected processes – Domsrvr.exe in our
example:

Using the Debug Diag Tool to Manually Generate a Dump File for a Hanging Process
If a process appears to be in a “hung” state and does not appear to be responding, please confirm this
by performing the following steps:

First, run the following command to see if the process responds to a request via the command line:
“pdm_diag -a {slump name of process}”

**to get the slump name of the process, you can run the slstat command and pipe it out to a file
by running the following command: “slstat > slstat.txt”

Example: If it was a webengine hanging, and you found that the slump name for the failing webengine
as per the slstat output is “web:local” you would run the command as follows to see if that webengine
process is responding: “pdm_diag -a web:local”

If you receive information back from the process, then the process IS actually responding. If you do not
receive information back from the process, and it appears the command is hanging, then the process is
most likely in a “hung state”.

Take a screenshot of this output, and attach this to the ticket later when you gather all the information
required by CA Support to troubleshoot the issue.

Then, follow these steps to manually generate at least 2 to 3 FULL dumps for the hanging process,
each approximately 2 minutes apart:

For our example again, we will use the domsrvr.exe process:

1. Launch the Debug Diag Tool by going to START > Programs > Debug Diagnostics Tool 2.0 >
DebugDiag 2.0 Collection, and navigate to the “Processes” tab:

2. Locate the process that is suspect of hanging, and right click on that process, and select “Create
Full Userdump”
 3. The tool will now generate a full userdump for the process you selected. When its complete, a
pop-up window will appear advising you of the location of the dump file and file name that it
created:

WHAT TO DO WHEN THE NEXT CRASH OR HANG OCCURS

When a crash occurs, the Debug Diag Tool will generate dump files. At that time, you will need to follow
what we call the “Crash Dump Template” document (See Below…), which contains a list of items and
information that CA Support will need you to provide in order for us to then go to our Sustaining
Engineering team with the dumps. This information is required in order to be able to properly analyze the
dump file(s), based on the appropriate code base for the version and patch level of the product you are
using. This analysis can give us a good idea of what the process was doing that the point of failure.

FILES TO BE COLLECTED BY CUSTOMER AND SEND IT TO CA SUPPORT

Windows Crash Dump Template

Please fill this out as best you can after you capture a dump file for a dying, crashing or hanging
process.
Simply insert your answers/information to these items in-line below each item.
You may cut and paste this template into the issue via support.ca.com, or you may save it and
upload it to the issue as an attachment.
If you are unsure about a specific item - please ask your CA Support Engineer for clarification.

1. Please review the stdlog file that captures the timeframe of when the dump occurred and supply us
with the following information:

Was the process ended by a SIGSEGV message, a SIGBUS message or any another
“FATAL” type message?
What is seen in the stdlog file right before, during, and after the time the process crashed?
What errors were reported in the logs right before, during and after the time when the process
crashed if any?

2. What utility or mechanism used to generate the dump file? For example:

Upload the .DMP file and any utility generated logs from that utility - Specify the filename of
 the dump file (or zip file that contains the dump file) here.
If a process is thought to be in a “hung” state – please attach the output of the command
“pdm_diag -a {slump name of process}” (specify the file name here)

3. Please specify the date/time the dump file was generated.

4. How many times has the failing process crashed since first reported?
5. Are there any possible reproducible steps noted prior to when this crash/hang occurs?
6. Supply a “Directory Listing” output of the Service Desk root directory (NX_ROOT) by using a
command line window, navigating to the directory where Service Desk is installed, and running the
command “dir/s > dir_listing.out” - this will generate a file called dir_listing.out. Please upload this
file and specify the name of the file (or zip file that contains the dir_listing.out file) here.

7. Please run the command “winmsd” for Windows 2003, or “msinfo32” for Windows 2008 this should
pop up a window with system information. Click on the file menu and select export or save to save
the output to a file. Please upload that file and specify the name of the file (or zip file that contains
the output file) here.

NOTE - on some environments, for security reasons, winmsd/msinfo32 may not run. In this case,
please specify the specs of the hardware, and whether or not it is VMware based, for the system
where the dump file was generated, here.

8. Navigate to the Service Desk\bin directory and run

pdm_ident {process name} > pdm_ident.out. - where {process name} is the name of the Service
Desk process for which the dump file was generated.

If the failing process is javaw.exe - you will need to run a pdm_ident on the sda65.dll file as the
javaw process does not contain pdm_ident information.

Please upload the pdm_ident.out file, and specify the name of the file (or zip file that contains the
output file) here.

9. Please attach your patch history file ($NX_ROOT/<machine name>.his) to the issue and specify the
name of the file (or zip file that contains the history file) here.

10. Please zip up the entire Service Desk\log directory and attach it to the issue, and specify the name of
the zip file here.

11. Please zip up the Service Desk\site\mods directory and attach it to the issue, and specify the name
of the zip file here.

12. Please upload the Windows event log files, and specify the name of the files (or zip file containing
the event logs) here.