P. 1
Nice Perform Troubleshooting Guide - Rev. A2

Nice Perform Troubleshooting Guide - Rev. A2

4.6

|Views: 23,007|Likes:
Published by tcpip_trashcan4602
For Nice Perform
For Nice Perform

More info:

Published by: tcpip_trashcan4602 on Apr 11, 2009
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

10/29/2015

pdf

text

original

NICE Perform®

Release 3.1

Troubleshooting Guide
February 2009

385A0755-01 Rev. A2

Insight from Interactions

®
TM

Information in this document is subject to change without notice and does not represent a commitment on the part of NICE Systems Ltd. The systems described in this document are furnished under a license agreement or nondisclosure agreement. All information included in this document, such as text, graphics, photos, logos and images, is the exclusive property of NICE Systems Ltd. and protected by United States and international copyright laws. Permission is granted to view and photocopy (or print) materials from this document for personal, non-commercial use only. Any other copying, distribution, retransmission or modification of the information in this document, whether in electronic or hard copy form, without the express prior written permission of NICE Systems Ltd., is strictly prohibited. In the event of any permitted copying, redistribution or publication of copyrighted material, no changes in, or deletion of, author attribution, trademark legend or copyright notice shall be made. All contents of this document are: Copyright © 2009 NICE Systems Ltd. All rights reserved.
This product is covered by one or more of the following US patents:
4,893,197 5,396,371 6,115,746 6,330,025 6,785,370 6,959,079 7,203,655 7,436,887 5,185,780 5,446,603 6,122,665 6,542,602 6,856,343 6,965,886 7,240,328 5,216,744 5,457,782 6,192,346 6,564,368 6,865,604 6,970,829 7,305,082 5,274,738 5,819,005 6,246,752 6,694,374 6,870,920 7,010,106 7,333,445 5,289,368 5,911,134 6,249,570 6,728,345 6,871,229 7,010,109 7,346,186 5,325,292 5,937,029 6,252,946 6,775,372 6,880,004 7,058,589 7,383,199 5,339,203 6,044,355 6,252,947 6,785,369 6,937,706 7,085,728 7,386,105

360o View, ACTIMIZE, Actimize logo, Alpha, Customer Feedback, Dispatcher Assessment, Encorder, eNiceLink, Executive Connect, Executive Insight, FAST, FAST alpha Blue, FAST alpha Silver, FAST Video Security, Freedom, Freedom Connect, IEX, Interaction Capture Unit, Insight from Interactions, Investigator, Last Message Replay, Mirra, My Universe, NICE, NICE logo, NICE Analyzer, NiceCall, NiceCall Focus, NiceCLS, NICE Inform, NICE Learning, NiceLog, NICE Perform, NiceScreen, NICE SmartCenter, NICE Storage Center, NiceTrack, NiceUniverse, NiceUniverse Compact, NiceVision, NiceVision Alto, NiceVision Analytics, NiceVision ControlCenter, NiceVision Digital, NiceVision Harmony, NiceVision Mobile, NiceVision Net, NiceVision NVSAT, NiceVision Pro, Performix, Playback Organizer, Renaissance, Scenario Replay, ScreenSense, Tienna, TotalNet, TotalView, Universe, Wordnet are trademarks and registered trademarks of NICE Systems Ltd. All other registered and unregistered trademarks are the property of their respective owners. Applications to register certain of these marks have been filed in certain countries, including Australia, Brazil, the European Union, Israel, Japan, Mexico, Argentina and the United States. Some of such registrations have matured to registrations.

385A0755-01 Rev. A2

For assistance please contact your local supplier or the nearest NICE Systems Customer Service Center:
EMEA Region: (Europe, Middle East, Africa) Tel: +972-9-775-3800 Fax: +972-9-775-3000 email: support@nice.com APAC Region: (Asia/Pacific) Tel: +852-8338-9818 Fax: +852-2802-1800 email: support.apac@nice.com The Americas Region: (North, Central, South America) Tel: 1-800-NICE-611 Fax: +720-264-4012 email: support.americas@nice.com Israel: Tel: 09-775-3333 Fax: 09-775-3000 email: support@nice.com

NICE invites you to join the NICE User Group (NUG). Visit the NUG Website at www.niceusergroup.org, and follow the online instructions.
International Headquarters-Israel Tel: +972-9-775-3100 Fax: +972-9-775-3070 email: info@nice.com United Kingdom Tel: +44-8707-22-4000 Fax: +44-8707-22-4500 France Tel: +33-(0)1-41-38-5000 Fax: +33-(0)1-41-38-5001 Feedback North America Tel: 1-800-663-5601 Fax: +201-356-2197 email: na_sales@nice.com Germany Tel: +49-(0)-69-97177-0 Fax: +49-(0)-69-97177-200 Hong-Kong Tel: +852-2598-3838 Fax: +852-2802-1800

All queries, comments, and suggestions are welcome!

• •

If you have access to niceweb.nice.com, click here to open a feedback form
-or-

Email: nicebooks@nice.com

For more information about NICE, visit www.nice.com

Revision History
Troubleshooting Guide

Revision A1

Modification Date December 2008

Description

Added the following cross-server troubleshooting flows:

• • • •

Playback Troubleshooting Flow on page 77 Archiving Troubleshooting Flow on page 117 Multi Data Hub Troubleshooting Flow on page 141

Added a section for the playback error “There are no recordings to retrieve”. See Understanding Playback Retrieval on page 116. Added a section for the Log Analyzer tool. See Using Log Analyzer to Check Logs on page 312. Added an explanation for the “Chain not found” message. See Troubleshooting the “Chain Not Found” Error Message on page 352. Added appendix for displaying NICE system information in the System Administrator. See Enabling Display of NICE System Information in System Administrator on page 573. Added cross references and small changes to the Playback Troubleshooting Flow on page 77. Added cross references and small changes to the Archiving Troubleshooting Flow on page 117.

• •

A2

February

• •

Blank page for double-sided printing.

Contents
1
Introducing NICE Perform Troubleshooting 23
About this Troubleshooting Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Sources of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Finding the Technical and User Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Setting an Alert for any Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Synchronizing the Guides to Your Local Machine . . . . . . . . . . . . . . . . . . . . 33 Indexing and Searching the Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2
Recording Troubleshooting Flow 37
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Checking Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 General Recording Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Troubleshooting Flow Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Troubleshooting Call Failures using Exceptions . . . . . . . . . . . . . . . . . . . . . 54 Troubleshooting No New Calls in Business Analyzer . . . . . . . . . . . . . . . . . . 59 Troubleshooting Calls with Poor Audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Troubleshooting Calls with Audio from Several Calls . . . . . . . . . . . . . . . . . 67 Troubleshooting TRS Recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Troubleshooting Integration Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3
Playback Troubleshooting Flow 77
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Understanding the Playback Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 General Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 7

Troubleshooting Playback Problem for All Calls . . . . . . . . . . . . . . . . . . . . . 83 Troubleshooting Playback Problem for Specific Calls . . . . . . . . . . . . . . . . . 85 Setting the Log Level in System Administrator . . . . . . . . . . . . . . . . . . . . . . . 87 Setting the Log Level in the .Config File . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 Setting the Player Log Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Checking the Workstation Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Checking the File Versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Checking the Locate Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 Checking the Storage Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Troubleshooting Player Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Playback Logic Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Logger Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Storage Center Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Checking the Playback Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Sample Player Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Sample Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Storage Center Disconnection Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Storage Center with Playback Logic Log . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Logger Disconnection Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Checking the NiceScreen Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Example of Troubleshooting using the Logs. . . . . . . . . . . . . . . . . . . . . . . . 108 PlaybackAdministration.txt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 LocateLog.txt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Collecting the Server Logs using the NICE Perform Logs Collector Tool . 113 Collecting the Client Logs using NICE Perform Logs Collector Tool. . . . . 113 Copying the Playlist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Dumping the Client Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Understanding Playback Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Other Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

4
Archiving Troubleshooting Flow 117
General Archiving Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Tape/DVD Archiving Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 8

Media Library Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Backup Server Device Detection Troubleshooting . . . . . . . . . . . . . . . . . . . 123 DVD-RAM Devices Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 USB Tapes Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 SCSI Tapes Not Detected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Other Backup Server Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Storage Center Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Setting the Log Level to DEBUG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Checking for Archived Calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Gathering Storage Center Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Checking the Storage Retention and Deletion Values . . . . . . . . . . . . . . . . 133 Verifying that the Storage Center Database is Selected . . . . . . . . . . . . . . 133 Verifying the Used and Free Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Verifying the Storage Unit Threshold . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Checking that the Archiving Rule Inserts New Calls . . . . . . . . . . . . . . . . . 135 Troubleshooting the Archiving Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Checking the Storage Center Archiving Tables . . . . . . . . . . . . . . . . . . . . . 137 Checking the Storage Center Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Checking the Nice Storage Streaming Service Logs . . . . . . . . . . . . . . . . . 139 Checking the Player Logs for Storage Center Errors . . . . . . . . . . . . . . . . . 139 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

5
Multi Data Hub Troubleshooting Flow 141
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Multi Data Hub Implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 System Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Rule Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Business Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Playback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Quality Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 My Universe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 9

Interaction Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Storage Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Checking Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Checking Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Troubleshooting Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Troubleshooting Communication Problems between Primary and Secondary Data Hubs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Troubleshooting Database Structure Differences . . . . . . . . . . . . . . . . . . . 147 Stopping Replication When Upgrading All Data Hubs . . . . . . . . . . . . . . . . 147 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Troubleshooting Saving in a Multi Data Hub Environment . . . . . . . . . . . . . 148 Changing Passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Enabling MSDTC to Function in Multi Site Environments. . . . . . . . . . . . . . 148 Other Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

6
Media Encryption Troubleshooting Flow 151
Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Changing the Logging Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Checking Server Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Checking the Logger/ScreenAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Checking the SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Checking the KSM Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Verifying the KSM is Functioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Checking the Client-side Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Checking the Server-side Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Checking for Certificate Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Trap Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Trap Messages for CIM Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Trap Messages for KSM Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 SAMS Log Messages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 ScreenAgent Module Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 Verifying the IIS User Permissions for the KSM Configuration . . . . . . . . . 164
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 10

Using a Fully Qualified Domain Name instead of a Host Name . . . . . . . . . 166 Data Not Entered in the nice_crypto Database . . . . . . . . . . . . . . . . . . . . . . 169 Renewing Certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Screens Not Recorded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 ScreenAgent Not Working Properly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Ensuring Connection between SQL Server and Applications Server . . . . 175 Installing the CA certificate on Vista Workstations. . . . . . . . . . . . . . . . . . . 175

7
Channel Mapping Troubleshooting Flow 177
Channel Mapping Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . 179 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Identifying a Channel Mapping Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Verifying the Channel Mapping Configuration . . . . . . . . . . . . . . . . . . . . . . 181 Troubleshooting an Out of Memory Error . . . . . . . . . . . . . . . . . . . . . . . . . 183 Troubleshooting Two Users Updating Channel Mapping Concurrently . . . 184 Troubleshooting Saving in a Multi Data Hub Environment . . . . . . . . . . . . . 184 Adding a New Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Open Case Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Open Case Procedures for System Administrator Parameters (CLS) . . . . 186 Open Case Procedures for the RCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 Troubleshooting Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Error Message appears when Updating Configuration . . . . . . . . . . . . . . . 190 A New or Modified Mapping Definition Does Not Work . . . . . . . . . . . . . . . 191 New Channel Mapping Definitions are not Saved . . . . . . . . . . . . . . . . . . . 191 Channel Mapping does not Work! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 Channel Mapping in System Administrator Differs from what is in Effect . . 192 Channels Definitions: Logger does not Appear . . . . . . . . . . . . . . . . . . . . . 192 Channels Definitions: Input Channels do not Appear . . . . . . . . . . . . . . . . 193 Source Definitions: Cannot define Virtual Extensions . . . . . . . . . . . . . . . . 193 Source Definitions: Physical Switch does not Appear . . . . . . . . . . . . . . . . 193 Dynamic Mapping: Screen Logger does not Appear . . . . . . . . . . . . . . . . . 194 Dynamic Mapping: Virtual Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 11

Deleting or Changing Logger, Switch, or CLS Definitions . . . . . . . . . . . . . 194 Procedure for Reassigning a Logger to a Different CLS . . . . . . . . . . . . . . 195

8
Applications Troubleshooting Flow 197
Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Checking the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Troubleshooting a Communications Failure . . . . . . . . . . . . . . . . . . . . . . . . 202 Troubleshooting Communications Failure with Importer or Reporter . . . . 203 Checking the Applications Server Services . . . . . . . . . . . . . . . . . . . . . . . . 205 Using NICE Services Configuration Manager . . . . . . . . . . . . . . . . . . . . . . 206 Solving Application Services User Rights Problem . . . . . . . . . . . . . . . . . . 207 Checking the Installation Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Checking the Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Setting the Log Level in System Administrator . . . . . . . . . . . . . . . . . . . . . 209 Setting the Log Level in the Config File . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Solving Desktop Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Verifying the .NET Framework Versions . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Set Security Wizard Continues to Appear After Installation . . . . . . . . . . . . 218 Problem Occurs on One Workstation . . . . . . . . . . . . . . . . . . . . . . . . . . 218 Problem Occurs on All Workstations . . . . . . . . . . . . . . . . . . . . . . . . . . 220 Solving Set Security Wizard Hangs or Displays an Error . . . . . . . . . . . . . . 221 Set Security Manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 Set Security Manually Using Caspol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 Solving Post Installation HTTP Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 Verifying the Applications are Not Loaded under the Internet Zone . . . . . . 228 Configuring NICE Perform as a Trusted Web Site . . . . . . . . . . . . . . . . . . . 228 Checking NICE Perform Initialization Errors . . . . . . . . . . . . . . . . . . . . . . . 229 Solving Applications Server Configuration Problem . . . . . . . . . . . . . . . 229 Troubleshooting Remote Name Could Not be Resolved Error . . . . . . . 230 Troubleshooting Unable to Connect to the Remote Server Error . . . . . 230 Troubleshooting Serialization Problem . . . . . . . . . . . . . . . . . . . . . . . . 230
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 12

Solving Service Unavailable Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 IIS AppPool Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Non-Paged Pool Memory Leak Problem . . . . . . . . . . . . . . . . . . . . . . . 233 Disabling Custom Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Solving ActiveX Control Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Troubleshooting Application Access (Power Management Issue) . . . . . . . 234 Troubleshooting Browser Problems Using Logs . . . . . . . . . . . . . . . . . . . . 237 Explorer Log (on the Client Machine) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Desktop Client Logs (on the Client Machine) . . . . . . . . . . . . . . . . . . . . . . 237 IIS - WWW Log File (on the Applications Server) . . . . . . . . . . . . . . . . . . . 238 IIS - HTTPERR Log File (on the Applications Server) . . . . . . . . . . . . . . . . 239 Application Event Viewer (on the Applications Server) . . . . . . . . . . . . . . . 239 Solving ActiveX Control Error on the Applications Server . . . . . . . . . . . . . 241 Optimizing Client Server Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 Optimizing Client Server Performance on Computers without Internet . . . 244 Redirecting the Client Request to the Local Server . . . . . . . . . . . . . . . . . . 244 Modifying the Security Settings in the Internet Explorer . . . . . . . . . . . . . . 245 Solving Email Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Verifying the SMTP Service Installation and Configuration . . . . . . . . . . . . 246 Troubleshooting SMTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 Troubleshooting Crystal Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 Using the Process Explorer Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Using the Desktop Tester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

9
Databases Troubleshooting Flow 253
General Databases Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 255 General Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Checking the Windows Event Viewer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Checking the SQL Server Error Logs and Events . . . . . . . . . . . . . . . . . . . . 256 Checking SQL Jobs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Checking the Connection to the NICE Perform Database Server . . . . . . . . 257 Checking the Database Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Checking for Missing Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 13

Checking Free Space in the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 Verifying the Log on as a Service User Rights Assignment . . . . . . . . . . . . 260 Verifying the Service Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 Verifying Domain Users Assigned to the User Rights Assignments . . . . . . 261 Checking the SQL Server Installation Folder . . . . . . . . . . . . . . . . . . . . . . . 263 Monitoring Performance Objects and Counters . . . . . . . . . . . . . . . . . . . . . 265 Checking for Performance Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266 Scanning the Disk for Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Checking Replication in Multi Data-Hub Environments . . . . . . . . . . . . . . . 268 Changing the Database Passwords in SQL Authentication . . . . . . . . . . . . 269 Changing the nice sa Login Password . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Changing the nice Login Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Checking the Database Backup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Increasing the Size of a Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Checking Performance Using the NICE Performance Collector . . . . . . . . . 274 Defining Autogrowth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Using the NICE DBSpace Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 Analyzing the Database Using NICE DBAgnose . . . . . . . . . . . . . . . . . . . . . 284 General Data Mart Troubleshooting Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 287 Data Mart Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Checking the Database Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Recreating Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Checking the Data Mart Database Version . . . . . . . . . . . . . . . . . . . . . . . . 289 Checking Connectivity Between the Database and Data Mart Servers . . . 289 Checking a Data Mart Job Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290

10
Customer Feedback Installation Troubleshooting Flow 291
General Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Troubleshooting Customer Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 Troubleshooting Flow Expanded . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Troubleshooting - General Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Troubleshooting - DTMF Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 Troubleshooting - CTI Info or Business Data Routing . . . . . . . . . . . . . 298
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 14

Surveys and Interactions are not Properly Correlated . . . . . . . . . . . . . . . . 301

11
High Density Logger Troubleshooting Flow 303
General High Density Logger Troubleshooting Flow . . . . . . . . . . . . . . . . . 304 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Checking the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Verifying the Logger Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306 Generating a Build Content Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Checking Logger Initiation Failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308 Setting the Debug Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Using Bug Find to Check Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Using Log Analyzer to Check Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Checking for Logger Configuration Errors . . . . . . . . . . . . . . . . . . . . . . . . . 314 Checking Logger Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Checking Map Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 Checking RCM Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 Synchronizing the Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 Testing the Logger COTS using the NICE Test Kit . . . . . . . . . . . . . . . . . . . 318 Tests and Log File Locations in a Windows Environment . . . . . . . . . . . . . 318 Tests and Log File Locations in a DOS Environment . . . . . . . . . . . . . . . . 320 Testing the NICE Boards using the NICE Test Kit . . . . . . . . . . . . . . . . . . . . 320 Testing the Logger using the NICE PST. . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Shutting Down and Starting Up the Logger . . . . . . . . . . . . . . . . . . . . . . . . . 322 Shutting Down and Starting CLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Using the Nicelog Setup Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

15

12
VoIP Troubleshooting Flow 329
Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 Isolating the VoIP Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330 Troubleshooting VoIP Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Troubleshooting the VoIP Logger LAN Connection . . . . . . . . . . . . . . . . . . 332 Using Logs to Check that the VoIP Logger is Configured to Receive RTP Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 Checking the RTP Packets using the Sniffer . . . . . . . . . . . . . . . . . . . . . . . 337 Troubleshooting Network and Environment Issues . . . . . . . . . . . . . . . . . . 338 Checking the CLS Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 Checking Channel Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 Using the Sniffer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 Troubleshooting N+1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 mascm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Configuring after Replacing a NIC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Replacing a CISCO Phone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Troubleshooting a Packet Size Problem . . . . . . . . . . . . . . . . . . . . . . . . . . 349 Troubleshooting RTP Timestamp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350 Troubleshooting RDP Delay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Troubleshooting the “Chain Not Found” Error Message . . . . . . . . . . . . . . 352 Troubleshooting Active VoIP Integrations . . . . . . . . . . . . . . . . . . . . . . . . . 353

13
Interactions Center Troubleshooting Flow 355
Interactions Center Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . 356 Interactions-based Call Recording Data Flow . . . . . . . . . . . . . . . . . . . . . . . 357 Interactions-based Call Recording Data Flow Details . . . . . . . . . . . . . . . . 358 Understanding the Total Call Recording Data Flow . . . . . . . . . . . . . . . . . . 360 Understanding the Total Call Recording Data Flow Details . . . . . . . . . . . . 361 Understanding the Screen Call Recording Data Flow . . . . . . . . . . . . . . . . . 363 Screen Call Recording Flow Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 16

Open Case Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 RecordingDecisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 Call Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 DB Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 TRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 RCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 System Administrator Parameters (CLS Plug-ins) . . . . . . . . . . . . . . . . . . . 371 Checking the Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 Checking the SNMP Traps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 Setting the Reporting Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Running the RCM Information Dumper . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 Checking the CLS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 Starting/Stopping the Interactions Center Services . . . . . . . . . . . . . . . . . . 378 Checking the Status of Interactions Center Components. . . . . . . . . . . . . . 379 Checking Components with the NICE Services Configuration Manager . . 379 Checking Processes Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380 Troubleshooting using the Interactions Center Logs . . . . . . . . . . . . . . . . . 381 Examples of Troubleshooting using the Logs. . . . . . . . . . . . . . . . . . . . . . . 383 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 Checking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408 Checking the CLS Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 Query the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 In the Business Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 Fine-tuning the CLS Server Configuration Parameters. . . . . . . . . . . . . . . . 411 Isolating a Screen Problem Further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 Shutting Down and Starting CLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414 Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415

14
NMS Troubleshooting Flow 417
Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 17

Troubleshooting NMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420 Client-Side Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Pinging the Monitored Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421 Checking the SNMP Service Configuration . . . . . . . . . . . . . . . . . . . . . . . . 421 Verifying that SNMP Agent is Sending Traps . . . . . . . . . . . . . . . . . . . . . . 425 Repairing the SNMP Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 Reinstalling the NICE Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 Server-Side Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 Collecting the System Log Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 Checking the NMS Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 Checking the CA-Unicenter Service and Sub-Services . . . . . . . . . . . . 428 Checking the Unicenter TND Agent Technology Service . . . . . . . . . . . 432 Checking the SNMP Trap Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434 Checking the SNMP Nice Trap Formatter Service . . . . . . . . . . . . . . . . 434 Checking the Trap Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 Checking the MIB File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Checking Polling Timeout Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 Troubleshooting Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 Scenario 1: Monitored Server Does Not Appear in the 2D Map . . . . . . 442 Scenario 2: Drilldown Fails in the 2D Map . . . . . . . . . . . . . . . . . . . . . . 442 Scenario 3: Only Some Agents are Reporting . . . . . . . . . . . . . . . . . . . 442 Scenario 4: There are No Traps in the Console . . . . . . . . . . . . . . . . . . 442 Scenario 5: No Email Notifications Are Received . . . . . . . . . . . . . . . . 443 Changing the System Administrator Server Name . . . . . . . . . . . . . . . . . . . 445 Checking for a CAClient Serialization Issue . . . . . . . . . . . . . . . . . . . . . . . . 445 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 Using the NMSUtil Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

15
Telephony Services Server Troubleshooting Flow 449
Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450 Gathering Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451 Checking the Telephony Services Server Configuration . . . . . . . . . . . . . . 452 Checking the Dialogic Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 18

Troubleshooting the Dialogic Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454 Troubleshooting the Dialogic Analog Interface . . . . . . . . . . . . . . . . . . . . . 454 Troubleshooting the Dialogic Digital Interface . . . . . . . . . . . . . . . . . . . . . . 457 Troubleshooting the Dialogic IP Interface . . . . . . . . . . . . . . . . . . . . . . . . . 460 Using the ISDIAG Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461 Testing the Extension Using the Dialogic Voice Demo . . . . . . . . . . . . . . . 464 Testing an Analog Extension Using the Dialogic Voice Demo . . . . . . . 464 Testing a Digital Extension Using the Dialogic Voice Demo . . . . . . . . . 467 Using the Universal Dialogic Diagnostics Utility . . . . . . . . . . . . . . . . . . . . 470 Troubleshooting using the NICE Playback Telephony Layer Tester . . . . . 474 Checking the Telephony Services Server Service . . . . . . . . . . . . . . . . . . . 478 Checking the Applications Server Service . . . . . . . . . . . . . . . . . . . . . . . . . 478 Checking the Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479 Setting the Logs Reporting Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479 Viewing the Telephony Services Server Logs . . . . . . . . . . . . . . . . . . . . . . 480 Viewing the NICE Playback Streaming Services Logs . . . . . . . . . . . . . . . . 481 Viewing the NICE Playback Administration Service Logs . . . . . . . . . . . . . 481 Viewing the NICE Player Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 Checking Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 482 Troubleshooting Playback to Extension Problems . . . . . . . . . . . . . . . . . . . 482 Checking Ports Using Port Tester . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Shutting Down and Starting the Telephony Services Server . . . . . . . . . . . 485 Troubleshooting Dial Plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

16
NiceScreen Troubleshooting Flow 489
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 Isolating the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 Gathering Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 Checking the NiceScreen Logger Service Status . . . . . . . . . . . . . . . . . . . 492 Checking the ScreenAgent Interactions Center Registration . . . . . . . . . . . 493 Checking the Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494 Checking SAMS for Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 19

Isolating the Problem Further . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 Checking the NiceScreen Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 Checking the ScreenAgent Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496 Checking the RCM Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Checking the Call Server Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Checking the Call Status in the Database . . . . . . . . . . . . . . . . . . . . . . . . . 497 Checking the Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497 Checking the Naming Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 Checking Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498 Verifying the ScreenAgent Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 500 Testing the ScreenAgent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500 Other Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501 Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 501 NICE ScreenAgent Installation/Operational Problems . . . . . . . . . . . . . . . . 503 NICE ScreenAgent Optimization and Performance Issues . . . . . . . . . . . . . 505 NiceScreen Logger Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . 507

17
Storage Center Troubleshooting Flow 513
Common Troubleshooting Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 Installation Freezes: SNMP Dependencies . . . . . . . . . . . . . . . . . . . . . . . . 514 Archiving Failure: Multiple Network Interface Cards . . . . . . . . . . . . . . . . . 515 Storage Center Is Not Communicating with the NMS . . . . . . . . . . . . . . . . 515 Verify SCLoader Service User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 Verify the SnmpAgent Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 Verify the SNMP Service Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 Allowing Duplicate Archiving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521 Storage Center Does Not Archive Calls . . . . . . . . . . . . . . . . . . . . . . . . . . 521 Other Useful Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524 Verifying the Storage Center Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . 526 Implementing the Troubleshooting Workflow . . . . . . . . . . . . . . . . . . . . . . . 527 Verifying that the Storage Center Database is Selected . . . . . . . . . . . . . . 527 Verifying the Used and Free Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528 Gathering Storage Center Activity Statistics . . . . . . . . . . . . . . . . . . . . . . . 528
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 20

Checking that the Archiving Rule Inserts New Calls . . . . . . . . . . . . . . . . . 528 Troubleshooting the Archiving Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529 Checking the Storage Center Archiving . . . . . . . . . . . . . . . . . . . . . . . . . . 531 Checking the Storage Center Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 531 Checking the Nice Storage Streaming Service Logs . . . . . . . . . . . . . . . . . 532 Verifying Playback from Storage Center Only . . . . . . . . . . . . . . . . . . . . . . . 533

A
Collecting Information Using the Log Collector 537
What is the Log Collector? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 What is a Component? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 What is a Report? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539 FAQs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 540 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 542 Collecting Information from NICE Perform Servers . . . . . . . . . . . . . . . . . . 543 Collecting Information from Your Local Machine . . . . . . . . . . . . . . . . . . . . 550 Saving and Exporting Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Importing Saved Reports for Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556 Scheduling Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 558 Viewing Collected Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 560 Viewing and Saving Reports in other Applications . . . . . . . . . . . . . . . . . . 562 Reports Created with the Log Collector. . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 Server-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 General Server-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 Applications Suite Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564 Playback Telephony Services Server Reports . . . . . . . . . . . . . . . . . . . 566 Logger Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566 SQL Server Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 Interactions Center Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 Storage Center Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567 Reporter Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568 NiceScreen Logger Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568 ScreenSense Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568
Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 21

Stream Server Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568 Business Services Framework (BSF) Server Reports . . . . . . . . . . . . . 569 CTI Integrations Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 Media Library Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 Audio Analysis Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569 VRG Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570 MPCM (Media Provider Controller Manager) Reports . . . . . . . . . . . . . 570 RTS Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570 Client-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 General Client-Side Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 Player Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 ScreenAgent Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 ScreenSense Agent Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571

B
Enabling Display of NICE System Information in System Administrator 573
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 Common Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 Verifying the System Administrator User . . . . . . . . . . . . . . . . . . . . . . . . . . 576 Adding the System Administrator User to Computers not in an Active Directory Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578 Editing DCOM Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582 Setting Security Permissions through the WMI Control . . . . . . . . . . . . . . . 588

Contents NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

22

1
Introducing NICE Perform Troubleshooting
This troubleshooting guide includes troubleshooting flows and procedures that are intended for use by support staff and NICE support engineers in Tier 1, 2 and 3. This guide does not include procedures for Tier 4.

Contents
About this Troubleshooting Guide................................................................................ 24 Sources of Information .................................................................................................. 26 Finding the Technical and User Guides ....................................................................... 27 Setting an Alert for any Changes .................................................................................. 30 Synchronizing the Guides to Your Local Machine ...................................................... 33 Indexing and Searching the Guides.............................................................................. 34

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

23

About this Troubleshooting Guide

About this Troubleshooting Guide
This troubleshooting guide includes troubleshooting flows and procedures that are intended for use by support staff and NICE support engineers in Tier 1, 2 and 3. This guide does not include procedures for Tier 4. This guide includes the following troubleshooting:

Recording Troubleshooting Flow: Includes a general recording troubleshooting flow, as

well as several recording troubleshooting scenarios. This flow refers to the relevant components, as required.

• •

Media Encryption Troubleshooting Flow: Includes a troubleshooting workflow, as well as

tables of trap messages and ScreenAgent Management System (SAMS) log messages.
Channel Mapping Troubleshooting Flow: Includes a channel mapping troubleshooting

flow, as well as provides quick solutions for common problems you face when defining channel mapping. The procedures are designed for the seasoned user who is familiar with the NICE Perform system and the System Administrator application. For complete instructions on using the System Administrator, see the System Administrator’s Guide.

Applications Server Troubleshooting Flow: Contains a troubleshooting workflow for the

NICE Perform Applications Server, including solving Desktop problems and troubleshooting Browser problems using Logs. A list of the services, config and log files is also provided.

Databases Troubleshooting Flow: Includes procedures for troubleshooting NICE Database

Server and Data Mart. These procedures must be perform only by authorized personnel. For full assistance, contact NICE Systems Customer Services.

Customer Feedback Installation Troubleshooting Flow: Provides a review of some information that will aid the installer in understanding the Customer Feedback workflow. Also provided is a workflow recommending how to proceed when troubleshooting a Customer Feedback system. High Density Logger Troubleshooting Flow: Contains a troubleshooting flow with

• •

troubleshooting tips and procedures for High Density Loggers.
Interactions Center Troubleshooting Flow: Includes a Interactions Center troubleshooting

workflow, Interactions-based call recording data flow, screen call recording data flow, total call recording data flow, and other troubleshooting issues.

• • • •

NMS Troubleshooting Flow: Includes an NMS troubleshooting flow and other

troubleshooting issues.
Playback Telephony Services Server Troubleshooting Flow: Includes a troubleshooting

flow and other troubleshooting issues.
NiceScreen Troubleshooting Flow: Lists some of the most common problems you may

encounter and solutions.
Storage Center Troubleshooting Flow: Describes common troubleshooting issues and

how to verify that the Storage Center was properly installed. It also includes a troubleshooting workflow to solve various issues.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

24

About this Troubleshooting Guide

• •

VoIP Troubleshooting Flow: Contains troubleshooting tips and procedures for when you

experience a problem with VoIP.
Collection Information Using the Log Collector: The Log Collector is an easy-to-use utility that gathers the information from all your NICE Perform servers and local workstation, in the form of reports, for easy transference for analysis.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

25

Sources of Information

Sources of Information
There are several additional sources of information.
1. There are also over 150 Technical and User Guides on ExtraNICE (http://www.extranice.com)

• • • • • • • •

Hardware Guides History Release Notes Installation Guides Reference Guides Release Notes Switch/Driver Integration Guides Upgrade/Migration Information User Guides

2. The NICE Perform Release 3.1 Maintenance Guide is especially useful and should be read

together with this guide.
3. Internal (for NICE-use only) sources include:

• • • •

Technical Documentation Team Site for technical documents in progress: http://niceweb.nice.com/sites/Technical_Writers_Documents/default.aspx Tier 4 Team Site for tools and utilities: http://niceweb.nice.com/teams/Tier4_RD/default.aspx Tier 3 Team Site: http://niceweb/sites/Tier3_CS/Tier3 working documents and manuals/Forms/AllItems.aspx Product Training Team Site: http://niceweb.nice.com/sites/ProductTraining_RD/default.aspx

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

26

Finding the Technical and User Guides

Finding the Technical and User Guides
There are over 150 guides on ExtraNICE.
1. Browse to http://www.extranice.com/. You must have access to the relevant content on

ExtraNICE.
2. If you are accessing ExtraNICE from outside of the NICE network, the following window

appears.
Figure 1-1 ExtraNICE Portal

3. Enter your User name and Password and click on Log On to ExtraNICE. The following

window appears.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

27

Finding the Technical and User Guides

Figure 1-2

ExtraNICE Portal Home Page

4. Select Products > NICE Perform > NICE Perform Release 3.1. The following window

appears.
Figure 1-3 NICE Perform Release 3.1

5. Expand the Content Type: Technical and User Guides. 6. Expand the required Files Types to view a list of manuals.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

28

Finding the Technical and User Guides

Figure 1-4

List of Manuals

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

29

Setting an Alert for any Changes

Setting an Alert for any Changes
You can set a daily alert of any changes to the content in SharePoint.
1. In ExtraNICE, select Products > NICE Perform > NICE Perform Release 3.1. The

following window appears.
Figure 1-5 ExtraNICE Portal - NICE Perform Release 3.1

2. Click on NICE Perform Release 3.1. The following window appears.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

30

Setting an Alert for any Changes

Figure 1-6

NICE Perform Release 3.1 - Actions

3. Select the Actions drop-down menu.
Figure 1-7 Actions Drop-down Menu

4. Select Alert Me. The following window appears.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

31

Setting an Alert for any Changes

Figure 1-8

Alerts Page

5. Select Send a daily summary. 6. Click OK. A daily alert is emailed to you detailing any changes or updates made to the

selected page.
7. Copy the changed files to your local folder.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

32

Synchronizing the Guides to Your Local Machine

Synchronizing the Guides to Your Local Machine
Use a tool of your choice to synchronize the Guides from SharePoint to your local machine. For demonstration purposes only we describe how to use Colligo Reader.
1. Download and install http://colligo.com/products/sharepoint/reader.asp. 2. Select Start > Program > Colligo Reader > Colligo Reader. 3. From the Site menu, select Download New Sites. The following window appears.
Figure 1-9 Download New Site Window

4. In the URL field, add http://www.extranice.com/EIS/Products/NICEPerform. 5. Add your User name and Password. 6. Click Download. The files are synchronized to your local machine. 7. The files are located in a folder, for example, C:\Documents and Settings\<username>\Local Settings\Application Data\ColligoOfflineClient\Storage5\Files\Sites\1\NICE Perform Release 3.1 .

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

33

Indexing and Searching the Guides

Indexing and Searching the Guides
Use a search tool of your choice to search for information. For demonstration purposes only we describe how to use Windows Desktop Search.
1. Download and install Windows Desktop Search

http://www.microsoft.com/windows/products/winfamily/desktopsearch/getitnow.mspx.
2. Click on the Indexing icon in the Taskbar.
Figure 1-10 Index Now

3. Select Windows Desktop Search Options. The following window appears.
Figure 1-11 Indexing Options

4. Click Modify. The following window appears.

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

34

Indexing and Searching the Guides

Figure 1-12

Indexed Locations

5. Select the location with the manuals and click OK. 6. Use your selected search tool to locate the relevant information.
Figure 1-13 Search Results

Chapter 1: Introducing NICE Perform Troubleshooting NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

35

Blank page for double-sided printing.

2
Recording Troubleshooting Flow
This section contains troubleshooting tips and procedures for when you experience a problem with recording.

Contents
Gathering Information .................................................................................................... 38 Checking Exceptions ..................................................................................................... 39 General Recording Troubleshooting Flow ................................................................... 50 Troubleshooting Call Failures using Exceptions ........................................................ 54 Troubleshooting No New Calls in Business Analyzer................................................. 59 Troubleshooting Calls with Poor Audio ....................................................................... 66 Troubleshooting Calls with Audio from Several Calls ................................................ 67 Troubleshooting TRS Recordings................................................................................. 69 Troubleshooting Integration Issues.............................................................................. 75

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

37

Gathering Information

Gathering Information
In order to narrow down the problem answer the following questions:
1. Is the recording TDM or VoIP? 2. For VoIP, is the recording environment passive or active? 3. For TDM, is the recording environment trunk-side, extension-side or observation? 4. Is the recording total or QM (interaction based)? 5. In a total recording environment is TRS configured? 6. Is the site N+1? 7. Is the recording summed or with speaker separation? 8. Is the channel mapping static or dynamic? If necessary export the channel mapping

definitions. See Exporting Channel Mapping Definitions in the Channel Mapping Guide.
9. Did the problem occur for a specific agent, group of agents or for all agents? 10. Which CTI integration and what version? 11. Check for exceptions by checking the tblExceptionXX table. See Checking Exceptions

on page 39.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

38

Checking Exceptions

Checking Exceptions
Query the database to see the exceptions. Most of the time exceptions are connected to recordings and not directly to the interaction IDs. In order to see the connection between an exception and interaction ID, query both the exception details and the related recording (which includes the interaction ID).
To check the CLS Exceptions in the Database: 1. Query the tblInteractionCatalog table to check for which period the calls occur. 2. Get all the exceptions related to interactions: SELECT * FROM tblExceptionXX, tblInteractionXX WHERE tblExceptionXX.iInteractionID = tblInteractionXX.iInteractionID 3. Get all exceptions related to recordings: SELECT * FROM tblExceptionXX, tblRecordingXX WHERE tblExceptionXX.iRecordingID = tblRecordingXX.iRecordingID 4. Check the vcExceptionDetail field in tblException. To check the CLS Exceptions in Business Analyzer:

In Business Analyzer, select the interaction and click the Exceptions tab in the Details pane.
Figure 2-1 Checking the CLS Exceptions in Business Analyzer

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

39

Checking Exceptions

ID 1

Exception Short Description Duplicate call start.

Meaning Duplicate start of call – another call with the same call key has started while this call was open. Call too long – the call was open for more seconds than the value of the registry parameter MaxOpenCallDuration and was therefore forcibly closed. Call flushed – a flush command was executed while the call was open.

Possible Cause Problem with the driver or PABX reports.

Recommended Action Check the driver or PABX. Collect the driver and CLS log files.

2

Maximum duration exceeded.

Either the call was longer than the parameters value, or there is a problem with the driver or PABX reports.

Check if the value of the Call Server parameter op_MaxOpenCallDuration matches the length of calls in the site.

3

Call flushed while open.

The driver may have gone down and come up.

Check that the driver is up. Review the driver log files, look for errors. If there is no error no action is needed. N/A Check the driver or PABX. Review the driver log files, look for errors. If there are errors, collect the driver and CLS log files.

4 5

N/A Call start not reported.

N/A End call without start (Default Start Time).

N/A Problem with the driver or PABX reports. Could be followed by reports with exp02 because the start and end call reports have different information.

6

N/A

N/A

N/A

N/A

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

40

Checking Exceptions

ID 7

Exception Short Description No available recording resource.

Meaning No available recording resource.

Possible Cause There are not enough voice recording channels to meet the number of recording requests in the site.

Recommended Action Could be caused by unclosed recorded calls – check for exp02 calls. If there is a substantial amount of calls with these exceptions, check RCM logs for Logger disconnections (Logger down) and if channels were allocated after connection reestablished with Logger. Check the Logger. Check the network connectivity between the Logger and the CLS. Check the RCM logs. Collect the Integration and CLS log files. Check if it is physically possible to logout during a call. Collect the Integration and CLS log files.

8

Logger not responding.

Logger not responding.

Problem with Logger or connection to Logger.

9 10

Unspecified recording failure. Agent logout during call.

Unspecified recording failure. Agent logout during the call.

Unknown failure cause. Received from the RCM. Problem with the driver or PABX call or logout reports.

11

Too many calls for agent/extension.

MaxExtentionOpenCall - too many open calls for the same agent/extension when compared to the op_MaxCallPerExt registry parameter.

The agent had more simultaneous open segments than the parameter value. Could be a problem with the driver or PABX call reports.

Check if the parameter value meets the requirements of the site. Look for calls with exp02 to see if there is a problem with the call reports.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

41

Checking Exceptions

ID 12

Exception Short Description Voice recording failed.

Meaning Voice recording failed.

Possible Cause Error code received from the telephony server. If this appears as e12 check the sub-exception ID for more information.

Recommended Action

• • •

Collect the Integration and CLS log files. Sub-exception 12 310 indicates the Logger is down. Sub-exception 12 1702 indicates no VoIP audio due to a configuration problem. This is dependant on the VoIP integration. Check the configuration, the forwarding data passed to the Capture (RCM logs), the Capture logs, as well as the forwarding device configuration. For example, no audio is received if no or the wrong forwarding information is passed to Capture, or if the audio is not forwarded by the telephony switch/forwarding device.

13

Screen Logger not responding.

Screen logger not responding.

Logger is down or network issue.

Check the NiceScreen Logger and ScreenAgent. Check the RCM logs for errors when calling screen capture to start record.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

42

Checking Exceptions

ID 14

Exception Short Description Screen recording failed.

Meaning Screen recording failed.

Possible Cause If this appears as e14, check the sub-exception ID for more information.

Recommended Action

Sub-exception 14 1002 indicates an error was received during screen recording. Either ScreenAgent was disconnected from the Interactions Center or ScreenAgent recording failed. Partial recording (up until the time of the error) may be available. Sub-exception 14 4 indicates there was an unspecified failure.


15 Unmapped voice recording. Unmapped voice recording. Problem in the voice channel configuration.

Switch the logs to DEBUG. Check channels configuration. See if there is a mapping configured for this call. Check the RCM logs. Check what the RCM received in the Start request.

16

Unmapped screen recording.

Unmapped screen recording.

The recording request was received with empty Station or IP address (depends on the screen allocation mode).

If the allocation mode is by IP address then check if the agent logged in. Check ports. Verify in RCM logs that the start request for recording the agent's screen contained the screen agent's IP. Check Logger for possible reasons for temporary failures.

17

Voice recording retry.

Recording voice succeeded only after retry (partial retry).

May be a temporary failure on the Logger.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

43

Checking Exceptions

ID 18

Exception Short Description Call Server service shutdown. N/A Logger not responding.

Meaning Call Server was down during the call. N/A The Logger did not respond to the start record command.

Possible Cause Call Server was down.

Recommended Action Check reason for Call Server failure. Collect CLS logs, event viewer and CPU Performance Monitor.

19 20

N/A Stop record command arrived before a response for the start record request arrived. This may occur for one of the following reasons:

N/A No action is needed if there was indeed a short call; otherwise collect CLS and driver log files. If the request was for 2 medias, try to understand the recording problem with the first media. Check the RCM logs. Check the Call Server logs.

• • • • •
21 22 N/A N/A N/A N/A N/A N/A

The call was very short (1 or 2 seconds). The request was for 2 medias. Success in both was required (usually QA). One media failed immediately. Stop record is sent for both medias.

N/A N/A

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

44

Checking Exceptions

ID 23

Exception Short Description Call start not reported.

Meaning Stop record without start (screen or voice).

Possible Cause Usually happens with very short calls. The request was for 2 medias. Success in both was required (usually QA). One media failed immediately. Stop record was sent for both medias. Or may be an internal problem.

Recommended Action No action is needed if there was indeed a short call; otherwise collect CLS and driver log files. If the request was for 2 medias, try to understand the recording problem with the first media. Check the RCM logs. Check the Call Server logs.

24

Error in stop record request.

Stop record with wrong ID. No start call request was found with this CLS Call ID.

Usually happens with very short calls. The request was for 2 medias. Success in both was required (usually QA). One media failed immediately. Stop record was sent for both medias. Or may be an internal problem.

No action is needed if there was indeed a short call; otherwise collect CLS and driver log files. If the request was for 2 medias, try to understand the recording problem with the first media. Check the RCM logs. Check the Call Server logs.

25

Too many requests for channel. RCM service down during call.

Too many recording requests for the same Logger and channel (more than 30). RCM was down during the call.

May occur due to a problem with the driver or PABX call reports – calls are not closed. RCM was down.

Collect CLS and driver log files. Check the Call Server logs. Collect CLS logs, event viewer and CPU Performance Monitor. Check the RCM logs. Check the reason that the RCM went down.

26

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

45

Checking Exceptions

ID 27

Exception Short Description Error on complete interaction start.

Meaning The contact started after its segment.

Possible Cause Problem in the driver or PABX call or logout reports.

Recommended Action Check the driver or PABX. Collect CLS and driver log files. Check the RCM logs for a long period in DEBUG mode.

28

No available recording resource.

No available recording resource. The Resource Call Manager cannot allocate more resources for this initiator.

Not enough Logger recording channels. Bad resource management configuration. There maybe a problem with calls that are not closed and therefore resources are not being freed up. Could be a problem with the drivers or PABX call reports. Call Server restarted.

See if the configuration meets the site needs. If not change it. Collect CLS and driver log files. Restart RCM. Check the RCM logs. Could be a problem with the driver or PABX reports. Check the driver or PABX. Collect CLS and driver log files. Collect CLS log files. Check why Call Server was closed.

29 30

Error on complete interaction close. Time Interval recording aborted. Client disconnect.

Segment was open when the contact closed. Block dummy call was closed due to a Call Server restart. Client was disconnected during the open block.

31

Client was disconnected.

Collect CLS log files. Check why the client was disconnected. Check the Call Server logs.

32

N/A

N/A

N/A

N/A

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

46

Checking Exceptions

ID 33

Exception Short Description Stop on demand not by initiator.

Meaning Stop on demand was performed on the interaction recording by a client who was not the recording initiator. Time field was changed by the DB Server. Interaction was inserted with time value lower then 1970. Stop time was lower than start time. The Call Server received a timeout error on the request to the RCM.

Possible Cause Stop on demand was performed on the interaction recording by a client who was not the recording initiator. Call Server error. Illegal time parameters were reported.

Recommended Action No action required.

34

Invalid call time report.

Collect CLS log files. Check the Call Server logs.

35

RCM service not responding.

RCM is too busy. Internal RCM error. Event caused the RCM not to respond, for example, detected many Loggers initializing. Business data length is incorrectly specified.

Collect CLS logs, event viewer and CPU Performance Monitor. Check the RCM logs.

36

Business data value too long.

String field was truncated by the DB Server. Call Server sent a string longer than allowed by the DB schema.

Collect CLS and driver log files. Check the maximum business data length, change it in the DB and restart the CLS.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

47

Checking Exceptions

ID 37

Exception Short Description Failed to record VoIP.

Meaning Failed to record. Audio not found in VoIP recording.

Possible Cause This usually results from a failure to sniff VoIP packets by the Logger. After the Start command to the Logger, the RCM waits 2.5 seconds and checks if the audio packets were received.

Recommended Action

• • • • •

Run a sniffer to check if packets were coming in. Collect CLS and Logger log files. Check for channel mapping failures. Check the RCM logs for the allocated channel. Check the Capture logs.

38

Failed to update VoIP data.

Update VoIP (which updates the IP for an open request and its participant) sent by the Driver failed.

Occurs, for example, when the update is for an unmapped device or a closed request, or because the update was not received by the RCM. Interaction Router could not find mapped segment on the Interactions Center that was selected for the other open segment under the same contact. Problem with integration. Calls may be reported late to the CLS when the system is under stress.

Check that the device in the Update VoIP sent from driver is mapped.

39

Complete interaction handled on multiple servers.

Split Contact - Segments for contact handled on more than one Interactions Center.

No action required.

40

Recording time mismatch.

Recording Time Mismatch – recording time was out of call time range.

Check the Call Server logs.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

48

Checking Exceptions

ID 41

Exception Short Description Unknown Initiator.

Meaning At the start of the call, the initiator participant has no User ID, that is, the User ID was not found in the UserAdmin Database.

Possible Cause User is not defined under UserAdmin.

Recommended Action Ensure the user exists and is defined in the DB.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

49

General Recording Troubleshooting Flow

General Recording Troubleshooting Flow
See Troubleshooting Flow Details on page 52 for a description of every step mentioned in the following flow.
Figure 2-2 General Recording Troubleshooting Flow

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

50

General Recording Troubleshooting Flow

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

51

General Recording Troubleshooting Flow

Troubleshooting Flow Details
1. Is the problem a system-level problem, that is, no recording at all? 2. If the problem is a system-level problem:

• • • •

Check that all services are up and running (Applications Server, Interactions Center and Loggers). Check that the Loggers are up. Check the communication between servers. Check that there is time synchronization between the various NICE servers.

3. Check if the problem is a data or audio problem, by running a query in Business Analyzer. If

there are no interactions, continue with Step 7.
4. Is the problem a Logger-level problem? 5. If the problem is a Logger-level problem: a.

Verify the Logger configuration in System Administrator, check that the Logger was added to the CLS and that it is up (by selecting it and verifying that you see all the Logger's details). Check that the Loggers are time synchronized. For VoIP environments, see the VoIP Troubleshooting Flow. Monitor the test call. If the test call appears in Monitor, try to ROD the test call.

b. Check the Logger communication with the Applications Server and Interactions Center. c.

d. For TDM environments, see the High Density Logger Troubleshooting Flow. e.

6. Make a test call: a.

b. If the test call does not appear in the Monitor, proceed with Step 7. c.

d. If the test call is not recorded, check the mapping.

For a trunk-side recording environment, add the participant trunk group and participant trunk number columns in Business Analyzer, and verify which trunk groups and trunk numbers were assigned to the interactions. Check the mapping for these trunks.

e. f.

For TDM trunk-side recording, check the trunk connections. If the test call is recorded, try to play back the test call in Business Analyzer. Verify that you can see the rule in the Rules Manager table in the DB. Verify that you do not see errors in the Scheduler and schedule sync.

g. If the test call can be played back in Business Analyzer, check if there are Scheduler rules.

h. If the test call cannot be played back in Business Analyzer, see the Playback

Troubleshooting Flow.
i.

For VoIP recording problems, see the VoIP Troubleshooting Guide.
Chapter 2: Recording Troubleshooting Flow 52

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

General Recording Troubleshooting Flow

7. Is the problem a CTI Server to Driver problem? 8. If the problem is a CTI Server to Driver problem: a.

Open the driver window. Check for event errors in the window (errors appear in red).

b. Make a call. c.

d. Trace the call in the Connection Manager and Driver logs (D:\Program Files\Nice Systems\CTI\Log) and try to locate errors. e.

See the relevant Integration Guide to troubleshoot the problem.

9. Is the problem a Driver to CLS problem? 10. If the problem is a Driver to CLS problem: a.

Trace the call in the Call Server and RCM logs (D:\Program Files\Nice Systems\Interactions Center\Log) and try to locate errors. Search by time or extension number to identify the problem.

b. See the Interactions Center Troubleshooting Flow. 11. Is the problem an Interactions Center process problem? 12. If the problem is an Interactions Center process problem: a.

If the event triggers a recording, identify the call initiator: ScreenSense, Total, QA/Rule-based, or ROD.

b. Check the call events and mappings in the Call Server logs and RCM logs (D:\Program Files\Nice Systems\Interactions Center\Log). For ScreenSense tagging requests there

are no RCM logs.
c.

Check the DBServer logs to check the end of the call. See the NICE Interactions Center Troubleshooting Flow.

d. Check the CLS exceptions. See Checking Exceptions on page 39. e.

13. Is the problem a CLS to Database problem? 14. If the problem is a CLS to Database problem: a.

Query SQL Server database (tblInteractions). See the Interaction Data Reference Guide. Database Server.

b. If the interaction is missing or not complete, check for a problem between the CLS and the c.

See the NICE Interactions Center Troubleshooting Flow.

d. See the Databases Troubleshooting Flow.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

53

Troubleshooting Call Failures using Exceptions

Troubleshooting Call Failures using Exceptions
This flow is appropriate for silent calls and other call failures.
1. In Business Analyzer, note the Interaction ID of the calls. 2. Check for interaction exceptions: a.

In the database expand nice_interactions > Tables.
Figure 2-3 tbllnteractionCatalog Table

b. Query the tblInteractionCatalog table to check for which period the calls occur.

c.

Run a query, where XX is the period in which the exceptions occur, and YY is the interaction ID:
SELECT * FROM tblExceptionXX where iInteractionID = YY

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

54

Troubleshooting Call Failures using Exceptions

Figure 2-4

Running a Query

d. Check the vcExceptionDetail field to identify the exception. e.

Troubleshoot the exception using Checking the CLS Exceptions in the Interactions Center Troubleshooting Flow. Run a query, where XX is the period in which the exceptions occur, and YY is the interaction ID:
SELECT * FROM tblRecordingXX where iInteractionID = YY.

3. Check the recordings. a.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

55

Troubleshooting Call Failures using Exceptions

Figure 2-5

Check the Recordings

b. Note the iLogger, iChannel and time of the recording. This will be used later to

troubleshoot the problem.
4. Depending on the type of recording make a note of the following information. Table 2-1: Recording Type Information Recording Type Recording Type Trunk/Digital Matrix Extension-side Information Information Trunk number, time slot, Logger, as well as agent and agent group (for interaction based recording) Extension number, channel mapped to the extension, Logger, as well as agent and agent group (for interaction based recording) Extension number, IP address, MAC address, Logger, as well as agent and agent group (for interaction based recording) Virtual extension, extension associated with the virtual extension, Logger, as well as agent and agent group (for interaction based recording)

Passive VoIP

Active VoIP (Service Observation)

5. For interaction based recording, check the rule: a.

On the Interactions Center open the D:\program files\Nice systems\Interactions Center\BIN\Log\Call Server.log.xxx.txt log files.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

56

Troubleshooting Call Failures using Exceptions

b. The following line indicates that the rule engine is checking if the call should be recorded: 18/07/08 16:04:38.317 DEBUG [Command Thread, 5680]: RecordingDecisions::Selector: Recieved a call for decision: Agent_ID=179389, Extension=25938, Switch_ID=25, Direction=0, Dnis=17209, Dialed_Num=, CLI=7172472759 ,BusinessData= c.

One of the following lines appears after the rule check to see if the call matches any of the active recording rules:

If the call does not match any of the active recording rules:
18/07/08 16:04:38.317 DEBUG [Command Thread, 5680]: CALLSRVR: No Recording Command Exist for AgentID 179389, Ext 25938

a.

If the call does match one of the active recording rules:
18/07/08 15:58:34.662 INFO [Command Thread, 5680]: Start record Request to RCM.

b. If the call is not recorded because it does not match one of the active rules, check in Rules

Manager that the Scheduler rule that is recording this agent or group of agents is valid and active.
6. Connect to the Logger using the Remote Desktop and check if there are any errors (use mstsc /console and not mstsc). See the High Density Logger Troubleshooting Flow. For the VoIP Logger, also check the IPCaptureLog.sav file. This contains the first error in the current

session, or if there are no errors in the current session, the first error in the last session when an error occurred. See the VoIP Troubleshooting Flow.
7. For trunk-side recording: a.

Use the Functional Tester to check the trunk status. Tool to solve the problem. See the Using the Nicelog Setup Tool in the High Density Logger Troubleshooting Flow.

b. If there is a problem with frame loss, trunk not synchronized, etc, use the NiceLog Setup

8. For extension-side recording: a.

Make a call from the extension, speak and leave the call open. If the activity is on a channel different to that noted in Step 4, check the channel mapping. Flow.

b. Use the Functional Tester to check which channel has activity. c.

d. If there is no activity, check the Logger. See to the High Density Logger Troubleshooting 9. For VoIP recording: a.

Use the Functional Tester to check which channel has activity. If there is no channel activity, the problem is most likely a VoIP Logger problem. in the VoIP Troubleshooting Flow. If there are packets, check the sniffing NIC configuration in the Logger IP tool.

b. Use the sniffer to check the RTP packets. See Checking the RTP Packets Using the Sniffer

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

57

Troubleshooting Call Failures using Exceptions

c.

If there are no packets, troubleshoot the network and environment. See the VoIP Logger Troubleshooting Flow. In particular check:

• •

Passive VoIP: Check the mirroring and network topology. Active VoIP: Check that the NIC IP is routable from the source IP address, and verify that the port intervals configured in the Logger IP Tool are open on the firewall for UDP packets.

10. For Avaya DMCC active VoIP recording: a.

Check in the Observer logs that the virtual extension is registered to the correct extension. If the correct extension was not used, check the configuration.

b. If the correct extension was used, run the sniffer to check the packets. c.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

58

Troubleshooting No New Calls in Business Analyzer

Troubleshooting No New Calls in Business Analyzer
1. Checking free space in the NICE_Interactions database. If required, increase the database

size, see the Databases Troubleshooting Flow.
a.

In the Start menu, select Programs > Microsoft SQL Server 2005 > SQL Server Management Studio, and click Connect.

b. In the Object Explorer, expand Databases, right-click on the relevant database, select Properties, and check the Space available. This provides the space of the entire DB with

no distinction between the data size and log size.
c.

Right-click the database, select Reports > Standard reports > Disk usage. The usage is displayed graphically with the separation for data and log. and select each database to see the used and free space percentages.
Figure 2-6 Database Size

d. In System Administrator, expand the Master Site > Database Servers > Databases

2. Check the NMS for trap 1023. To find the reason for the failure to insert the record, the DBSrvr log files (on the Interactions Center in D:\program files\Nice systems\Interactions Center\BIN\Log\DBServer.log.xxx.txt files) should be backed up

and sent to NICE Customer Support. The metadata for the Call might be inserted into the database by the TRS (if it exists), but in order to ensure that this is the case, NICE Customer Support should be consulted.
3. Check in the Driver window that calls are being sent to the CLS by looking for Establish

events.
13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: [07/21/06 18:37:49] DELIVERED TO APPLICATION: MonitorCrossRefID 63 EstablishedEvent ::= { establishedConnection { callID 15008, deviceID "5204", devIDType 0 }, answeringDevice
Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 59

Troubleshooting No New Calls in Business Analyzer

13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296: 13699296:

{ deviceID "5204", deviceIDType 55, deviceIDStatus 0 }, callingDevice { deviceID "97228776", deviceIDType 55, deviceIDStatus 0 }, calledDevice { deviceID "1691", deviceIDType 55, deviceIDStatus 0

4. Check the Call Server log files (on the Interactions Center in D:\program files\Nice systems\Interactions Center\BIN\Log\Call Server.log.xxx.txt files) for the call reported event. For more information on troubleshooting using log files, see the NICE Interactions Center Troubleshooting Flow.

Insert Interaction to DB
InteractionData *********** properties: ======== LocalStartTime = 17/02/2008 13:16:34 . LocalStopTime = 17/02/2008 13:16:39 . GMTStartTime = 17/02/2008 11:16:34 . GMTStopTime = 17/02/2008 11:16:39 . OriginatorInteractionID 11269314 . members: ======== InteractionId : 11269314 interactionOriginalID : 11269314 Duration : 00:00:05 OpenReasonId : 128 CloseReasonId : 128 SwitchId : 1 InitiatorUserId : 10 QATypeId : False InteractionTypeId : 2 RecordedTypeId : 0 TypeData : CallInteractionData *********** properties: ======== members: ======== PbxCallIndex :

=

11269314
Chapter 2: Recording Troubleshooting Flow 60

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting No New Calls in Business Analyzer

CallDirectionTypeId : 1 CompoundId : 11269311 IsPlaybackCall : False ********************* IsEvaluated : False iMediaTypesId : 2 additionalDataCollection[0] : participantsArray[0] : ParticipantData *********** properties: ======== members: ======== ParticipantId : 11269312 Station : 1 SwitchId : 1 IsFirstUser : True IsInteractionInitiator : False DeviceTypeId : 0 DeviceId : 0 ParticipantTypeId : 1 UserId : 10 ********************* participantsArray[1] : ParticipantData *********** properties: ======== members: ======== ParticipantId : 11269313 Station : 144 SwitchId : 0 IsFirstUser : False IsInteractionInitiator : True DeviceTypeId : 0 DeviceId : 0 ClientId : 0 ParticipantTypeId : 2 1

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

61

Troubleshooting No New Calls in Business Analyzer

********************* recordingsArray[0] : RecordingData *********** properties: ======== RecordingGMTStartTime = 17/02/2008 13:51:26 . RecordingGMTStopTime = 17/02/2008 13:51:29 . members: ======== Logger : 10225 Channel : 5 iMediaTypeId : 2 InitiatorTypeId : 64 InitiatorUserId : 0 RecordingSideTypeId : 3 RecordingRecordedTypeId : 3 RecordedParticipantId : 11269312 TimeDiff : -02:34:50.2600000 WrapupTime : 00:00:00 m_iTransientIndex : 0 ********************* recordingsArray[1] : RecordingData *********** properties: ======== RecordingGMTStartTime = 17/02/2008 13:51:26 . RecordingGMTStopTime = 17/02/2008 13:51:29 . members: ======== Logger : 10225 Channel : 5 iMediaTypeId : 2 InitiatorTypeId : 64 InitiatorUserId : 0 RecordingSideTypeId : 3 RecordingRecordedTypeId : 3 RecordedParticipantId : 11269313 TimeDiff : -02:34:50.2600000 WrapupTime : 00:00:00 m_iTransientIndex : 0 *********************

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

62

Troubleshooting No New Calls in Business Analyzer

*********************

Insert Contact (If Compound Ended)
ContactData *********** properties: ======== GMTStartTime = 17/02/2008 11:16:34 . GMTStopTime 11:16:39 . members: ======== ContactId : 11269311 Duration : 00:00:05 OpenReasonId : 256 CloseReasonId : 256 TransferSiteId : 0 TransferContactId : 0 RecordedTypeId : 0 QATypeId : False DirectionTypeId : 1 IsEvaluated : False iMediaTypesId : 2 InitiatorSwitchId : 1 InitiatorStation : 1 InitiatorUserId : 10 m_attachedInteractionInfo : InteractionUpdatableData *********** properties: ======== members: ======== m_interactionsInfo[0] : InteractionUpdatableInfo *********** properties: ======== InteractionId = 11269314 . QATypeId False . members: ======== ********************* ********************* *********************
Chapter 2: Recording Troubleshooting Flow 63

=

17/02/2008

=

False . IsOverwrite

=

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting No New Calls in Business Analyzer

5. Check the Call Server log files (on the Interactions Center in D:\program files\Nice systems\Interactions Center\BIN\Log\DBServer.log.xxx.txt files) for the call inserted

event. For more information on troubleshooting using log files, see the NICE Interactions Center Troubleshooting Flow.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

64

Troubleshooting No New Calls in Business Analyzer

6. Check if the call was inserted into the database table: a.

Query the tblInteractionCatalog table to check for which period the calls occur.

b. Check the tblInteractionXX and tblRecordingXX tables. 7. If the calls are in the database the problem may be in the Business Analyzer query: a.

Try to use the NICE user when querying. Collect the logs from the Applications Server.

b. Ensure that you do not have any expression or filter applied to the query. c.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

65

Troubleshooting Calls with Poor Audio

Troubleshooting Calls with Poor Audio
1. If VoIP calls are recorded with poor audio, see the VoIP Troubleshooting Flow. 2. For extension-side recording, check the connections between the handset, PABX, ALI board

and Punch Down Block (Krone).
3. For trunk-side recording, check for an error on the channel using the Functional Tester.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

66

Troubleshooting Calls with Audio from Several Calls

Troubleshooting Calls with Audio from Several Calls
The following symptoms may occur:

• • •

Call duration equals the "op_MaxOpenCallDuration" parameter (by default 5 hours). tblException shows INF002, EXP002, Exp005, INF029. Calls have the wrong voice, calls are cut off or start from the middle and this is not a mapping problem.

To troubleshoot the problem: 1. Check the integration troubleshooting in the relevant Integration Guide. 2. Ensure that all devices are monitored. Some integrations require that all ACDs, VDNs, hunt

group IVRs be monitored.
3. Ensure that the latest Update is installed. 4. Check what information is required to open a case as described in the Integration Guide. 5. Check which call to analyze: a.

Business Analyzer, note the Interaction ID (XXXX) of the call. Query the tblInteractionCatalog table to check for which period the calls occur.
Figure 2-7 tblInteractionCatalog Table

b. In the database expand nice_interactions > Tables. c.

d. Check the Compound ID for this call, by running the following query: select iCompoundID from tblCallInteractionXX where iInteractionID = 'XXXX' e.

Check how many calls belong to the Compound ID, by running the following query:
select * from tblCallInteractionXX where iCompoundID = 'XXXX'

6. If there are multiple results it means that the compound was wrongly associated with other

calls. Focus on the first call and note the:
Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 67

Troubleshooting Calls with Audio from Several Calls

• • • • •

PBX Call ID:
select IPBXCallID from tblCallInteraction where iInteractionID = 'XXXX'

Compound ID
select IcompoundID from tblCallInteraction where iInteractionID = 'XXXX'

Station:
select nvcStation from tblParticipantXX where iInteractionID = 'XXXX'

Switch ID:
select iSwitchID from tblParticipantXX where iInteractionID = 'XXXX'

Time of the call:
select dtInteractionLocalStarttime from tblInteractionXX where iInteractionID = 'XXXX'

7. Investigate the call events: init: The caller picks up the phone. Delivered: The call between the two parties is ringing. Establish: The recipient picks up the phone. Connection clear: The call is over. Transfer: The call is transferred from one agent to another. Held / Retrieve: One of the parties pressed hold or retrieved the call. a.

Look for the Call ID and time in the driver and CTI logs (D:\NICECTI\Integration\Log). or transfer. If there is no connection clear or transfer, then it is a third party problem or not all devices are being monitored (see Step 2).

b. Look for the device number and try to follow the flow of events up to the connection clear

c.

If there is a transfer event then there will be a new call ID, so continue the check using the device number.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

68

Troubleshooting TRS Recordings

Troubleshooting TRS Recordings
1. Switch the log reporting level to debug: a.

In System Administrator, verify that you are working in Technician mode. tab.

b. In the Organization tree, select CLS Definitions > CLS Server. Click the Report Level c.

Expand TRS. Repeat for Call Server and RCM. In System Administrator, verify that you are working in Technician mode. driver.

d. Set the File to Debug. Do not set the Screen to Debug as this will affect performance. e.

2. Switch the driver logs to Debug: a.

b. In the Organization tree, select CTI Integrations > Drivers, and select the relevant c.

Expand the Driver Reporting Level. Select the checkboxes for the reporting levels you want to include, and click OK.

d. Select the desired row and click Edit. e.

3. Check the volume of TRS calls. In Reporter, run the TRS Interaction Volume Report and TRS Interaction Volume by Logger Report . In order to identify the TRS status, this report

presents the interaction volume up until to the latest TRS run.
Figure 2-8 TRS Interaction Volume Report

4. Are there too many TRS calls? Continue with step 15.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

69

Troubleshooting TRS Recordings

5. If there are fewer calls than expected or no calls, check if the CLS/Driver was down for an

extended period while the Logger was recording.
6. If there are fewer calls than expected or no calls, check if the problem is related to specific

Loggers by checking the TRS Interaction Volume by Logger report. If it is a general problem, continue with Step 9.
7. Check the connectivity between the Loggers and the Interactions Center: a.

Ping the Loggers and Interactions Center.

b. Check if Trap ID 1095 clsTRSFailed QueryLogger or 1096 clsTRSSucceededQueryLogger appears. For further details, see the NMS User's

Guide.
c.

Check the TRS logs (on the Interactions Center in D:\program files\Nice systems\Interactions Center\BIN\Log\TRS.log.xxx.txt files). Density Logger Troubleshooting Flow.

d. Check the Logger logs (in D:\NTLogger\logger\log\LogFilex.DAT). See the High 8. Check if the problematic Loggers are associated with the same Interactions Center. If the

problem is related to a specific Interactions Center:
a.

Check if the NICE CLS TRS service is started.

b. Check the TRS traps. For further details, see the NMS User's Guide. Table 2-2: TRS Traps Trap ID 1089 clsTRSisDown Description The TRS service is not running. Interactions Center Module: TRS Recommended Action Check the SNMP messages and the TRS log files and try to determine the reason that the service is down. Check if the service was closed down manually. 1090 clsTRSIsUp TRS Service has started and initialized properly. Interactions Center Module: TRS 1093 clsTRSFailed toQueryCLS DB The TRS has failed to execute an SQL command. Interactions Center Module: TRS Check if the SQL Server is up and running. Verify that the server machine where the Interactions Center runs has connection to the server machine where the nice_interactions database is installed. Analyze the TRS log files.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

70

Troubleshooting TRS Recordings

Table 2-2: TRS Traps (Continued) Trap ID 1094 clsTRS SucceededTo QueryCLSDB 1095 clsTRSFailed QueryLogger Description TRS successfully connected and queried the nice_interactions database. Interactions Center Module: TRS Indicates a communication problem with the Logger. Interactions Center Module: TRS Verify that the Logger is up and running. Verify that the Interactions Center is properly connected to the Logger. Check if RCM also has a communication problem with the same Logger. Recommended Action

1096 clsTRS Succeeded QueryLogger c.

TRS successfully connected and queried the Logger. Interactions Center Module: TRS

In System Administrator, check the TRS configuration.

d. Check the TRS logs (on the Interactions Center in D:\program files\Nice systems\Interactions Center\BIN\Log\TRS.log.xxx.txt files). 9. If the problem is not related to a specific Interactions Center: a.

Change the Logger log levels to debug:

• • •

Open d:\NTlogger\logger\config\SysRC.dat in Notepad. In [system -default] set the LogeLevel2File and LogLevel2Screen to 5 instead of 3. Reboot the Logger. See Shutting Down and Starting Up the Logger in the High Density Logger Troubleshooting Flow.

b. Check that the Logger is up. c.

Check the Logger logs. See the High Density Logger Troubleshooting Flow. In System Administrator, expand the Master Site > Data Marts and select the required Data Mart server. In the SQL Jobs area, select the Data Mart Population Schedule job. The job steps are detailed in the History area.

10. Check that the Data Mart Population Schedule job ran: a.

b. Click the Monitor tab. c.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

71

Troubleshooting TRS Recordings

Figure 2-9

Data Mart Population Schedule Job

11. Collect the relevant log files using Perform Mate including the Driver, Call Server, RCM,

TRS, DBServer and any problem Logger logs. See the High Density Logger Troubleshooting Flow and NICE Interactions Center Troubleshooting Flow.
12. Collect the following data tables:

• •

tblInserterLoggers tblInserterCalls (relevant part only):

13. In System Administrator, check the TRS parameters: a.

In the Organization tree, select CLS Definitions > CLS Server.

b. Select the required CLS Server, and click the Configuration tab. c.

Click

next to TRS. Note the parameter values.

IMPORTANT Any change of parameters should be done ONLY in consultation with NICE Support. d. For a list of the parameters see the NICE Interactions Center Troubleshooting Flow. 14. Check if the TRS parameters were modified: a.

In Audit Trail create and run the following advanced query.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

72

Troubleshooting TRS Recordings

Figure 2-10

Advanced Query

b. Check if the parameter value has changed, and what the previous and new values are.
Figure 2-11 Audit Trail

15. If there are too many TRS calls: a.

In Business Analyzer, check if the same calls appear as inserted by both the Interactions Center (an agent name appears) AND the TRS (User- Unmapped appears). If this is the case, provide a list with these calls with their recording times, Logger ID and channel ID.

b. In Reporter, run the TRS Interaction Volume Report and TRS Interaction Volume by Logger Report in order to identify whether the problem is related to a specific TRS or

specific Loggers.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

73

Troubleshooting TRS Recordings

c.

If the problem is related to a specific Logger:

• •

Run a query in Business Analyzer to see whether the increase is for a specific channel. Try to playback several of these calls and see whether the calls are invalid, noisy, ring tones, DTMF or IVR calls. Check for driver problems, for example, internal errors, disconnections between the drivers and the PABX or internal PABX, and so on. See the troubleshooting in the relevant Integrations Guide. Check for Interactions Center or Interactions Center subcomponents problems. For further information, see the NICE Interactions Center Troubleshooting Flow.

d. If the increase is not related to a specific Logger, check the following:


e.

Collect the relevant log files using Perform Mate including the Driver, Call Server, RCM, TRS, DBServer and any problem Logger logs. See the High Density Logger Troubleshooting Flow and NICE Interactions Center Troubleshooting Flow. Collect the following data tables:
tblInserterLoggers tblInserterCalls (relevant part only):

f.

• •

16. Switch all log levels (CLS Server, Drivers and Loggers) back to what they were before you

started this troubleshooting procedure.

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

74

Troubleshooting Integration Issues

Troubleshooting Integration Issues
Various tools and utilities for various additional troubleshooting and performance issues are detailed in the relevant Integration Guide:

• • •

NICE Events Spy enables you to trace events after they were transferred from the PABX to the Connection Manager, enabling you to detect bugs or malfunctions. NICE Debug Service enables you to gather data critical for solving problems stemming from

the transfer of events between the switch and the Connection Manager.
NICE Connection Manager Monitor tool enables you to view the contents of the

Connection Manager's tables. It also enables you to verify if:

• • • • •

Devices are monitored Monitored devices are filtered Displays the loaded CTI links Displays connected clients.

Log Manager system logs all significant system activity and maintains a log of all data,

enabling you to view the history of all relevant system activity. The Log Manager system has three main components:

• • • •

Log Manager Log Manager Services Log Viewer

CAPI Spy enables you to monitor all messages sent by the CTI driver to the CLS CAPI (Call

Server). Examination of these messages enables you to pinpoint whether the problem is in the CTI driver or in the CLS CAPI server. CAPI Spy has two main components:

• •

CAPI Spy Plug-in CAPI Spy Utility

Chapter 2: Recording Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

75

Blank page for double-sided printing.

3
Playback Troubleshooting Flow
This chapter describes the playback flow and common troubleshooting issues when playback is not functioning properly.

Contents
Gathering Information .................................................................................................... 78 Understanding the Playback Flow ................................................................................ 79 General Troubleshooting Flow ...................................................................................... 81 Troubleshooting Playback Problem for All Calls......................................................... 83 Troubleshooting Playback Problem for Specific Calls ............................................... 85 Setting the Log Level in System Administrator........................................................... 87 Setting the Log Level in the .Config File ...................................................................... 88 Setting the Player Log Levels ....................................................................................... 90 Checking the Workstation Ports ................................................................................... 92 Checking the File Versions ............................................................................................ 92 Checking the Locate Order............................................................................................ 94 Checking the Storage Center ........................................................................................ 95 Troubleshooting Player Errors ...................................................................................... 96 Example of Troubleshooting using the Logs.............................................................108 Example of Troubleshooting using the Logs.............................................................108 Collecting the Server Logs using the NICE Perform Logs Collector Tool .............. 113 Collecting the Client Logs using NICE Perform Logs Collector Tool...................... 113 Copying the Playlist ..................................................................................................... 114 Dumping the Client Memory ........................................................................................ 115 Understanding Playback Retrieval.............................................................................. 116 Other Troubleshooting ................................................................................................. 116
Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 77

Gathering Information

Gathering Information
Important issues to consider when troubleshooting for playback misfunction are:

• • • • •

Is this a new installation? If yes, when was the installation executed? Were there any changes to the environment that might have affected the system? If yes, check these changes first. How often does the problem occur? Can the problem be retraced? If yes, specify the steps. Were the Servers restarted? If yes, did this solve the problem?

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

78

Understanding the Playback Flow

Understanding the Playback Flow
The Playback Flow is described in the chart and procedure below.
Figure 3-1 Playback Flow

1. The user selects an interaction in Business Analyzer (Client) and clicks Play. 2. The Business Analyzer (Client) sends the playback request to the Player. 3. The Player sends the playback request, including the call parameters (site ID, call ID, segment

or complete, user ID of users requesting playback, user ID of the agent to play back) to the Playback Administration on the Applications Server.
4. The Playback Administration checks the privileges with the User Admin to ensure that the

user is allowed to playback the interaction.
5. The Playback Administration then gathers the interaction data from the Database Server:

• • • •

tblRecordingXX tblInteractionXX tblParticipantXX tblContacXX for complete calls

6. The Playback Administration uses tblRecordingXX to locate where the interaction is stored

based on the locate order.
NOTE: Recordings with Recording Type = Not Recorded are ignored.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

79

Understanding the Playback Flow

7. The Playback Administration gathers the locate results. 8. The Playback Administration builds the playlist (xml) for each located media, including the

start and end times and where the media is stored.
9. The playlist is sent to the Player. If the playlist contains no medias, the user receives a No recordings were found message. See Understanding Playback Retrieval on page 116 for

an explanation of this message.
10. The NICE Player eliminates any redundant medias according to the participants. 11. The NICE Player/Stream Server fetches the media from the Loggers and/or Storage Center,

and plays the call.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

80

General Troubleshooting Flow

General Troubleshooting Flow
General troubleshooting is described in the flowchart and procedures below.
Figure 3-2 Troubleshooting Flow

1. What method of playback are you using?

• • •

Playing back to an extension. Playing back directly from the Logger or Storage Center, that is, with a direction connection between the Player and the Logger or Storage Center. Playing back via a Stream Server. See Choosing a Stream Server Configuration in the System Administrator's Guide.

2. If your problem is playing back to an extension, see Playback Telephony Server Services

Troubleshooting.
3. Is the problem playing back a specific call or all calls?
Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 81

General Troubleshooting Flow

4. If the problem is for all calls, see Troubleshooting Playback Problem for All Calls

on page 83.
5. If the problem is for specific calls only, see Troubleshooting Playback Problem for Specific

Calls on page 85
6. If the site is a Multi Data Hub environment:

• •

Check if the agent and the Logger are located in the same site and physical location. Try to playback the call locally. If this works, the problem is most likely a network issue, for example, timeout, latency, and so on. Check where the call is physically located in locatelog.txt. See Example of Troubleshooting using the Logs on page 108.

7. If you have N+1 or 1+1 the call may be recorded on a couple of Loggers:

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

82

Troubleshooting Playback Problem for All Calls

Troubleshooting Playback Problem for All Calls
Troubleshooting playback problems for all calls is described in the flowchart and procedures below.
Figure 3-3 Troubleshooting Flow for Playing Back All Calls

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

83

Troubleshooting Playback Problem for All Calls

If the problem is for all calls: 1. Set the appropriate log level:

• • •

Set the Log Level in the System Administrator. Setting the Log Level in System Administrator on page 87. Set the Log Level in the Config File. See Setting the Log Level in the .Config File on page 88. Set the Player Log Level. See Setting the Player Log Levels on page 90.

2. Check that the Applications Server and Database Servers are up. See Applications

Troubleshooting Flow on page 197 and Databases Troubleshooting Flow on page 253.
3. Check the time synchronization between the Applications Server and Database Server. 4. If playing back over a WAN, check if there is a latency problem or communication errors. 5. If other users can play back the calls on the same workstation when logging in, check the user

permissions in the User’s Admininstrator application.
6. If you can play back the calls when logged-in to a different workstation, check the workstation

connectivity and also check workstation problems. See Solving Desktop Problems on page 217.
7. In System Administrator, check the range of the workstation ports, and ensure that these ports

are open. See Checking the Workstation Ports on page 92.
8. On the Applications Server, check for a conflict between the NICE updates in the ServerBin

and ClientBin folders. See Checking the File Versions on page 92.
9. If a No Recording Found message appears for the calls in the Player, check the Locate Order. See Checking the Locate Order on page 94. 10. Check that the Storage Center is archiving. See Checking the Locate Order on page 94 and

Checking the Storage Center on page 95.
11. If an error message other than No Recording Found message appears in the Player, see

Troubleshooting Player Errors on page 96.
12. Troubleshoot using the Playback Administration Service Logs. See Example of

Troubleshooting using the Logs on page 108.
13. If this is an encrypted environment, see Media Encryption Troubleshooting Flow

on page 151.
14. Submit information to NICE Customer Support:

• • • •

Dump the Client Memory. See Dumping the Client Memory on page 115. Copy the Playlist. See Copying the Playlist on page 114. Collect the Server Logs using the NICE Perform Logs Collector Tool. See Collecting Information Using the Log Collector on page 537. Collect the Client Logs using the NICE Perform Logs Collector Tool. See Collecting Information Using the Log Collector on page 537.
Chapter 3: Playback Troubleshooting Flow 84

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Playback Problem for Specific Calls

Troubleshooting Playback Problem for Specific Calls
Figure 3-4 Troubleshooting Flow for Playing Back Specific Calls

If the problem is for specific calls: 1. Set the appropriate log level:

• •

Set the Log Level in the System Administrator. See Setting the Log Level in System Administrator on page 87. Set the Log Level in the Config File. See Setting the Log Level in the .Config File on page 88.
Chapter 3: Playback Troubleshooting Flow 85

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Playback Problem for Specific Calls

Set the Player Log Level. See Setting the Player Log Levels on page 90.

2. If other users can play back the calls on the same workstation when logging in, check the user permissions in User Admininstration. 3. Is there anything unique about the calls? (for example, were they all recorded on the same

Logger, or at the same time, or are they recordings of the same agent or phone number).
4. If the problems relate to a specific remote Logger, check for Firewall or network issues, for

example DNS configuration. See High Density Logger Troubleshooting Flow on page 303.
5. If a No Recording Found Message appears for the calls in Player, check the Locate Order.

See Checking the Locate Order on page 94.
6. Check that the Storage Center is archiving. See Checking the Storage Center on page 95. 7. If an error message other than a No Recording Found message appears in the Player, see

Troubleshooting Player Errors on page 96.
8. Troubleshoot using the Playback Administration Service Logs. See Example of

Troubleshooting using the Logs on page 108.
9. If this is an encrypted environment, see Media Encryption Troubleshooting Flow

on page 151.
10. Submit information to NICE Customer Support:

• • • •

Dump the Client Memory. See Dumping the Client Memory on page 115. Copy the Playlist. See Copying the Playlist on page 114. Collect the Server Logs using the NICE Perform Logs Collector Tool. See Collecting Information Using the Log Collector on page 537. Collect the Client Logs using the NICE Perform Logs Collector Tool. See Collecting Information Using the Log Collector on page 537.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

86

Setting the Log Level in System Administrator

Setting the Log Level in System Administrator
To set the log level in System Administrator: 1. In System Administrator, verify that you are working in Technician mode: In the Organization tree, select Organization. Then mark the Technician Mode checkbox and click Save. 2. In the Organization tree, select Applications > Configurations > General. 3. Select the Logs Level tab.
Figure 3-5 Logs Level Tab

4. In the Master Level and File Level columns, note the current log level for Playback Administration and Playback Locate. 5. In the Master Level and File Level columns, change the log level for Playback Administration and Playback Locate to Debug. 6. After troubleshooting ensure that you reset these values according to the values you noted in

Step 4 on page 87.
WARNING Failure to switch the log levels back to the default values might cause system problems due to a performance overload.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

87

Setting the Log Level in the .Config File

Setting the Log Level in the .Config File
Each of the components on the Applications Server has .config files stored in the following folder:
<Application Server Installation Path>\ServerBin\.

Playback Administration Loggers

NiceApplications.Playback.Administration.Startup.exe. config LocateService PBResourceManager

IMPORTANT Backup your .config files before editing them. If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file. In any event, any change of configuration files should be done ONLY in consultation with NICE Support!!!!.

To set the log level in the .config file: 1. Navigate to <Application Server Installation Path>\ServerBin\. 2. In Notepad, open the relevant .config file. 3. Expand the <log4net> section. 4. Expand the required logger name sections (<logger name="NiceApplications.Playback">, <logger name="LocateService"> and <logger name="PBResourceManager">). 5. Note the values in both Level and Threshold. 6. Set the debug level, by changing the values on both Level and Threshold to be defined as ALL.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

88

Setting the Log Level in the .Config File

Figure 3-6

Log Levels in the .config File

7. Save and close the .config file. There is no need to restart the Applications Server. 8. After troubleshooting ensure that you reset these values according to the values you noted in

Step 5 on page 88.
WARNING Failure to switch the log levels back to the default values might cause system problems due to a performance overload.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

89

Setting the Player Log Levels

Setting the Player Log Levels
You need to set the Player log level in System Administrator and in the NiceApplications.Playback.GUI.exe.config file to enables playback logic logging. For further information, see Troubleshooting Player Errors on page 96.
To set the Player log levels: 1. To set the Player log levels in System Administrator: a.

Verify that you are working in Technician mode. Click the Log Level tab.
Figure 3-7 Log Level Tab

b. In the Organization tree, select Playback > Player Settings. c.

d. Note the current Log Level setting. e. f.

Set the Log Level to Debug. Logout and then login for the change to take effect. in Step d on page 90.

g. After troubleshooting ensure that you reset these values according to the values you noted

IMPORTANT Failure to switch the log levels back to the default values might cause system problems due to a performance overload.

To create a log file of the playback logic errors: 1. To create a log file of the playback logic errors in the NiceApplications.Playback.GUI.exe.config config file: a.

Set the EnablePBLogicLog value to true:
<add key="EnablePBLogicLog" value="true" />

b. In the Registry Editor, navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Nice Systems\PBLogic\Log, and note the values of the MMLClient_LogLevel and the PBLogic_LogLevel. c.

Set the MMLClient_LogLevel and PBLogic_LogLevel to 0 for maximum details.
Chapter 3: Playback Troubleshooting Flow

90

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Setting the Player Log Levels

Figure 3-8

Registry Key

d. After troubleshooting ensure that you reset these values.

• •

Set the EnablePBLogicLog value to false. In the Registry Editor, reset the MMLClient_LogLevel and PBLogic_LogLevel values according to the values you noted in Step b on page 90.

IMPORTANT Failure to switch the log levels back to the default values might cause system problems due to a performance overload.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

91

Checking the Workstation Ports

Checking the Workstation Ports
The settings apply to all Workstations!
To check the Workstation ports: 1. In System Administrator, navigate to Applications > Configurations > General. 2. Expand the Workstation Settings area. Then select as follows:

• •

Random Ports - all ports on each workstation must be open. Specific Ports Range - a range between 1024 and 5000 (the minimum range is 11 ports.

For example, 2001 - 2011 inclusive).

Checking the File Versions
On the Applications Server, check for a mismatch between the NICE updates in the ServerBin folder (contains binaries and DLL files for server-based applications) and the ClientBin folder (contains the binaries and DLL files needed by clients).
To check the file versions: 1. Browse to D:\Program Files\Nice Systems\Applications\ServerBin. 2. Check the NiceApplications.Playback.xxx.dll version numbers: a.

In Explorer, right-click on Name, and select More.
Figure 3-9 Choose Details Window

b. Select File Version and check OK. c.

Check the file version numbers against the D:\Program Files\Nice
Systems\Applications\ClientBin.
Chapter 3: Playback Troubleshooting Flow 92

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Checking the File Versions

3. If the NICE Playback Telephony Services service or the NICE Playback Streaming

service is installed, ensure that all the required NICE updates are installed. Check the following version numbers:

• •

NiceApplications.Playback.Streaming.xxx.dll NiceApplications.Playback.Telephony Services.xxx.dll

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

93

Checking the Locate Order

Checking the Locate Order
The Locate Order is the order in which all storage media are searched when a query is run. If your site is installed with a Storage Center, you can select to search the Storage Center before the Loggers when running a query. If most of the interactions in your query results are stored on a Storage Center, this will produce faster results. Locate Order can also be defined per Location. If a user is defined on a location which has a Locate Order defined, the locate order of the Location takes precedence over the locate order defined for the site and is used. For more information on the logs, Example of Troubleshooting using the Logs on page 108.
Locate Order No Storage Center Before Source Logger Description Do not search the Storage Center (there is no Storage Center at the site). First, search the Storage Center and then search the Loggers. If most of the interactions in your query results are stored on a Storage Center, this will produce faster results. First, search the Logger where the interaction was recorded and then search the Storage Center. Search all the Voice Loggers and then search the Storage Center. Search only the Storage Center and not the Loggers. You define which Loggers to search, and in what order to search them. The source and spare Logger will always be checked first, prior to this customized order. You will not be able to playback interactions that are not yet archived to the Storage Center. If a Logger is missing from the customized order you will be unable to playback those interactions. Troubleshooting If there is a Storage Center and this is incorrectly configured the calls will not be played back. You will not be able to playback interactions that are not yet archived to the Storage Center if the Logger that they are stored on is down.

After Source Logger

You will not be able to playback interactions that are not yet archived to the Storage Center if the Logger that they are stored on is down.

After All Loggers Storage Center Only Customized Order

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

94

Checking the Storage Center

Checking the Storage Center
Check if the Storage Center is archiving the calls.
To check the Storage Center: 1. Check the directories on the Storage Center to check if the calls are archiving there. 2. Check the archiving to the Storage Center. a.

In Reporter, import the Health Checks Report template. Check the Archiving Coverage per Storage Center View.
Archiving Coverage Report

b. Run the Archiving Coverage Report. c.

Figure 3-10

3. To further troubleshoot Storage Center problems, see Storage Center Troubleshooting Flow

on page 513.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

95

Troubleshooting Player Errors

Troubleshooting Player Errors
The player errors are recorded in the Player log files, *.txt, in %APP_DATA%\NicePlayer\Release3. There are various types of errors that may occur in the Player:

• • •

Playback errors Logger errors Storage Center errors

Playback Logic Errors
The following playback logic errors may appear in the Player Logs. These errors have the MMLAPIBase.PbLogicLog prefix.
Error OK Fail No. 1 -1 Description OK. General failure. Action Required None Collect the Player logs and submit them to NICE Customer Support. Refer to Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113. Collect the Player logs and submit them to NICE Customer Support. Refer to Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113. Collect the Player logs and submit them to NICE Customer Support. Refer to Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113. Check the connectivity to the Storage Center or the NiceScreen Logger. Troubleshoot the Storage Center or NiceScreen Logger. See the Storage Center Troubleshooting Flow on page 513 or the NiceScreen Troubleshooting Flow on page 489.

Illegal parameter

-3

One or more of the input parameters has an invalid value.

Improper mode

-4

The playback logic is not in a proper mode to perform the requested operation.

Connection failed

-5

Failed to connect to (or disconnect from) the NiceScreen Logger or the Storage Center. NiceScreen Loggers are sometimes referred to as Multi Media Loggers (MML).

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

96

Logger Errors

Error End of requested segment

No. -7

Description No more buffers are available for the requested segments.

Action Required Collect the Player logs and submit them to NICE Customer Support. See the Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113. Collect the Player logs and submit them to NICE Customer Support. See Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113. Collect the Player logs and submit them to NICE Customer Support. See Collecting the Client Logs using NICE Perform Logs Collector Tool on page 113.

No packets yet (for monitoring)

-8

The segment exists but its next requested buffer was not found.

Segment not found

-9

The requested segment was not found in the NiceScreen Logger or the Storage Center.

Logger Errors
The following Logger errors may appear in the Player Logs.
Error No. -9 -1

Description Handler is invalid. Communication failure.

Action Required

Failed to connect to the Logger. To check this, ping the Logger from the client, and check any Firewall settings between the Logger and client.

0 1 11 12 57 200 201

Operation successful. Command failed. Invalid parameter, for example, invalid tag. Token does not exist. Not enough resources for playback. Audio Over LAN & Import: Invalid token Audio Over LAN & Import: DB error or record aborted.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

97

Logger Errors

Error No. 203

Description Audio Over LAN & Import: No free virtual output channel.

Action Required The Logger has no free connections. Each media uses a connection to the Logger. To check this, wait a while and then retry playing back the interaction.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

98

Storage Center Errors

Storage Center Errors
The following Storage Center errors may appear in the Player Logs.
Error No. 300007

Description The Storage Center failed to connect to a backup device on a Logger. This should not be considered an error if there are no backup devices in the site. The Storage Center did not archive or it had an error in the locate flow.

Action Required Check the Logger where the backup device is located.

300008

See:

• • • •

Checking the Locate Order on page 94 Checking the Storage Center on page 95 Setting the Player Log Levels on page 90 Storage Center Troubleshooting Flow on page 513.

Checking the Playback Logs
The server log files are found in the <Application Server Installation Path>\ServerBin\ folder unless specified otherwise.
PlaybackAdministration.txt.x Playback Administration NiceApplications.PlaybackAdministration.SystemFramework.log PlaybackRM.txt.x LocateLog.txt.x Business Analyzer NBA.log. NBADefault.log.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

99

Sample Player Logs

Sample Player Logs
Sample Log
2008-07-31 13:39:15,781 DEBUG; [MMLAPIBASE.PBLOGICLOG] - DebugInfo - CNMLApi : CNMLApi : NML_ReleaseBuffer() - Released buffer context 20000004. [Issued by component] [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - CNMLApi : CNMLApi : NML_ReleaseBuffer() - Released buffer context 20000004. [Issued by component] [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : [I] GetNextPacket() - Got Packet (pContext=536870916, Type=0, Size=4096, TimeTag=29941032.3332150528, Flags=0, HeaderSize=24, DataSize=4096 [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : [I] GetNextPacket() - Got Packet (pContext=536870916, Type=0, Size=4096, TimeTag=29941032.3332150528, Flags=0, HeaderSize=24, DataSize=4096 [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : [I] GetNextPacket() - Got Packet (PacketGUID=0-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0, MACSize=0, PacketMAC=) [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : [I] GetNextPacket() - Got Packet (PacketGUID=0-0-0-0-0-0-0-0-0-0-0-0-0-0-0-0, MACSize=0, PacketMAC=) [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : <<GetNextPacket() [Duration = 16 msec] [ThreadId 12144, ErrorCode 0] 2008-07-31 13:39:15,781 DEBUG; [MMLAPIBase.PbLogicLog] - DebugInfo - LogicImp : LogicImp : <<GetNextPacket() [Duration = 16 msec] [ThreadId 12144, ErrorCode 0].

Storage Center Disconnection Log
2008-09-25 12:33:05,250 INFO ; [ExpandView.OnTmrScaleTick] - Position: 32515 2008-09-25 12:33:05,500 INFO ; [ExpandView.OnTmrScaleTick] - Position: 32765 2008-09-25 12:33:05,750 INFO ; [ExpandView.OnTmrScaleTick] - Position: 33015 2008-09-25 12:33:05,906 ERROR; [MMLAPIBase.GetPacket] m_pbLogic.GetNextPacket failed res = CONNECTIONFAILED 2008-09-25 12:33:05,921 INFO ; [VoiceMMLServer.GetPacket] - screen GetNextPacket failed at NiceApplications.Playback.MediaSourceAPI.MMLAPIBase.GetPacket(Boolean& lastPacket, Byte[]& data, tSecuredPacketParams& packetParams) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean& lastPacket, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize)
Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 100

Sample Player Logs

at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetPacket(Boolean& lastPacket, Boolean& active, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() 2008-09-25 12:33:05,921 ERROR; [AudioBufferBase.FetchingThreadFunction] screen GetNextPacket failed at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean& fetchDone) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop() at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction() 2008-09-25 12:33:05,921 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed to Seek 2008-09-25 12:33:05,921 ERROR; [MMLAPIBase.GetPacket] m_pbLogic.GetNextPacket failed res = CONNECTIONFAILED 2008-09-25 12:33:05,921 INFO ; [VoiceMMLServer.GetPacket] - screen GetNextPacket failed at NiceApplications.Playback.MediaSourceAPI.MMLAPIBase.GetPacket(Boolean& lastPacket, Byte[]& data, tSecuredPacketParams& packetParams) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean& lastPacket, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetPacket(Boolean& lastPacket, Boolean& active, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() 2008-09-25 12:33:05,921 ERROR; [AudioBufferBase.FetchingThreadFunction] screen GetNextPacket failed at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean& fetchDone) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop() at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction()

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

101

Sample Player Logs

2008-09-25 12:33:05,921 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed to Seek 2008-09-25 12:33:05,921 ERROR; [CommonViewManager.ErrorOccurred] ErrorOccurred, description: SCREEN_SERVER_SCREEN_FAILED_TO_GET_NEXT_PACKET 2008-09-25 12:33:05,921 ERROR; [CommonViewManager.ErrorOccurred] ErrorOccurred, description: SCREEN_SERVER_SCREEN_FAILED_TO_GET_NEXT_PACKET 2008-09-25 12:33:05,921 DEBUG; [CommonViewManager.DisableCloseForms] >>DisableCloseForms

Storage Center with Playback Logic Log
2008-09-25 12:58:41,218 WARN ; [MMLAPIBASE.PBLOGICLOG] - Warning MMLPoint::SendBlockingMsgToChannel : timeout expired. ChannelId(1), Opcode(203) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : timeout expired. ChannelId(1), Opcode(203) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : error detected while waiting. ChannelId(1), Opcode(203), lastError(553648143) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : error detected while waiting. ChannelId(1), Opcode(203), lastError(553648143) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError PlaybackPoint::GetBuffer : SendSyncCommandToChannel failed. Channel id(1) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError PlaybackPoint::GetBuffer : SendSyncCommandToChannel failed. Channel id(1) [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError - CNMLApi : CNMLApi : NML_GetBuffer() - NMLAPI_PB_GetBuffer() returned 0,0.GetLastError returned 0 [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 FATAL; [MMLAPIBase.PbLogicLog] FatalError - CNMLApi : CNMLApi : NML_GetBuffer() - NMLAPI_PB_GetBuffer() returned 0,0.GetLastError returned 0 [ThreadId 5464, ErrorCode 0] 2008-09-25 12:58:41,218 ERROR; [MMLAPIBase.PbLogicLog] Error - CNMLApi : CNMLApi : ConnectionFailureCallback() - Disconnection occured for ClientHandle a86860. [ThreadId 5964, ErrorCode 0] 2008-09-25 12:58:41,218 ERROR; [MMLAPIBase.GetPacket] m_pbLogic.GetNextPacket failed res = ConnectionFailed

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

102

Sample Player Logs

2008-09-25 12:58:41,218 ERROR; [MMLAPIBase.PbLogicLog] Error - CNMLApi : CNMLApi : ConnectionFailureCallback() - Disconnection occured for ClientHandle a86860. [ThreadId 5964, ErrorCode 0] 2008-09-25 12:58:41,218 INFO ; [VoiceMMLServer.GetPacket] - screen GetNextPacket failed at NiceApplications.Playback.MediaSourceAPI.MMLAPIBase.GetPacket(Boolean& lastPacket, Byte[]& data, tSecuredPacketParams& packetParams) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean& lastPacket, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetPacket(Boolean& lastPacket, Boolean& active, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() 2008-09-25 12:58:41,218 ERROR; [AudioBufferBase.FetchingThreadFunction] screen GetNextPacket failed at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean& fetchDone) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop() at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction() 2008-09-25 12:58:41,218 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed to Seek 2008-09-25 12:58:41,218 ERROR; [CommonViewManager.ErrorOccurred] ErrorOccurred, description: SCREEN_SERVER_SCREEN_FAILED_TO_GET_NEXT_PACKET 2008-09-25 12:58:41,234 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : timeout expired. ChannelId(2), Opcode(203) [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : timeout expired. ChannelId(2), Opcode(203) [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : error detected while waiting. ChannelId(2), Opcode(203), lastError(553648143) [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 WARN ; [MMLAPIBase.PbLogicLog] - Warning MMLPoint::SendBlockingMsgToChannel : error detected while waiting. ChannelId(2), Opcode(203), lastError(553648143) [ThreadId 4100, ErrorCode 0]

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

103

Sample Player Logs

2008-09-25 12:58:41,234 FATAL; [MMLAPIBase.PbLogicLog] FatalError PlaybackPoint::GetBuffer : SendSyncCommandToChannel failed. Channel id(2) [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 FATAL; [MMLAPIBase.PbLogicLog] FatalError PlaybackPoint::GetBuffer : SendSyncCommandToChannel failed. Channel id(2) [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 FATAL; [MMLAPIBase.PbLogicLog] FatalError - CNMLApi : CNMLApi : NML_GetBuffer() - NMLAPI_PB_GetBuffer() returned 0,0.GetLastError returned 0 [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 FATAL; [MMLAPIBase.PbLogicLog] FatalError - CNMLApi : CNMLApi : NML_GetBuffer() - NMLAPI_PB_GetBuffer() returned 0,0.GetLastError returned 0 [ThreadId 4100, ErrorCode 0] 2008-09-25 12:58:41,234 ERROR; [MMLAPIBase.GetPacket] m_pbLogic.GetNextPacket failed res = ConnectionFailed 2008-09-25 12:58:41,234 INFO ; [VoiceMMLServer.GetPacket] - screen GetNextPacket failed at NiceApplications.Playback.MediaSourceAPI.MMLAPIBase.GetPacket(Boolean& lastPacket, Byte[]& data, tSecuredPacketParams& packetParams) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetActivePacket(Boolean& lastPacket, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.MediaSourceAPI.VoiceAPI.GetPacket(Boolean& lastPacket, Boolean& active, Byte[]& audioBuffer, DateTime& startTime, DateTime& stopTime, CompressionType& compression, Guid& guid, UInt32& actualDataPosition, UInt32& actualDataSize) at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() 2008-09-25 12:58:41,234 ERROR; [AudioBufferBase.FetchingThreadFunction] screen GetNextPacket failed at NiceApplications.Playback.Streaming.Nice.VoiceMMLServer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean& fetchDone) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop() at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction() 2008-09-25 12:58:41,234 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed to Seek 2008-09-25 12:58:41,234 ERROR; [CommonViewManager.ErrorOccurred] ErrorOccurred, description: SCREEN_SERVER_SCREEN_FAILED_TO_GET_NEXT_PACKET

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

104

Sample Player Logs

Logger Disconnection Log
2008-09-25 12:43:38,828 INFO ; [RenderVoiceMediaBase.WriteAudioToDSBuffer] - MediaId 3 writeSize 16000 clentBufferLength 16000 2008-09-25 12:43:38,843 ERROR; [LoggerPBAPI880.GetNextPacketFromLogger] NC_StreamFetchData failed res = -16 2008-09-25 12:43:38,843 ERROR; [VoiceServer.GetPacket] GetPacket error VOICE_SERVER_CONNECTION_FAILURE 2008-09-25 12:43:38,843 ERROR; [AudioBufferBase.FetchingThreadFunction] FETCH DATA FAILED at NiceApplications.Playback.Streaming.Nice.VoiceServer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.GetPacket(Boolean& fetchDone) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Pop() at NiceApplications.Playback.MediaServices.Logic.AudioBuffer.GetPacket() at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.FetchingThreadFunction() 2008-09-25 12:43:38,843 INFO ; [AudioBufferBase.FetchingThreadFunction] - m_state changed to Seek 2008-09-25 12:43:38,843 INFO ; [RenderVoiceMediaBase.OnAudioBufferError] - Got buffering event while playing propogating event 2008-09-25 12:43:38,859 INFO ; [PlaybackController.OnBufferingEvent] - Controller got Buffering event while playing or seeking 44 2008-09-25 12:43:38,859 INFO ; [PlaybackController.PlayFrom] - PlayFrom 9250 2008-09-25 12:43:38,859 INFO ; [PlaybackController.Seek] - Seek 9250 2008-09-25 12:43:38,890 INFO ; [RenderVoiceMediaBase.GetFreeSpaceSize] - playCursor 151948, LastPlayCursorPosition 135118 2008-09-25 12:43:38,890 INFO ; [RenderVoiceMediaBase.WriteAudioToDSBuffer] - MediaId 1 writeSize 16000 clentBufferLength 16000 2008-09-25 12:43:38,921 INFO ; [PBViewManager.OnBuffring] - [WMS] Buffering Started 2008-09-25 12:43:38,921 INFO ; [ExpandView.OnBuffring] - *** Not Gui thread using this.Invoke *** 2008-09-25 12:43:38,921 INFO ; [AudioBufferBase.Seek] - Seek m_seekPosition=9250, m_state=Working 2008-09-25 12:43:38,921 INFO ; [CompressedInBuffer.Seek] - Seek : 9250 2008-09-25 12:43:38,921 DEBUG; [VoiceServer.InternalJumpToPos] - JumpToPos Token 191 startTime 25/09/2008 09:45:13 404 StopTime 25/09/2008 09:47:04 403

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

105

Sample Player Logs

2008-09-25 12:43:38,937 INFO ; [AudioBuffer.GetPacket] - Poped:Packet: Media 1, Time 25/09/2008 09:45:22.120-25/09/2008 09:45:26.210, Position 8716-12806 (=4090ms), Data 4090by (=4090ms), Active 2008-09-25 12:43:38,937 ERROR; [LoggerPBAPI880.StartPlayback] AUD_AudioOverLAN_Ex failed res = -1 2008-09-25 12:43:38,937 INFO ; [VoiceServer.InternalJumpToPos] - AUD_AudioOverLAN_Ex failed at NiceApplications.Playback.MediaSourceAPI.LoggerPBAPI880.StartPlayback(Int32 token, DateTime startTime, DateTime endTime) at NiceApplications.Playback.Streaming.Nice.VoiceServer.InternalJumpToPos(DateTime startTime) 2008-09-25 12:43:38,937 ERROR; [VoiceServer.InternalJumpToPos] JumpToPos error VOICE_SERVER_START_OVER_LAN_FAILED 2008-09-25 12:43:38,937 DEBUG; [AudioProcessing.DecodePacket] - DecodePacket, startPos=9250, endPos=110999, packetStartPos=8716, packetEndPos=12806 2008-09-25 12:43:38,937 INFO ; [RenderVoiceMediaBase.Seek] - AUD_AudioOverLAN_Ex failed at NiceApplications.Playback.Streaming.Nice.VoiceServer.InternalJumpToPos(DateTime startTime) at NiceApplications.Playback.Streaming.Nice.MediaDataServerBase.JumpToPos(Int64 ticks) at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.MediaServerSeek(Int64 position) at NiceApplications.Playback.MediaServices.Logic.AudioFetcher.StartFetch(Int64 position) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.SetPosition(Int64 position, Boolean& isWorking) at NiceApplications.Playback.MediaServices.Logic.CompressedInBuffer.Seek(Int64 position) at NiceApplications.Playback.MediaServices.Logic.CompressedBuffer.Seek(Int64 position) at NiceApplications.Playback.MediaServices.Logic.AudioBufferBase.Seek(Int64 position) at NiceApplications.Playback.MediaServices.Logic.RenderVoiceMediaBase.Seek(Int64 position) 2008-09-25 12:43:38,937 INFO ; [OutputDS.RenderVoiceSeekCallback] RenderVoiceSeekCallback stat = VOICE_SERVER_START_OVER_LAN_FAILED 2008-09-25 12:43:38,937 INFO ; [RenderVoiceMediaBase.WriteAudioToDSBuffer] - MediaId 1 writeSize 16000 clentBufferLength 16000 2008-09-25 12:43:38,953 INFO ; [AudioBuffer.GetPacket] - Poped:Packet: Media 1, Time 25/09/2008 09:45:26.210-25/09/2008 09:45:30.300, Position 12806-16896 (=4090ms), Data 4090by (=4090ms), Active

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

106

Sample Player Logs

Checking the NiceScreen Logs
Settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration file located in the NiceScreen Logger installation directory. LogFile_Path indicates the location of the NiceScreen log files. The logs are located in the installation folder, by default in:
C:\Programs Files\Nice Systems\Multimedia Logger. To check the NiceScreen Logs:

Review the NiceScreen Logger logs (logfileXX.log).

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

107

Example of Troubleshooting using the Logs

Example of Troubleshooting using the Logs
This section includes sample logs useful for troubleshooting playing back an interaction. For further information on the playback process, see Understanding the Playback Flow on page 79.
To Troubleshoot using the Logs: 1. In PlaybackAdministration.txt (see PlaybackAdministration.txt on page 108), search for: a. Start PreparePlayListForInteractions: This indicates the start of the playback process.

b. Locate Results: Check if the locate process succeeded. In this example two parts of the

call are located.
c. RecordingId:XXX: This indicates the recording ID for each part of the call. d. Build media per participant: This indicates the start of the media building process (per

media).
e. New Voice Media start or New Screen Media start: This indicates that the interaction

has been located and that the media has been built. If the locate succeeded, but the media is missing, it could indicate a timing problem on the client.
2. If the locate failed, open LocateLog.txt (see LocateLog.txt on page 110) and search for: a. AddRequest … recording Id: XXX: This indicates that a locate request for a recording

was initiated.
b. StorageCenterLocator.LocateRecordingCallback … Operation Result = 0: A zero

(0) indicates that the recording was located on the Storage Center. A value other than zero (0) indicates that the part of the call is missing from the Storage Center.
c. with result Found: This indicates that the locate process was successful. d. with result NotFound: This indicates that the locate process was not successful. This

may indicate a time difference between the Interactions Center and Logger, or the Interactions Center and Applications Server. Refer to TN0752: Windows Network Time Protocol Configuration. This note provides procedures for configuring the Windows Network Time Protocol on NICE servers.
e.

If there are Storage Center errors, see the Storage Center Logs to establish the reason for the error. Review the Storage Center Logs. By default, the log files are stored in the same directory where the Storage Center is running. The default log file names are:

• •

StorageCenterLog.log StorageCenterLog1.log through StorageCenterLog100.log

PlaybackAdministration.txt
2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Start PreparePlayListForInteractions 2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Create play list items
Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 108

Example of Troubleshooting using the Logs

2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Bring segments from db for requested calls 2008-08-24 09:59:04,479 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - query url: zandberg:62074 2008-08-24 09:59:04,620 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Build stages for interactions using Analyzer class 2008-08-24 09:59:04,620 DEBUG; [PlayListManager.BuildPlayItemsForInteractions] - : Create play media 2008-08-24 09:59:04,620 DEBUG; [PlayListManager.CreatePlayMedia] - : prepare locate requests 2008-08-24 09:59:04,620 DEBUG; [PlayListManager.CreatePlayMedia] - Add locate requests for call 0 2008-08-24 09:59:04,620 INFO ; [Participant.BuildRecordings] - Added another recordings: 417 for the participant ID 11 2008-08-24 09:59:04,635 DEBUG; [PBAdminServer.GetServerIdOfLogger] GetServerIdOfLogger took:00:00:00 2008-08-24 09:59:04,635 INFO ; [Participant.BuildRecordings] - Added another recordings: 416 for the participant ID 13 2008-08-24 09:59:04,651 DEBUG; [PBAdminServer.GetServerIdOfLogger] GetServerIdOfLogger took:00:00:00.0156250 2008-08-24 09:59:04,651 DEBUG; [PlayListManager.CreatePlayMedia] - : run Locate 2008-08-24 09:59:04,651 DEBUG; [PBAdminServer.GetLoggersOnSite] - GetLoggersOnSite took: 00:00:00 2008-08-24 09:59:04,651 DEBUG; [PBAdminServer.GetLoggersOnSite] - GetLoggersOnSite took: 00:00:00 2008-08-24 09:59:04,667 DEBUG; [PBAdminServer.GetLoggersOnSite] - GetLoggersOnSite took: 00:00:00 2008-08-24 09:59:04,667 DEBUG; [PBAdminServer.GetLoggersOnSite] - GetLoggersOnSite took: 00:00:00 2008-08-24 09:59:04,698 DEBUG; [LocateManager.LocateFinished] - Locate callback recieved 0 2008-08-24 09:59:04,698 INFO ; [LocateManager.LocateFinished] - Locate Callback Request 0 Times: 633518804008930000, 633518804238930000 2008-08-24 09:59:04,729 DEBUG; [LocateManager.LocateFinished] - Locate callback recieved 1 2008-08-24 09:59:04,729 INFO ; [LocateManager.LocateFinished] - Locate Callback Request 0 Times: 633518804008930000, 633518804238930000 2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - Locate Request took: 00:00:00.0781250

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

109

Example of Troubleshooting using the Logs

2008-08-24 09:59:04,729 DEBUG; [PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] :LOCATE RESULTS 2008-08-24 09:59:04,729 DEBUG; [PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] - LOGGERID :1722315 CHANNELID:49 TOKEN:0 OFFLINEMEDIANUMBER:0 DEVICENUMBER:0 ISPARTIAL:FALSE RECORDINGS: INTERACTIONID:326924 RECORDINGID:417 SERVERID:30 SITEID:1/N 2008-08-24 09:59:04,729 DEBUG; [PLAYLISTMANAGER.WRITELOCATERESULTSTOTRACE] LOGGERID :1722315 CHANNELID:19 TOKEN:0 OFFLINEMEDIANUMBER:0 DEVICENUMBER:0 ISPARTIAL:FALSE RECORDINGS: INTERACTIONID:326924 RECORDINGID:416 SERVERID:30 SITEID:1/N

2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - : BUILD MEDIA PER
PARTICIPANT

2008-08-24 09:59:04,729 INFO ; [PlayListManager.BuildMediaForParticipant] - Building Media: Stage Times : 7/17/2008 8:33:20 AM, 7/17/2008 8:33:43 AM Locate Times: 633518804008930000,633518804238930000 2008-08-24 09:59:04,729 DEBUG; [PlayListManager.BuildMediaForParticipant] - : NEW VOICE
MEDIA START

2008-08-24 09:59:04,729 INFO ; [PlayListManager.BuildMediaForParticipant] - Building Media: Stage Times : 7/17/2008 8:33:20 AM, 7/17/2008 8:33:43 AM Locate Times: 633518804008930000,633518804238930000 2008-08-24 09:59:04,729 DEBUG; [PlayListManager.BuildMediaForParticipant] - : NEW VOICE
MEDIA START

2008-08-24 09:59:04,729 DEBUG; [PlayListManager.CreatePlayMedia] - : Finish CreatePlayMedia 2008-08-24 09:59:04,729 DEBUG; [PlayListManager.PreparePlayListForInteractions] - : Build xml document 2008-08-24 09:59:04,729 DEBUG; [PlayListManager.PreparePlayListForInteractions] - : Return PlayList

LocateLog.txt
2008-08-24 09:59:04,651 [20] DEBUG; [LocateServiceServer.AddRequest] - ADDREQUEST: Request id:23 LocateOrder = Custom start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli: 893 for RECORDING ID: 417 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,651 [20] DEBUG; [LocateServiceServer.EnterCheckLoggerState] - The request 23 for recording Id: 417 interactionId: 326924 serverId:30 siteId: 1 moved to CheckLogger state 2008-08-24 09:59:04,667 [20] DEBUG; [LocateServiceServer.AddRequest] - ADDREQUEST: Request id:24 LocateOrder = Custom start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli: 893 for RECORDING ID: 416 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,667 [20] DEBUG; [LocateServiceServer.EnterCheckLoggerState] - The request 24 for recording Id: 416 interactionId: 326924 serverId:30 siteId: 1 moved to CheckLogger state
Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 110

Example of Troubleshooting using the Logs

2008-08-24 09:59:04,667 [STP SmartThreadPool Thread #31] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate calls Storage Center: zandberg:30003 recordings: Id: 417 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,667 [STP SmartThreadPool Thread #31] INFO ; [StorageCenterLocator.OriginalLocate] - Call LocateRecording for Storage Center: zandberg:30003 recordings: Id: 417 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,667 [STP SmartThreadPool Thread #32] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate calls Storage Center: zandberg:30003 recordings: Id: 416 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,698 [14] INFO ;
[STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - GOT LOCATE CALLBACK FOR RECORDING 1;30;326924;417 OPERATION R ESULT = 0

2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #31] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate for recording 1;30;326924;417 returned with 1 results 2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #31] DEBUG; [LocateServiceServer.HandleState] - The request 23 for recording Id: 417 interactionId: 326924 serverId:30 siteId: 1 finished state CheckLogger with result Found 2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #31] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - FinishLocate for recording Id: 417 interactionId: 326924 serverId:30 siteId: 1 Results: Request id:23 results number: 1 results: source: SC start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli: 893 2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #31] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - Locate callback called with times: 633518804008930000, 633518804238930000 2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #31] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - Finish request Avail: 499 Active999 2008-08-24 09:59:04,698 [STP SmartThreadPool Thread #32] INFO ; [StorageCenterLocator.OriginalLocate] - Call LocateRecording for Storage Center: zandberg:30003 recordings: Id: 416 interactionId: 326924 serverId:30 siteId: 1 2008-08-24 09:59:04,714 [14] INFO ;
[STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - GOT LOCATE CALLBACK FOR RECORDING 1;30;326924;416 OPERATION R ESULT = 0

2008-08-24 09:59:04,729 [STP SmartThreadPool Thread #32] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate for recording 1;30;326924;416 returned with 1 results 2008-08-24 09:59:04,729 [STP SmartThreadPool Thread #32] DEBUG; [LocateServiceServer.HandleState] - The request 24 for recording Id: 416 interactionId: 326924 serverId:30 siteId: 1 finished state CheckLogger WITH RESULT FOUND

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

111

Example of Troubleshooting using the Logs

2008-08-24 09:59:04,729 [STP SmartThreadPool Thread #32] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - FinishLocate for recording Id: 416 interactionId: 326924 serverId:30 siteId: 1 Results: Request id:24 results number: 1 results: source: SC start time: 7/17/2008 8:33:20 AM Milli: 893 stop time: 7/17/2008 8:33:43 AM Milli: 893 2008-08-24 09:59:04,729 [STP SmartThreadPool Thread #32] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - Locate callback called with times: 633518804008930000, 633518804238930000 2008-08-24 09:59:04,729 [STP SmartThreadPool Thread #32] DEBUG; [LocateServiceServer.SendLocateFinishedCallback] - Finish request Avail: 499 Active999 2008-08-20 19:02:56,620 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] - Got locate callback for recording 1;26;239481090;4323375 Operation Result = 300008
2008-08-20 19:02:56,620 [8860] INFO ; [STORAGECENTERLOCATOR.LOCATERECORDINGCALLBACK] - ERROR WHEN LOCATE RECORDING 1;26;239481090;4323375 OPERATION RESULT = 300008

2008-08-20 19:02:56,620 [STP SmartThreadPool Thread #7] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate for recording 1;26;239481090;4323375 returned with 0 results 2008-08-20 19:02:56,620 [STP SmartThreadPool Thread #7] DEBUG; [LocateServiceServer.HandleState] - The request 7 for recording Id: 4323375 interactionId: 239481090 serverId:26 siteId: 1 finished state StorageCenterLocate with result NotFound 2008-08-20 19:05:52,077 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] - Got locate callback for recording 1;26;253423018;4569139 Operation Result = 300007 2008-08-20 19:05:52,077 [8860] INFO ; [StorageCenterLocator.LocateRecordingCallback] Error when locate recording 1;26;253423018;4569139 Operation Result = 300007 2008-08-20 19:05:52,077 [STP SmartThreadPool Thread #86] DEBUG; [StorageCenterLocator.OriginalLocate] - Locate for recording 1;26;253423018;4569139 returned with 0 results 2008-08-20 19:05:52,077 [STP SmartThreadPool Thread #86] DEBUG; [LocateServiceServer.HandleState] - The request 70 for recording Id: 4569139 interactionId: 253423018 serverId:26 siteId: 1 finished state StorageCenterLocate with result NotFound

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

112

Collecting the Server Logs using the NICE Perform Logs Collector Tool

Collecting the Server Logs using the NICE Perform Logs Collector Tool
The Log Collector Tool collects all relevant playback information at one click of a button. This information includes all configuration parameters and all playback related log files. This information can be saved and used when opening or escalating cases. For more information, see Collecting Information Using the Log Collector on page 537.

Collecting the Client Logs using NICE Perform Logs Collector Tool
Use the NICE Perform Logs Collector to collect the Player and ScreenAgent Log files. For more information, see Collecting Information Using the Log Collector on page 537.
Player:

All *.txt; *.comments; *config; *.xml files from %APP_DATA%\NicePlayer\Release3 directory. All *.* files from directory specified by the Common AppData registry key.
NOTE: Collecting all the necessary information can speed up resolution of a support case.

ScreenAgent:

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

113

Copying the Playlist

Copying the Playlist
The playlist is built each time an interaction is played, and the previous playlist is overwritten. The playlist.xml comprises several elements.

Element Item Stage Time Participant

Description Part of an interaction. There maybe multiple items in an interaction. Typically an item comprises one stage. An additional stage may be added by, for example, a conference call. The Start and End time of the stage (Year, Month, Day Hour, Minute, Second, Millisecond). A participant in the interaction stage. There will usually be multiple participants. This includes information:

• •
ContactKey

Media information including the media type (voice or screen), the media original location (Logger ID and Channel ID) ,and the Logger token. Participant information, including Agent ID, User ID, and Device ID. No participant information will appear for customers.

Included only if this is a complete call.

To copy the playlist: 1. If you have a problem playing back an interaction, copy the following file immediately and

submit it to NICE Customer Support:
<Installation Folder>\NICE Player\temp\Playlist.xml 2. For further information on troubleshooting using the playlist, see Troubleshooting Player

Errors on page 96.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

114

Dumping the Client Memory

Dumping the Client Memory
If requested to by NICE Customer Support, you may need to dump the client memory. Do this using ADPlus, a Windows debugging tool that starts the CDB debugger and executes a set of debugger commands to gather information. For more information regarding ADPlus, see http://support.microsoft.com/kb/286350.
To Dump the Client Memory: 1. Copy the debug tools directory onto the client machine (C:\Program Files\Debugging Tools for Windows\). Alternatively, connect a USB Flash drive to the problematic machine

and run the dump command from the USB Flash drive.
2. Identify the process ID: a.

Run Internet Explorer. You should only have one Internet Explorer window open. From View > Select Columns. Note the Internet Explorer Process ID.

b. In Windows Task Manager, click the Processes tab. c.

d. Select PID (Process Identifier) and click OK. e.

3. From the Debugging Tools for Windows directory or USB Flash drive, run the following

command:
adplus -hang -p XXXX -quiet

• • •

The dump is created in the Debugging Tools for Windows directory. To specify an output path use the -o option. If the process throws a fatal exception and then terminates, replace the -hang option with the -crash option. This takes the dump before the process terminates. If any question dialogs appear during the dump process, click OK.

4. The cdb.exe command window appears, and the dump process starts. 5. The command window closes when the dump is finished and the results are saved in a file with the format <Mode>_Date_<Date> _Time_<Time>, for example: Hang_Mode__Date_12-27-2005__Time_17-37-1919. 6. Verify that this is the date and time that you took the dump, ZIP the entire directory (take into

account that the file can be large), and mail or FTP it to NICE Customer Support.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

115

Understanding Playback Retrieval

Understanding Playback Retrieval
When you cannot play back a recording from the Business Analyzer application and receive the error There are no recordings to retrieve, you need to verify that the Media Library and Storage Center are installed. If they are not installed, you can retrieve the entire storage media that includes the specific recording from the Backup Manager.

Other Troubleshooting
A No recording was found message could be related to activation of CTI Analysis in the integration. Check the Playback Administration service log file as well as the Business Analyzer service log file.

Chapter 3: Playback Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

116

4
Archiving Troubleshooting Flow
This chapter describes common troubleshooting issues for the various archiving methods used with NICE Perform Release 3.1.

Contents
General Archiving Troubleshooting Flow .................................................................. 118 Tape/DVD Archiving Troubleshooting ........................................................................ 119 Media Library Troubleshooting ...................................................................................121 Backup Server Device Detection Troubleshooting ...................................................123 Other Backup Server Troubleshooting.......................................................................125 Storage Center Troubleshooting.................................................................................126 Setting the Log Level to DEBUG..............................................................................128 Checking for Archived Calls .....................................................................................129 Gathering Storage Center Information .....................................................................132 Checking the Storage Retention and Deletion Values .............................................133 Verifying that the Storage Center Database is Selected ..........................................133 Verifying the Used and Free Space..........................................................................133 Verifying the Storage Unit Threshold........................................................................134 Checking that the Archiving Rule Inserts New Calls ................................................135 Troubleshooting the Archiving Rule .........................................................................135 Checking the Storage Center Archiving Tables........................................................137 Checking the Storage Center Logs ..........................................................................138 Checking the Nice Storage Streaming Service Logs................................................139 Checking the Player Logs for Storage Center Errors ...............................................139 Other Troubleshooting Issues ..................................................................................140

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

117

General Archiving Troubleshooting Flow

General Archiving Troubleshooting Flow
1. Which archiving activity are you troubleshooting?

• • •

Online archiving on a Logger. Archiving on tape/DVD. Organizing the archiving media at a site with Media Library. The Media Library assigns sequential numbers to the cassettes, and the Media Library printer prints labels automatically. Archiving on Storage Center.

2. If your problem is archiving online to a Logger, see the High Density Logger

Troubleshooting Flow on page 303 and VoIP Troubleshooting Flow on page 329.
3. If your problem is archiving to tape, see Tape/DVD Archiving Troubleshooting on page 119. 4. If your problem is with the Media Library, see Media Library Troubleshooting on page 121. 5. If your problem is with the Backup Server, see Backup Server Device Detection

Troubleshooting on page 123.
6. If your problem is archiving to Storage Center, see Storage Center Troubleshooting

on page 126.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

118

Tape/DVD Archiving Troubleshooting

Tape/DVD Archiving Troubleshooting
See the Backup Configuration Guide and Archiving Devices and Media Reference Guide for further information.
To troubleshoot tape/DVD archiving: 1. View the status of the drive: a.

Log in to the NICE Perform Applications Suite.

b. From the Accessories menu, select Backup Manager. c.

Double-click the NiceLog icon

.

d. Select a Logger. Logger details (Server Name, Logger ID, Host/IP Address and Backup Mode) appear in the Logger Details area; device information (if media is loaded) appears in the Devices Information area including:

• •
e. f.

Device Assignment shows the current operation taking place: Automatic Archiving, Retrieval, Retrieve while Archiving, Manual Archiving, Erase. Device Status shows the current status: Loading, Empty, Ejecting, Retrieving, Suspended.

In the Device Information area, select a row representing a device. Click the Device Info The Device Info window appears.
Figure 4-1 Device Info Window

button on the Backup Manager toolbar.

2. Ensure that there is a media in the drive. 3. Check that the retention period of the media has not expired yet.. 4. Check that the media was closed properly and is valid for usage in append mode. 5. Ensure that the media is not full or write protected. 6. Ensure that the media version is supported. For a list of the supported media, see the Archiving

Devices and Media Reference Guide.
7. Verify that the archiving device can be viewed as a drive. 8. Verify that the correct driver is installed for the archiving device. See Archiving Devices

Driver Installation in the Archiving Devices and Media Reference Guide.
Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 119

Tape/DVD Archiving Troubleshooting

9. If necessary, use a cleaning media to clean the archiving drive. For a list of supported cleaning

media and how to clean the archiving device, see the Archiving Devices and Media Reference Guide.
WARNING If an archiving drive is accidentally cleaned with a spent cleaning cassette, the used-up cassette will slowly contaminate the drive. The contamination can spread to the archiving medium. The archiving medium can then contaminate a new archiving drive and cause it to fail. 10. Test the archiving drive using the NICE Test Kit. a.

If you do not have the NICE Test Kit you can download it from ExtraNICE > Services >
Diagnostic Utilities.

b. See the General High Density Logger Troubleshooting Flow on page 304 and the

NICE Test & Recovery Kit User Manual.
c.

Use the following tests that run under Windows:
Test/Operation Backup Sony AIT & DDS Tape Tool Test Description Checking SONY AIT IDE Backup Devices. Firmware Upgrade for SONY IDE & SCSI Backup Test DVD Creator Application Backup DVD Read & Verify Creating a Test DVD Checking DVD Drives Log File Location Root:\Programs\SONY Tape Tool for Windows\S.N.dat S.N indicates the tape serial number No log file. Root:\Programs\BurnInTest\BIT_lo g*.log

d. Use the following tests that run under DOS: Test/Operation Backup SONY AIT & DDS Test (SCSI) Test Description Checking SONY AIT & DDS SCSI Backup Devices Log File Location Root:\Programs\SONY\NTAU\*.ana Root:\Programs\SONY\Ship\*.abc

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

120

Media Library Troubleshooting

Media Library Troubleshooting
See the Backup Configuration Guide.
To troubleshoot Media Library: 1. Verify that all Media Library Loggers appear in the Media Library window as Active =1.
Figure 4-2 Media Library Window

2. If automatic archiving is not working, try to archive manually. 3. If cannot print lables because of archiving issues using the Backup Server, see Tape/DVD

Archiving Troubleshooting on page 119.
4. Check that all the required updates are installed. 5. Check if the Media Library is not printing labels for one or more of the Loggers. 6. Check the Media Library printer status and which Loggers are associated to a specific Media

Library printer:
a.

In System Administrator, in the Organization tree, select Media Library Servers and select the required Media Library Server. Check the Printer Status in the General tab. Check which Loggers are associated to the Media Library Printer.

b. Selected the required Media Library printer. c.

d. Click the Loggers tab. e.

7. Check the network connection between the Media Server and the affected Loggers. 8. Check that the printer is plugged in and online.
Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 121

Media Library Troubleshooting

9. Check the location of the printer sensors and check the label cartridge in the printer. 10. Use the test button on the printer to print a test label. 11. The following are the relevant Media Library tables:

• • • • •

Nice_admin.tblMLibLoggers nice._admin.tblSystemResources includes all defined Media Library Loggers with iLoggerId (iLoggerid must be = iResouseId from tblSystemResources) Nice_interaction.tblMLCassestes describes cassettes that Media Library created labels

for.
Nice_interaction.tblMLChannels describes for a given tape the channels and times. Nice_interaction.tblMLLog is the Media Library log.

12. In System Administrator, navigate to Applications > Configurations > Locate Settings

and check that Storage Center is configured for locating and check the Locate Order. For more information, see Checking the Locate Order on page 94.
13. Increase the reporting level for the NICE Playback Administration service, which includes the

Locate service and Storage Center service.
14. Check the Locate and Storage Center log files at the following paths:

• •
a.

Locate service log file: NiceSystem\Applications\ServerBin\Locate.log Storage Center log file: NICE Systems\Storage Center\StorageCenterLog.log Check that the StorageCenterLog.log does not contain error messages related to the Storage Center startup process.

b. Check that the Locate.log reflects the correct locate process for off-line call playback:

• • • • • •

A locate request for a specific InteractionID is received. The NICE Playback Administration service (locate) checks the source Logger and returns Result NotFound. The NICE Playback Administration service (locate) checks the rest of the Loggers and returns Result NotFound. The NICE Playback Administration service (locate) checks the Storage Center and returns Result OfflineFound. The NICE Playback Administration service (locate) activates RetrieveServiceSystem and gets a list of tape drives. If one or several interactions are not online, the Retrieval Wizard appears. See “Retrieving Recordings of Interactions” in the Business Analyzer Guide.

15. Ensure that the NICE Storage Center Service and Nice Storage Streaming Service (PlaybackServiceSC) are running (Note: A site using the Media Library must have the NICE

Storage Center to enable retrieval of recordings.)

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

122

Backup Server Device Detection Troubleshooting

Backup Server Device Detection Troubleshooting
The Backup Server uses DVD-RAM devices, USB tapes, or SCSI tapes to back up data. This section troubleshoots how to enable detection of each of these backup methods:

• • •

DVD-RAM Devices Not Detected on page 123 USB Tapes Not Detected on page 124 SCSI Tapes Not Detected on page 124

DVD-RAM Devices Not Detected
Follow the procedure below when the Backup Server does not detect DVD-RAM devices.
To troubleshoot DVD-RAM devices not detected by the Backup Server: 1. Check that the hardware connections of the DVD-RAM devices are correctly attached. 2. Navigate to Control Panel > System , and click the Hardware tab. Click the Device Manager button to open the Device Manager. Check that the DVD-RAM devices are listed. 3. Navigate to Control Panel > Add or Remove Programs and check that the WriteDVD!

application appears in the list of installed programs.
4. Restart the computer. 5. From My Computer, check that a removable device appears for each DVD-RAM device

installed on the Backup Server.
6. From the Device Manager (Control Panel > System > Hardware > Device Manager),

check that the name of each DVD drive includes a specific identifier.
Figure 4-3 DVD Drive Name with Identifier

Correct

Wrong

7. If the Logger is installed as a service: a.

From the Start menu of the Backup Server machine, select Run and enter gpedit.msc. The Group Policy window appears.
Chapter 4: Archiving Troubleshooting Flow

123

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Backup Server Device Detection Troubleshooting

b. Navigate to Windows Settings > Security Settings > Local Policies > Security Options, and check that the following Policy is disabled: Devices: Restrict CD-ROM access to locally logged on user only.

USB Tapes Not Detected
Follow the procedure below when the Backup Server does not detect USB tapes.
To troubleshoot USB tapes not detected by the Backup Server: 1. Check that certified drivers are installed for the USB tapes. 2. Additional installed backup devices prevent the detection of USB tapes. Check that no

DVD-RAM devices or SCSI/IDE tapes are connected to the Backup Server.

SCSI Tapes Not Detected
Follow the procedure below when the Backup Server does not detect SCSI tapes.
To troubleshoot SCSI tapes not detected by the Backup Server:

Installed DVD-RAM devices prevent the detection of SCSI tapes. Check that no DVD-RAM devices are connected to the Backup Server.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

124

Other Backup Server Troubleshooting

Other Backup Server Troubleshooting
• •
If an error occurs in the DVD, do not restart the Logger before ejecting the archiving media from all DVD drives in the system. Before performing a DVD device assignment change, such as from auto-archiving to retrieval or vice versa, ensure that the Logger's front panel doors are fully open because the DVD tray will automatically eject after the assignment change. Failing to do so can cause errors in the Logger. When browsing in the My Computer folder, do not browse in the removable disk drives or CD drives as this might cause an interruption in the Logger archiving operation.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

125

Storage Center Troubleshooting

Storage Center Troubleshooting
The diagram below describes the workflow for troubleshooting NICE Storage Center.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

126

Storage Center Troubleshooting

NOTE:

• •

In the Storage Center troubleshooting flow on page 126, SC refers to Storage Center, as a matter of convenience. The main steps for troubleshooting the NICE Storage Center are detailed below. The numbers ( ) in the flow on page 126 refer to these steps.

1. Set the log level to debug. See Setting the Log Level to DEBUG on page 128. 2. Check if the system is recording. 3. Check if the Storage Center is archiving any calls (See Checking for Archived Calls

on page 129):

• • • •
a.

In Business Analyzer run a query and check if the calls are archived. Physically check on the Storage Center where the files are archived. Use Storage Center Investigator to check the archiving. Use Reporter and run the Archiving Coverage Report. Check the connectivity between the Storage Center and Loggers, the Storage Center and Database Server and the Storage Center and Rule Engine (located on the Applications Server). Check the relevant services on the (for further information on service, see the Maintenance Guide):

4. If the system is not archiving any calls:

b. Check that the Storage Center ports are open. See the Server Hardening Guide. c.

• • • •

Check the Storage Center services: LogService, Nice Storage Streaming Service, and Nice Storage Center Service. Check the rule services on the Applications Server: Nice Rule Engine service and Nice RulesManager Service. Check the Nice Playback Administration service. Check the SQL Services on the Database Server.

d. Check that active storage rules are defined. In Rules Manager, click Storage and check the

storage rules.
e. f.

Check the storage deletion and retention values. See Checking the Storage Retention and Deletion Values on page 133. Verify that the Storage Center Database is selected. See Verifying that the Storage Center Database is Selected on page 133.

g. Check if there is a space problem. See Verifying the Used and Free Space on page 133. h. Check the Storage Unit maximum threshold configured in System Administrator. See

Verifying the Storage Unit Threshold on page 134.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

127

Storage Center Troubleshooting

i.

If you need to define more storage space, add another quota unit (under the quota group) and restart the Storage Center. The Storage Center swaps to the next quota. There is no need to create a new rule, as this can lead to calls not being archived (from the previous rule). Check that all the required updates are installed. Check the Locate Order. See Checking the Locate Order on page 94. Check the Playback logs. See Example of Troubleshooting using the Logs on page 108.

j. k. l.

m. For playback problems, see the General Troubleshooting Flow on page 81. 5. If the system is not archiving specific calls, check for a pattern and establish which calls are

not being archived:
a.

Calls from a specific Logger. See the General High Density Logger Troubleshooting Flow on page 304 or the VoIP Troubleshooting Flow on page 329. personal calls and excluded via a rule). In this case, update the rule.

b. Calls which meet certain criteria, for example, long calls (these may be regarded as c.

Calls from new agents. In this case, update the rule. Calls from extensions that have been changed. In this case, update the rule.

d. Calls from new extensions. In this case, update the rule. e.

6. Check that the storage rule inserts calls. See Checking that the Archiving Rule Inserts New

Calls on page 135.
7. Troubleshoot the storage rules. See Troubleshooting the Archiving Rule on page 135. 8. Check the archiving table in the database. See Checking the Storage Center Archiving

Tables on page 137.
9. Check the relevant logs. See Checking the Storage Center Logs on page 138, Checking the

Nice Storage Streaming Service Logs on page 139, and Checking the Player Logs for Storage Center Errors on page 139.

Setting the Log Level to DEBUG
You change the level of the Storage Center log files and of the Nice Storage Streaming Service log files to DEBUG.
To set the log level to DEBUG: 1. Set the Storage Center log level from LOW to DEBUG by changing the entry in the registry: a.

Click Start > Run. From the Edit menu, select Find, and enter DEBUGCATEGORYMASK. Restart the Storage Center service.
Chapter 4: Archiving Troubleshooting Flow 128

b. Enter Regedit. c.

d. Set the value to 255. e.

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Storage Center Troubleshooting

2. Change the log level in Nice Storage Streaming Service log files to DEBUG. a.

Navigate to the Nice Storage Streaming Service storage directory and open Nice.Storage.StorageStreaming.StorageStreamingHost.exe.

b. In the Nice Storage Streaming Service configuration file, change the log Filter attribute to DEBUG. c.

Save the Nice Storage Streaming Service configuration file.

Checking for Archived Calls
You can check if calls are archived by running a query in Business Analyzer, checking physically in the folders on the Storage Center or using the Storage Center Investigator.
To check for archived calls: 1. Run a query in Business Analyzer to check which segments were archived: a.

Run a query, for example, Segment - Last 7 days. .

b. Click Preferences c.

In the Table View area, move the Voice Archive Status and Screen Archive Status columns from the Available Columns to the Selected Columns list box.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

129

Storage Center Troubleshooting

d. Click

Save. The icon

indicates a segment that is not archived.

2. Check the folders on the Storage Center to see if files have been written: a.

The Storage Center contains Storage Groups which are a logical grouping of one or more Storage Units. Each time you create an archiving rule (via the Rules Manager), you associate the rule with a Storage Group. The a Storage Unit contains either:

• •

Directory Per Day: Creates subdirectories under the defined directory path (for each

day of the year).
Multiple Directories: Creates subdirectories under the defined directory path (based

on the destination type). Files are written to the subdirectories in a cyclic manner: When one subdirectory reaches full capacity, files are written to the next subdirectory.
b. In the example below, recordings are archived in Directory Per Day storage units,

organized by server name\task name\archiving date.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

130

Storage Center Troubleshooting

3. Use Storage Center Investigator to check the archiving. See Using the Storage Center

Investigator on page 109.
a.

Double-click the

icon on the desktop to activate the application.

b. Click Start Storage Center Investigator in automatic mode. c.

In the Storage Center Installation directory field, verify that the Storage Center path is correct. In the Automatic Reporting Settings fields:

d. In the Available Databases area, select the checkbox of the database you want to query. e.

• •
f.

Enter the desired time interval to generate automatic reports. Enter the desired number of days backwards to include in the reports.

Click Next to view the results. sub-tab to view query results.

g. In the NICE Storage Center Investigator Results tab, click the Database overview

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

131

Storage Center Troubleshooting

4. Check the archiving to the Storage Center. In Reporter, run the Archiving Coverage Report, and check the Archiving Coverage per Storage Center view.

Gathering Storage Center Information
Use the Storage Center Investigator to verify that all the required files are archived on the file system. The Storage Center Investigator is installed together with the Storage Center in the same installation directory. The default path is: C:\Program files\NICE Systems\Storage Center. See Using the Storage Center Investigator on page 107 for detailed instructions on how to use the Storage Center Investigator.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

132

Storage Center Troubleshooting

Checking the Storage Retention and Deletion Values
The Storage retention value is the number of retention days for the files in the Storage Group, meaning the number of days that the recordings must remain in the Storage Group. By default, there is no mandatory retention. The retention value is usually set to the amount of days that the site is obligated to save recordings. The maximum retention period is seven years (based on the NICE database design). When you define a storage rule, you can define a value for Force Deletion, meaning that files archived by a specific rule will be deleted after a certain number of days. What happens if a conflict occurs between the group's Retention value and a rule's Force Deletion value (such as the group's Retention value is set to 100 days)? The Force Deletion value of the rule takes preference! The group Retention value is a more general definition; the archiving rule Force Deletion value is specific to a group of archived files.

Verifying that the Storage Center Database is Selected
In the System Administrator, check that the Storage Center database is selected.

In System Administrator, in the Organization tree, under Master Site > Database Servers, General tab, check that the nice_storage_center database is selected in the Databases on this SQL Server area.

Verifying the Used and Free Space
Check the used and free space on the Storage Center servers and storage areas.
1. In System Administrator, in the Organization tree, under System Monitoring > Storage Center Servers branch, expand the hierarchy to view a list of all Storage Centers. 2. Expand a Storage Center to view its storage areas.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

133

Storage Center Troubleshooting

3. Select a Storage Center or one of its storage areas. The Status window appears.

Verifying the Storage Unit Threshold
Check the storage unit threshold on the Storage Center server.
To verify the storage unit threshold: 1. Log in to the NICE Perform Application Suite. In the navigation bar, click Accessories, and select System Administrator.

2. From the Settings menu, select Technician Mode.

3. In the Organization Tree, select the Storage Center Servers branch. 4. Expand a Storage Center to view its storage groups and units (SCGroup > SCUnit), and

check the Maximum Threshold.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

134

Storage Center Troubleshooting

Figure 4-4

Storage Details

Maximum Threshold

Checking that the Archiving Rule Inserts New Calls
Archiving to Storage Center starts with the archiving rule. The rule determines which calls needs to be archived and then adds the required information to the tblStorageCenter and tblSCTasks tables in the database.
1. Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the

relevant calls to verify that the Rule Engine inserts new calls. From Storage Center Investigator, query iArchiveID to see if the ID number increases, and check the Start and Stop times of the last interactions. When you know the time of the last interaction, you can verify whether new interactions were inserted.
2. Check whether new tasks are added to the tblSCTasks table in the nice_admin database. If

tasks are added, check that they match the rule filters. If the Rule Engine inserts new calls into tblStorageCenterXX, the archiving rule is probably correct and the problem is in the Storage Center. In this case, continue with Checking the Storage Center Archiving Tables on page 137. If there are no new calls, continue with Troubleshooting the Archiving Rule on page 135.

Troubleshooting the Archiving Rule
Troubleshooting the archiving rule comprises the following steps:
1. Check the Rule Engine Log file for errors. 2. Check that there is an Active Rule in the Rule Manager, and check its schedule. 3. Check the Storage Center rule schedule. See the procedure below. 4. Check that the Rule Engine generates actions. See page 136. To check the Storage Center rule schedule: 1. In the nice_rule database, check the next scheduled time of the Storage Center rule by

running the following query:
SELECT tblREStorageCenterRule.nvcRuleName, tblRELocalRuleSchedule.iScheduleID, tblRELocalRuleSchedule.dtNextScheduleTime,
Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 135

Storage Center Troubleshooting

tblRELocalRuleSchedule.dtLastRunTime FROM tblRELocalRuleSchedule INNER JOIN tblREStorageCenterRule ON tblRELocalRuleSchedule.iScheduleID = tblREStorageCenterRule.iScheduleID WHERE (tblREStorageCenterRule.iStatus = 1)

A section of the tblREStorageCenterRule table appears below, showing the format of the date fields.

dtNextScheduleTime is the GMT time. dtLastRunTime is the local time of the Applications Server. 2. Check that the Rule Engine database is not full. To check that the Rule Engine generates actions: 1. Check that the Rule Engine generates actions for the Storage Center Rules by running the

following query:
SELECT * FROM tblREStorageCenterAction 2. In the query results, check the start time, stop time and interaction IDs in the results of the

query.
3. If the table does not fill up, check that the Event Providers collect the interactions by running

the query:
SELECT * FROM tblRECallServerEvent 4. Check that the results contain calls that are not archived.

• •

If the table does not contain the calls, the error can be in the Event Collection step. Call NICE Customer Services. If the interaction is in the Event table, the problem can be in the Generation step. The reason could be one of the following:

• •

The Generation Query takes a lot of time and does not complete because of timeout. Maybe there are missing indexes. See the Database Administrator. The rule has problems with the file name mask and the post process of the Storage Center rule. The database function spREStorageCenterRulePostProcessing failed. Check that the File name mask is not empty by running the following query:

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

136

Storage Center Troubleshooting

SELECT nvcRuleName, iRuleId, nvcDescription, iStatus, nvcFileNameMask FROM tblREStorageCenterRule

Check that nvcFileNameMask is not empty for the required rule.

The problem could be in the Execution step. In a Multi Data Hub environment, the secondary Rule Engine Log file can contain the following message: [StorageCenterExecuter.OnProcessWorkItem] Failed to execute Storage Center action System.Data.SqlClient.SqlException: The operation could not be performed because the OLE DB provider 'SQLOLEDB' was unable to begin a distributed transaction. [OLE/DB provider returned message: New transaction cannot enlist in the specified transaction coordinator.] OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin:JoinTransaction returned 0x8004d00a].

It may be that the Microsoft Distribute Transaction Coordinator (MSDTC) is not configured properly. Storage Center rules require that the MSDTC works correctly. This can be checked by performing the following command:
BEGIN TRANSACTION SELECT * FROM nice_rule_link.nice_admin.dbo.tblSCTasks COMMIT

If the command fails, MSDTC must be fixed.

In a Multi Data Hub environment, the problem could be in the Rules Manager behavior. Rules defined by the Rules Manager are relevant for all sites. The Rules definitions are written to the Master site and are replicated to all Secondary Data Hubs.

Checking the Storage Center Archiving Tables
1. Check the iFSArchiveClass column in the tblStorageCenterXX table. The values represent

the following:
ID 0 1 2 3 Status Not archived yet to Storage Center file system. Archiving in progress to Storage Center file system. Archived to Storage Center file system. Call is being deleted from Storage Center file system. Action Check if the Storage Center service is up and check the Storage Center log files. Check the Storage Center log files. Everything is OK.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

137

Storage Center Troubleshooting

ID 4 5 6 7

Status Call deleted from Storage Center file system. Archiving failed to Storage Center file system. Call deletion from Storage Center file system failed. Call was not found in the Logger.

Action

Check the Storage Center log files. Check the Storage Center log files. Check why the call is not found in the Logger

2. Check the iEsmArchiveClass column in the tblStorageCenterXX table. The values

represent the following:
ID 0 1 2 3 4 5 6 Status Not archived yet to ESM. Archiving in progress to ESM. Archived to ESM. Call is being deleted from ESM. Call deleted from ESM. Archiving failed to ESM Call was not found in the Logger. Check the Storage Center log files. Check the Storage Center log files. Action Check if the Storage Center service is up and check the Storage Center log files. Check the Storage Center log files. Everything is OK.

Check the Storage Center log filesr

Checking the Storage Center Logs
1. Review the Storage Center logs. By default, the log files are stored in the same directory

where the Storage Center is running. The default log file names are:

• •

StorageCenterLog.log StorageCenterLog1.log through StorageCenterLog100.log
Time Event Category Description

Owner

Thread ID Chapter 4: Archiving Troubleshooting Flow 138

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Storage Center Troubleshooting

2. Track the task snapshots in the log files. Each task snapshot details the current tasks and the

number of elements distributed among Loggers.
Task Name Task ID Rule ID

Logger ID

Number of Elements

3. Ensure that the logs do not contain errors relating to the Storage Center startup process. 4. To collect the logs, use the Log Collector. See the Collecting Information Using the Log

Collector on page 537 for further information.

Checking the Nice Storage Streaming Service Logs
You check the Nice Storage Streaming Service logs to verify that playback performed from Storage Center is successful. By default, the log files are stored in the following directory:...\Program Files\Nice Systems\Storage Center\Nice Playback Service SC\StorageStreaming<Process ID>Logs. The Process ID is a value that changes each time the system is restarted. The following procedure lists the text strings you need to find in the Nice Storage Streaming Service logs in order to verify that playback from the Storage Center was successful.
To check the Nice Storage Streaming Service logs: 1. Search for the text ERR:, and verify that it does not appear in the log file. 2. Search for the text StartOutgoingSessionEvent, and verify that it appears. 3. Search for the text OpenSession, and verify that it appears. 4. In the same row as the OpenSession text, verify that Result=OK appears. 5. Search for the text StartStreamingPackets, and verify that it appears. 6. Search for the text StopOutgoingSessionEvent, and verify that it appears. 7. Search for the text StopStreamingPackets, and verify that it appears.

Checking the Player Logs for Storage Center Errors
The player errors are recorded in the Player log files, in *.txt format, in %APP_DATA%\NicePlayer\Release3. The following Storage Center errors may appear in the Player logs.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

139

Storage Center Troubleshooting

Error No. 300007

Description The Storage Center failed to connect to a backup device on a Logger. This should not be considered an error if there are no backup devices in the site. The Storage Center did not archive or it had an error in the locate flow.

Action Required Check the Logger where the backup device is located.

300008

See the Playback Troubleshooting Flow on page 77.

Other Troubleshooting Issues

If the Storage Center is not communicating with the NMS, see the Storage Center Installation Guide. Perform the following verification tests to ensure that all the objects are configured properly:

• • • •

Verify Nice Storage Center Service User Verify the SnmpAgent Object Verify the SNMP Service Settings

If interactions are not being archived on a Storage Center server with multiple active network interface cards, check that the Storage Center IP address defined in the System Administrator is the primary network connection. For more details, see the Storage Center Installation Guide. If there are non-archived calls in the Storage Center tables, use the Storage Center Investigator to check if the iRuleID of the calls is still active. Storage Center will not archive calls that belong to an inactive rule. In this case, change the iRuleID to an active one. See Using the Storage Center Investigator on page 107.

Chapter 4: Archiving Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

140

5
Multi Data Hub Troubleshooting Flow
This chapter describes common troubleshooting issues when employing a Multi Data Hub with NICE Perform Release 3.1.

Contents
Gathering Information ..................................................................................................142 Multi Data Hub Implications.........................................................................................143 Checking Connectivity .................................................................................................144 Checking Replication ...................................................................................................145 Troubleshooting Replication .......................................................................................147 Troubleshooting Saving in a Multi Data Hub Environment ......................................148 Changing Passwords ...................................................................................................148 Enabling MSDTC to Function in Multi Site Environments ........................................148 Other Troubleshooting .................................................................................................149

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

141

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

142

Multi Data Hub Implications

Multi Data Hub Implications
A Multi Data Hub system has site implications that affects the different system components. Changes in the components are all done automatically with no user intervention or configuration.

Database
• •
The Admin, Interaction, Audit and Rule databases are mandatory on all Data Hubs. The nice_interactions databases are not replicated across Data Hubs.

System Administrator
• •
Administration updates are blocked when the Primary Data Hub is not available. The login mechanism will check login token on all Data Hubs. Users will belong to a specific Data Hub.

Rule Engine
• • •
All rules are replicated from the Master Data Hub to the Secondary Data Hubs. Each local rule engine will handle the local rules. Storage Center rules are an exception as they are all stored only on the Master Data Hub. This means that all Data Hubs operate using the same Storage Center rules.

Business Analyzer
• • •
Public, private and saved items are replicated between all Data Hubs. User preferences are local. Running a query on Multi Data Hubs:

• • • • • •

The Unified Query engine queries the local Data Hub. Each remote Data Hub with agents from the query filter will be also queried. All results will be combined into one result.

Locate The Unified Query service will connect all relevant Locate services in all Data Hubs. The Locate process looks only in the local Data Hub according to the locate order.

Playback
• •
Each Data Hub has its own Media Server. One or more Playback Servers can be configured in each Data Hub.

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

143

Checking Connectivity

Monitor
• •
User preferences are saved on the local Data Hub. Monitor will subscribe to events to the local CLS or remote Monitor service.

Quality Management
• • •
All QM objects (Evaluations, Coaching Packages, Calibrations and Clips) will be saved in the user's local Data Hub. Evaluations on remote calls will be replicated to the remote Data Hubs. Form definitions are replicated to all remote servers.

My Universe

Preferences are stored locally.

Interaction Server
• •
All recording rules are replicated. Only handles local Integrations.

Storage Center
• •
Tasks are read from the Master Data Hub only. Archiving can be done from all Data Hubs.

Checking Connectivity
Check connectivity between sites (PING using IP address and hostname).

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

144

Checking Replication

Checking Replication
Use the following methods to check replication in Multi Data Hub environments.
To check replication in multi data hub environments: 1. In the NICE System Administrator, verify that the replication is defined using only the

machine names.
2. Check the connectivity between the Primary and Secondary Data Hubs. 3. Connect to the SQL Server Management Studio. In the Object Explorer, navigate to Replication and check if there are components with an error. 4. In the NICE System Administrator: a.

Navigate to Master Site > Database Servers > Databases and click the Replication tab.

b. Verify that no errors appear in the Status column. 5. If the Primary and Secondary Data Hubs have different database structures, the replication will

fail at those specific points. To ensure that the replicated data is identical, run tests on all Data Hubs to ensure that additional information (business data, any external additions such as Professional Services solutions) can be accessed from different sites.
6. Verify that the same version of the NICE Perform Database and the same updates are installed

on all Data Hubs. Check the following tables in the nice_admin database:
a.

The database version appears in the tblDatabaseDetails table.
Figure 5-1 tblDatabaseDetailsTable

b. The update versions appear in the tblDatabaseHotFix table.
Figure 5-2 tblDatabaseHotFixTable

NOTE: Only those updates that were installed using an Update Pack appear in the tblDatabaseHotFix table; updates that were installed manually do not appear.

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

145

Checking Replication

7. When a Multi Data Hub environment is upgraded, replication should be removed during the

procedures. After the upgrade has been completed, create the replication again. Then test that the replication is functioning properly.

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

146

Troubleshooting Replication

Troubleshooting Replication
In most problems that deal with replication, the Primary Data Hub is usually updated with information. If there is a problem updating the Primary Data Hub, the replication itself will not begin and the problem lies elsewhere. It is therefore important to first verify that the Primary Data Hub has been updated and only then start troubleshooting replication problems.

Troubleshooting Communication Problems between Primary and Secondary Data Hubs
Secondary Data Hubs are read only. All updates are written to the Master Data Hub and replicated to the Secondary Data Hub.
To troubleshoot communication problems: 1. If there is a communication problem between the Primary and Secondary Data Hubs, data

updates on the Secondary Data Hub will not take effect. These data updates will not be stored locally and will have to be completed once the communication problems have been resolved.
2. If a Secondary Data Hub is currently disconnected from the Master Data Hub, updates from

other Data Hubs will not be seen. Since all information is stored in the distribution database on the Master Data Hub, these data updates will be completed once the Data Hub is connected again.

Troubleshooting Database Structure Differences
The database structure can be changed in different ways, for example, names, number of fields, field type, field length, additional or fewer tables. If a Master Data Hub and Secondary Data Hub have different database structures the replication will fail at those specific points.
NOTE: It is very important not to make "manual" changes to the Secondary Data Hubs as these will be overwritten when the data is replicated from the Primary Data Hub. To check the database structure: 1. To ensure that the data is identical, run tests on all Data Hubs to ensure that additional

information (business data or any external additions such as Professional Services solutions) can be accessed from the different Data Hubs.
2. For example, an error from the Nice DW Population Schedule job may indicate a problem in the tblReservedXX tables (containing the business data fields).

Stopping Replication When Upgrading All Data Hubs
Upgrades must be applied to all Data Hubs simultaneously. An upgrade can be an Update or Service Pack. Since upgrades may cause a change within the SQL structure, it is important to remove the replication before implementing the upgrade.

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

147

Troubleshooting Saving in a Multi Data Hub Environment

NOTE: You must bring down communication between all the Data Hubs when upgrading. To stop replication when upgrading all data hubs: 1. Remove the replication prior to the upgrade. The replication should be stopped to all Data

Hubs, not just the Data Hub that is being upgraded
2. Once the upgrade is complete, create the replication again. 3. Test the site operation.

Other Troubleshooting Issues
• •
Replication works with the Machine Name only. Any change to the database scheme of replicated or un-replicated tables requires that the replication be shut down for that site.

Troubleshooting Saving in a Multi Data Hub Environment
In a Multi Data Hub environment, some of the information in nice_admin is replicated. If replication is down, you will be unable to save in System Administrator.

Changing Passwords
If the nicesa user password is changed, the replication agents will indicate that the password is not valid and the replication will not work and must be recreated.
To change nicesa password in Multi Data Hub environment: 1. Remove the replications. 2. Change the nicesa password. 3. Recreate the replications.

Enabling MSDTC to Function in Multi Site Environments
The SQL Server uses Microsoft Distribute Transaction Coordinator (MSDTC) to execute distributed transactions. MSDTC will not function properly in an environment where the two servers are using the same Windows OS image, for example where the 'Ghost' program is used to copy the Windows image to the disk. The reason for the problem is that both servers that are installed from the same image, have an identical key in the MSTDC section in the SQL registry. If the distributed transaction fails and the registry keys are different, there may be other reasons for the failure. See TN0676 Enabling Microsoft Distribute TransactionCoordinator (MSDTC) to Function in Multi Site Environments. This note describes how to fix the problem without re-installing the entire system.

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

148

Other Troubleshooting

Other Troubleshooting

A replication error Cannot insert explicit value for identity column in table 'tblColumnInfo' appears. This occurs if the subscription table tblColumnInfo.iColumnId identity is set to true. To solve the problem set the identity value to False. If there is a problem with replication, submit the following log file to NICE Customer Support:
Program Files\NICE Systems\ Applications\ServerBin\NiceApplications.SystemAdministrator.Host.exe.log

Chapter 5: Multi Data Hub Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

149

Blank page for double-sided printing.

6
Media Encryption Troubleshooting Flow
This chapter includes tables of trap messages and ScreenAgent Management System (SAMS) log messages as well as common troubleshooting issues.

Contents
Troubleshooting Workflow ..........................................................................................152 Trap Messages ..............................................................................................................159 SAMS Log Messages ...................................................................................................163 Verifying the IIS User Permissions for the KSM Configuration................................164 Using a Fully Qualified Domain Name instead of a Host Name ...............................166 Data Not Entered in the nice_crypto Database ..........................................................169 Renewing Certificates ..................................................................................................171 ScreenAgent Not Working Properly............................................................................172 Ensuring Connection between SQL Server and Applications Server .....................175 Installing the CA certificate on Vista Workstations ...................................................175

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

151

Troubleshooting Workflow

Troubleshooting Workflow
If you experience a problem with Media Encryption follow the steps below to isolate the problem.
1. Change the logging level to ALL. See Changing the Logging Level on page 153. 2. On the Key Storage Manager (KSM), check the connectivity between the Applications Server

and the Database. See Checking Server Connections on page 153.
3. Verify the connectivity between the Logger/ScreenAgent and the KSM on the Applications

Server by pinging the Applications Server.
4. Verify the connectivity between the KSM on the Applications Server and the Cryptographic

Database by pinging the SQL Server which has the NICE Cryptographic Database (nice_crypto).
5. Use Telnet to check that the Logger/ScreenAgent can reach the Applications Server on port

443.
6. Check the Logger/ScreenAgent. See Checking the Logger/ScreenAgent on page 153. 7. Check the KSM status. See Checking the KSM Status on page 154. 8. If the KSM status is FAILED, verify that the KSM is functioning properly. See Verifying the

KSM is Functioning on page 154.
9. Check the CIM trap messages. See Checking the SNMP Traps on page 154. 10. Check the KSM trap messages. See Checking the SNMP Traps on page 154. 11. Check the client-side logs. See Checking the Client-side Logs on page 156. 12. Check the server-side logs. See Checking the Server-side Logs on page 156. 13. Check for certificate problems. See Checking for Certificate Problems on page 157.

Gathering Information
Important issues to consider are:
1. Is this a new installation? If yes, when was the installation performed? 2. Were there any changes to the environment that might have influenced the system? If yes,

check these changes first.
3. How often does the problem occur? 4. Is the problem reproducible? If yes, what were the steps? 5. Were the Servers restarted? If yes, did this resolve the problem?

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

152

Troubleshooting Workflow

Changing the Logging Level
To generate more detailed logs, you change the logging level in the Logger and the ScreenAgent.
To change the logging level:

On the Logger/ScreenAgent, in ConfigPlugin, set the debug level, by changing the values in both Level and Threshold to ALL:
<logger name="NiceApplications.SystemAdministrator"> <param name="Level" value="ALL" /> <!-- Define some output appenders -->

<log4net>

<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender"> <param name="Threshold" value="ALL" />

Checking Server Connections
You can check on the KSM whether the files were sent to the SQL Server database. If the folder contains files, this indicates a connection problem between the Applications Server and the SQL Server database.
To check the Applications Server to Database connection:

On the KSM, check the EncryptionBackupFiles folder located at the following path:
...\<location of applications>\KeyStoreManager\WebService\EncryptionBackupFiles.

If this folder contains files, this indicates a problem with the connection between the Applications Server and the SQL Server database.

Checking the Logger/ScreenAgent
You need to verify that the encryption services are running properly on the Logger/ScreenAgent.
To check the Logger/ScreenAgent: 1. On the Logger/ScreenAgent, verify that the LogService located in C:\Program Files\Common

Files\Nice\LogService\Bin\LogSysSrvr.exe is running.
2. Verify that the CIMService.exe service is running. 3. Verify that CIMService.exe service has Local System Account privileges. 4. Verify that the service has privileges to write to the disk. 5. Verify that there is enough space on the disk.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

153

Troubleshooting Workflow

6. Check if the CimClinet, CimHistory and CimKSM files exist in the C:\document and settings\all users\application data\Nice\CryptoInfo folder on the Logger/ScreenAgent.

The size of the CimKSM file is 0 KB. If the CimKSM file is not 0 KB, then there is a problem with the connection between the Logger and the KSM.

Checking the SNMP Traps
In the SNMP Console, view the events generated by the CIM and KSM. See the NMS User's Guide or Trap Messages on page 159 for a list of the trap messages and recommended actions.

Checking the KSM Status
To check the KSM status: 1. In the navigation bar, click Accessories, and select System Administrator. 2. Verify that you are working in Technician mode: From the Settings menu, select Technician Mode.

3. In the Organization tree, select Security > Media Encryption.

4. Click Test to check the KSM status. If the status is Failed, verify that the KSM is functioning

properly. See Verifying the KSM is Functioning on page 154.

Verifying the KSM is Functioning
When you cannot connect to the KSM, you do not know if the problem lies with the KSM or with the client machine. In this procedure, you verify that the KSM is functioning properly.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

154

Troubleshooting Workflow

To verify if KSM functions properly: 1. On the KSM server, navigate to IIS > Web Sites > Default Web Site > KSMWebService. 2. In the right pane, right-click KSMService.asmx and select Browse.

The Internet Explorer opens.
3. In the Address field, change the URL address to: https://<KSM machine name>/KSMWebService/KSMService.asmx.

If the following screen appears, then the problem is probably with the client workstation.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

155

Troubleshooting Workflow

NOTE: If you did not replace the name of the local host with the KSM machine name, the following window appears:

Make sure that you click No. (If you click Yes, the verification process is skipped.)

Checking the Client-side Logs
Check the following client-side logs:

Nice Player <Drive>:\Documents and Settings\Administrator\Local Settings\Application Data\NicePlayer\Release3\NicePlayer.txt

Logger/ScreenAgent <Drive>:\Documents and Settings\All Users\Application Data\Nice\Log\CryptoInfoManagerLogExceptions.log

Encryption <Drive>:\Documents and Settings\Administrator\Local Settings\Application Data\NICE\EncryptionPluginClientLog.txt

Checking the Server-side Logs
Check the following server-side logs:

KSM C:\Program Files\NICE Systems\Applications\KeyStorageManager\WebService\KeyStorageManager.txt

Encryption C:\ Program Files\NICE Systems\Applications\ServerBin\NiceApplications.EncryptionPlugin.log

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

156

Troubleshooting Workflow

Checking for Certificate Problems
The certificates might have expired.
To check for certificate problems: 1. Check with your certification authority if your certificates have expired. 2. On the Applications Server, check in the IIS log files (KeyStorageManager.txt.x) for the

following type of error:
ERROR: [SNMP.RegistrateToTheSNMPConnector] SnmpConnector: New SNMPConnectorObject was added to the connector.

This indicates a problem with the site certificate. Check for other errors as well.
3. On the Logger/ScreenAgent, check in the Event Viewer if there are any CIMService error

messages. In the Source EFS and Description areas, the following error indicates a problem with the site certificate.
An error with Source: EFS and Description: EFS recovery policy contains invalid recovery certificate.

4. On the Logger/ScreenAgent, check the properties of the CryptoInfo folder: a.

Go to C:\Documents and Settings\All Users\Application Data\CryptoInfo, right-click the CryptoInfo folder and select Properties.
Chapter 6: Media Encryption Troubleshooting Flow 157

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Workflow

b. Select Advanced. The Advanced Attributes window appears.

c.

Select Encrypt contents to secure data. If you receive the following error, this indicates a problem with the site certificate.

5. Issue new certificates for the SQL Server and IIS.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

158

Trap Messages

Trap Messages
Trap Messages for CIM Module
Trap No. 6141 Trap Cryptographic Information Manager cannot connect to KSM. Cause There is no certificate. (This trap will be sent with the no certificate trap). There is a problem in the KSM (not running). The KSM URL obtained from the System Administrator is invalid. 6142 Cryptographic Information Manager cannot connect to System Administrator. Cryptographic Information Manager cannot save encryption data to a persistent file. There is no connection to the System Administrator. System Administrator URL is invalid. The file attribute is read only. Action Install CA root on the machine. Check the KSM. Check the DB of the SysAdmin for the KSM URL – see that its valid.

Check that the SysAdmin is up. Install the client machine once again and change the SysAdmin URL. Change the attribute of the files to read/write. The path is C:\Documents and Settings\All Users\Application Data\Nice\CryptoInfo.

6143

6144

Cryptographic Information Manager is not initialized.

There is probably a problem in the initialization (such as encryption not configured). This trap will be sent together with another trap. The encryption is not yet configured in the system. The KSM is not yet configured. There is no certificate installed on the client machine. Check the entire encryption components and check that the KSM is configured properly. Install the CA root in client machine. (Make sure that the CA is installed both on current user and on the local machine.) There is no SysAdmin in the registry – it was not provided in the client installation. Run the client installation and set the SysAdmin URL.

6145

Encryption is not configured in the system. The Customer has no valid certificate for encryption.

6146

6147

SystAdmin URL registry key is empty.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

159

Trap Messages

Trap No. 6148

Trap KSM URL does not exist in the System Administrator. There is no license for encryption in the system. The Logger login token is not valid.

Cause The SysAdmin URL is not aware of the KSM URL, therefore when asked for the KSM URL it returned null. The encryption feature does not exist in the client site. The login token against SysAdmin is invalid probably because of mismatch between DLLs in the CIM server and in the SysAdmin.

Action Check the SysAdmin database for the KSM URL.

6149

Check that there is license for encryption for the client. Check the version of the SysAdmin DLLs in the CIM Server - check that this is the correct version.

6150

Trap Messages for KSM Module
Trap No. 5200 Trap Cannot establish a secure connection between Key Storage Manager and the database. Cause a. Database is down. b. Connection problem to the database. c. Secure connection problem to the database: no CA root installed on the KSM machine, problem with the SQL certificate. Action a. Start the database. b. Verify that the connection to the database is OK (using the Query Analyzer). c. Check that there is a CA root installed on the KSM machine. Verify that there is a certificate assigned to the SQL service and that it is valid.

5201

Secure connection between Key Storage Manager and database was established successfully. The database certificate is missing/corrupted: data cannot be encrypted/decrypted. The database certificate is valid: data can be encrypted/decrypted.

Fixes the above trap.

5202

The database certificate is missing. Therefore, no encryption or decryption of the encrypted data is possible. Fixes the above trap.

Call NICE Support Services.

5203

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

160

Trap Messages

Trap No. 5204

Trap Cannot establish a secure connection to remote Key Storage Manager.

Cause a. Remote KSM machine is down. b. No connection to the remote KSM machine. c. Remote KSM web service is unavailable. d. There is no CA root installed on the local KSM machine.

Action a. Start the machine. b. Verify that the connection to the remote machine is OK. c. Use the Media Encryption branch in order to test the connection to the remote KSM. d. Verify that there is a CA root installed on the local KSM machine. Also, use the Media Encryption branch from the local KSM machine in order to test the connection to the remote KSM.

5205

Secure connection to remote Key Storage Manager was established successfully. Database master key cannot be opened: data cannot be encrypted/decrypted.

Fixes the above trap.

5206

Restore procedure of the database was incomplete: the database master key was not restored correctly.

Check in the nice_crypto database, security, certificate folder, that the certificate exists. Notify NICE Support Services.

5207 5208

Database master key opened successfully. Duplicated GUID was stored to the database. KSM module is up.

Fixes the above trap. A GUID that already exists. The KSM web service is up. The KSM failed to write to the backup file. This could happen if the KSM application pool user does not have privileges to the EncryptionBackUpFiles folder. Use the Media Encryption branch in order to test the KSM status. KSM also tries to create a file in the EncryptionBackUpFiles folder. Notify NICE Support Services.

5209

Failed while writing Cryptographic Information to file.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

161

Trap Messages

Trap No. 5210

Trap Succeeding in writing Cryptographic Information to file. Failed while reading Cryptographic Information from file.

Cause Fixes the above trap.

Action

5211

a. The KSM failed reading the backup file. This could happen if the KSM application pool user does not have privileges to the EncryptionBackUpFiles folder. b. Can also happen if the file was written and encrypted by a different user (The KSM application pool user was replaced).

a. Verify that the KSM application pool user has read privileges to the EncryptionBackUpFiles folder. b. Check which user encrypted the file, by right-clicking the file > Properties. Then in the General tab, click Advanced > Details. If the user is not the current KSM application pool user, change it to the user who encrypted the file, and wait for the file’s content to be written to the database.

5212

Succeeded in reading Cryptographic Information from file.

Fixes the above trap.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

162

SAMS Log Messages

SAMS Log Messages
ScreenAgent Module Messages
Message. Problem to connect to server Server not initialized Problem connecting to KSM Problem connecting to SysAdmin Encryption not configured Client is not connected Problem to create client connection Problem in client connection SysAdmin URL does not exist KSM URL does not exist Problem saving Crypto to file No license for encryption in site CIM Error No encryption license in the System. Cause CIM Error CIM Error Network, Server is down, wrong IP, etc. Network, Server is down, wrong IP, etc. Encryption is not configured in SysAdmin CIM Error CIM Error CIM Error Verify server is up, IP is correct, etc. Verify server is up, IP is correct, etc. Configure Encryption. Action ScreenAgent Behavior ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds ScreenAgent will record with encryption. ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds ScreenAgent will retry until the operation succeeds ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds. ScreenAgent will retry until the operation succeeds. ScreenAgent will record without encryption.

Enter the SysAdmin URL in the registry. Enter the KSM URL in the registry. If the site is to be an encryption site, install a license. If the site is not an encryption site, reinstall the ScreenAgent with no encryption. Install certificate.

No certificate on the machine

Certificate was not installed on the workstation.

ScreenAgent will record with encryption.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

163

Verifying the IIS User Permissions for the KSM Configuration

Verifying the IIS User Permissions for the KSM Configuration
If there is an error when configuring the KSM, you need to verify that the appropriate permissions are granted to the IIS user for the following folders:

• • •

Temp folder EncryptionBackupFiles folder WebService folder

IMPORTANT : Make sure that all permissions are also inherited by sub-folders.

To verify IIS user identity permissions: 1. Open the C:\Windows\Temp folder. 2. Click the Security Tab.

The following window appears.

3. Select the group or user names of the current IIS user. 4. In the Permissions for Administrators area, verify that the following permissions are selected in the Allow column:
Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 164

Verifying the IIS User Permissions for the KSM Configuration

• • •

Read & Execute Read Write

5. Click OK. 6. Navigate to the

...\<location of applications>\KeyStoreManager\WebService\EncryptionBackupFiles folder and repeat Step 2 and Step 3.
7. In the Permissions for Administrators area, verify that the Full Control permission is selected in the Allow column. 8. Navigate to the ...\<location of applications>\KeyStoreManager\WebService\WebService

folder and repeat Step 2 and Step 3.
9. In the Permissions for Administrators area, verify that the Full Control permission is selected in the Allow column.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

165

Using a Fully Qualified Domain Name instead of a Host Name

Using a Fully Qualified Domain Name instead of a Host Name
If the IIS and SQL certificates require a Fully Qualified Domain Name (FQDN), you need to ensure that all calls to the KSM Web service and connections to the Database Server are made through the FQDN, as opposed to through the host name.
IMPORTANT : Ping the Database Server using the FQDN to verify that it can connect in this manner before changing the host name. The FQDN is displayed in the Full computer name field of the Computer Name tab in the System Properties window. (Right-click My Computer and select Properties to open the System Properties window.)

To use an FQDN for the KSM Web service and SQL Server: 1. Ping the Database Server using the FQDN to verify that it can connect in this way. 2. From the System Administrator application, expand the Master Site (Primary Data Hub) branch and select Applications > Application Servers > <name of server where the Key

Storage Manager is installed>.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

166

Using a Fully Qualified Domain Name instead of a Host Name

The Application Server Services screen appears.

3. In the Host Name/IP Address field of the Key Storage Manager (KSM), enter the FQDN

instead of the host name.
4. Restart the CIM service on all Loggers and ScreenAgents. 5. Select Database Servers.

The Database Server screen appears.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

167

Using a Fully Qualified Domain Name instead of a Host Name

Figure 6-1

Database Server Screen

6. In the Host Name/IP Address field of the SQL database, enter the FQDN instead of the host

name.
7. Restart the SQL Server.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

168

Data Not Entered in the nice_crypto Database

Data Not Entered in the nice_crypto Database
Sometimes the data in the CryptoInfoData.1.txt file is not entered as expected into the nice_crypto database. This happens if the current KSM user is not the user who originally encrypted the file. In this procedure, we do the following:

• • •

Verify the identify of the original user who encrypted the file Change the KSM user back to the original user in order to access the file Change the original KSM user back to the current KSM user

To enter data from CryptoInfoData.1.txt file into the nice_crypto database: 1. Right-click the CryptoInfoData.1.txt file, and select Properties. NOTE: By default, the CryptoInfoData.1.txt file is located at the following path: ...\Program Files\NICE Systems\Applications\KeyStorageManager\WebService\ EncryptionBackupFiles 2. On the General tab, click Advanced.

The Advanced Attributes window appears.

3. Click Details.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

169

Data Not Entered in the nice_crypto Database

The Encryption Details window appears.

4. The user who originally encrypted the file appears in the Data Recovery Agents For This File As Defined By Recovery Policy area. 5. If the original user is different than the current user, configure the Key Storage Manager server

with the original user.
6. Restart IIS. 7. Wait for the nice_crypto database to access the CryptoInfoData.1.txt file. 8. Change the original user back to the current user.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

170

Renewing Certificates

Renewing Certificates
Instead of renewing certificates, you need to issue new certificates for the SQL Server and IIS.

Screens Not Recorded
At times, screens are not recorded when encryption is enabled, and the following error message appears:
CCimHelper: GenerateCryptoInfo failed. Retrying...: Problem saving Crypto to file. state = -4&gt.

In this case, the Antivirus software is interfering with screen recording.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

171

ScreenAgent Not Working Properly

ScreenAgent Not Working Properly
ScreenAgent is not working properly, and the following appears in the ScreenAgent log file:
EVENT: 28/03/07 14:07:08:276 CCimHelper:GenerateCryptoInfo failed. Retrying...: Problem saving Crypto to file. state = -4; Context = (3464).

This problem may occur if the CimClinet, CimHistory and CimKSM files are missing from the C:\document and settings\all users\application data\Nice\CryptoInfo folder on the workstation. This problem occurs as a result of a Windows EFS (Encrypting File System) configuration issue. The certificate must be renewed.
To check ScreenAgent: 1. Check if the CIMService.exe service is running. 2. Verify that CIMService.exe service has Local System Account privileges. 3. Verify that the service has privileges to write to the disk. 4. Verify that there is enough space on the disk. 5. Check if the CimClinet, CimHistory and CimKSM files exist in the C:\document and settings\all users\application data\Nice\CryptoInfo folder on the workstation. The size of the CimKSM file is 0 KB.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

172

ScreenAgent Not Working Properly

6. Check in the Event Viewer if there are any CIMService error messages. An error with Source: EFS and Description: EFS recovery policy contains invalid recovery certification indicates a problem with the site certificate.

7. Check if there are any error messages in the SAMS log. 8. Check the CryptoInfo folder properties: a.

Right-click on the CryptoInfo folder and select Properties.

b. Click Advanced.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

173

ScreenAgent Not Working Properly

c.

Select Encrypt contents to secure data. If you get the following error this indicates a problem with the site certificate.

9. Renew the certificate as described in Renewing Certificates on page 171.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

174

Ensuring Connection between SQL Server and Applications Server

Ensuring Connection between SQL Server and Applications Server
Sometimes the FQDN of the SQL Server in the SQL certificate differs from the FQDN configured in the Applications Server. When this is the case, the following error message appears in the log file:
The certificate's CN name does not match the passed value.

To correct this problem, you need to make sure that the FQDN of the SQL Server and its assigned FQDN in the System Administrator are the same.
To ensure a connection between the SQL Server and the Applications Server: 1. Ping the SQL Server from the Applications Server to get its FQDN in the Applications Server. 2. If the FQDN in the Applications Server differs from the FQDN in the SQL certificate, add the

FQDN of the SQL Server to the DNS. The connection between the SQL Server and the Applications Server is ensured.

Installing the CA certificate on Vista Workstations
See the Microsoft Vista Workstation Configuration Guide for information regarding installing the CA certificate on Vista workstations.

Chapter 6: Media Encryption Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

175

Blank page for double-sided printing.

7
Channel Mapping Troubleshooting Flow
This section provides quick solutions for common problems you face when defining Channel Mapping. The procedures in this section are designed for the seasoned user who is familiar with the NICE Perform system and the System Administrator application. The procedures in this section are designed to be directly solve the problem at hand. For complete instructions on using the System Administrator, see the System Administrator’s Guide.

Contents
Channel Mapping Troubleshooting Workflow ..........................................................179 Gathering Information ..............................................................................................179 Identifying a Channel Mapping Problem ..................................................................179 Verifying the Channel Mapping Configuration..........................................................181 Troubleshooting an Out of Memory Error.................................................................183 Troubleshooting Two Users Updating Channel Mapping Concurrently....................184 Troubleshooting Saving in a Multi Data Hub Environment.......................................184 Adding a New Agent ................................................................................................184 Other Troubleshooting Issues ..................................................................................185 Open Case Procedures ................................................................................................186 Open Case Procedures for System Administrator Parameters (CLS) .....................186 Open Case Procedures for the RCM .......................................................................187 Troubleshooting Scenarios .........................................................................................190 Error Message appears when Updating Configuration ............................................190 A New or Modified Mapping Definition Does Not Work............................................191 New Channel Mapping Definitions are not Saved....................................................191 Channel Mapping does not Work! ............................................................................191 Channel Mapping in System Administrator Differs from what is in Effect ................192 Channels Definitions: Logger does not Appear........................................................192 Channels Definitions: Input Channels do not Appear ..............................................193 Source Definitions: Cannot define Virtual Extensions ..............................................193 Source Definitions: Physical Switch does not Appear..............................................193
Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 177

Dynamic Mapping: Screen Logger does not Appear ...............................................194 Dynamic Mapping: Virtual Extensions......................................................................194 Deleting or Changing Logger, Switch, or CLS Definitions........................................194

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

178

Channel Mapping Troubleshooting Workflow

Channel Mapping Troubleshooting Workflow
If you experience a problem with Channel Mapping follow the steps below to isolate the problem.

• • • • • • • • • •

Gathering Information Identifying a Channel Mapping Problem Verifying the Channel Mapping Configuration Troubleshooting an Out of Memory Error Troubleshooting Two Users Updating Channel Mapping Concurrently Troubleshooting Saving in a Multi Data Hub Environment Adding a New Agent Other Troubleshooting Issues Open Case Procedures for System Administrator Parameters (CLS) Open Case Procedures for the RCM

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Identifying a Channel Mapping Problem
Check whether the problem is a Channel Mapping problem using one of the following methods.
To identify a Channel Mapping problem: 1.

Check for a Channel Mapping problem using ROD:
a. Generate a simple test call. b. Verify that you can see the test call in the Monitor application. If the call does not appear

in the Monitor application, see the Recording Troubleshooting Flow and the Interactions Center Installation Guide.
c. If the test call appears in Monitor, try to ROD the test call. If the test call is not recorded,

check the mapping.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

179

Channel Mapping Troubleshooting Workflow

For a trunk-side recording environment, add the participant trunk group and participant trunk number columns in Business Analyzer, and verify which trunk groups and trunk numbers were assigned to the interactions. Check the mapping for these trunks.
2.

Check for a Channel Mapping problem in the logs:
a. Generate a simple test call. b. Verify that you can see the call in the Interactions Center RCM log files. See the

Interactions Center Installation Guide
3.

Check for a Channel Mapping problem by querying the database for Channel Mapping exceptions:
a. Query the tblInteractionCatalog table to check over what period the calls occur. b. Get all the exceptions relating to interactions: SELECT * FROM tblExceptionXX, tblInteractionXX WHERE tblExceptionXX.iInteractionID = tblInteractionXX.iInteractionID c. Get all exceptions relating to recordings: SELECT * FROM tblExceptionXX, tblRecordingXX WHERE tblExceptionXX.iRecordingID = tblRecordingXX.iRecordingID d. Check the vcExceptionDetail field in tblException.

4.

Check for a Channel Mapping problem by checking for exceptions in Business Analyzer. Select the interaction and click on the Exceptions tab in the Details pane.

No 15

Exception Unmapped voice recording.

Description Unmapped voice recording.

Possible Cause Problem in the voice channel configuration.

Recommended Action Switch the logs to DEBUG. Check channels configuration. See if there is a mapping configured for this call. Check the RCM logs. Check what the RCM received in the Start request. If the allocation mode is by IP address, then check if the agent logged in. Check ports. Verify in RCM logs that the start request for recording the agent's screen contained the screen agent's IP.

16

Unmapped screen recording.

Unmapped screen recording.

The recording request was received with empty Station or IP address (depends on the screen allocation mode).

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

180

Channel Mapping Troubleshooting Workflow

Verifying the Channel Mapping Configuration
The Channel Mapping configuration is dependent on:

• • • •

Recording Mode: Total or Interaction based. Recording Type: Plain, Time Selective, Dynamic Matrix, Passive VoIP or Active VoIP Mapping Type: Static or Dynamic Recording Method: Extension or Trunk

To verify the Channel Mapping configuration: 1.

Check if changes were made to the site without making the necessary changes to the Channel Mapping configuration, for example:

• • • • • •

Logger was reinstalled. Channels were changed. Capture type was changed. Changes were made to the monitored devices is System Administrator without reflecting the changes in Channel Mapping. For Avaya DMCC, changes in virtual extensions were made without remapping all sources. For N+1, any change in the chain, for example, an addition of a Logger to a chain, converting all Loggers to 1+1, and so on, without reflecting the changes in Channel Mapping.

2.

For extension-side recording, verify the following:
a. Channel definition (Recording type = Plain or Time Selective, Summation Support and

Inserter Support).
b. Source definition (extensions). c. Static Channel Mapping definition. 3.

For trunk recording, if the call was recorded but with the wrong voice or silence, the trunk may be mapped to the wrong Logger channel. Verify that the mapping matches the physical connection to the Loggers. For trunk-side recording, verify the following:
a. Channel definition (Recording type = Plain, Time Selective or Digital Matrix, Summation

4.

Support and Inserter Support).
b. Source definition (trunks). c. For Plain or Time Selective channels, static Channel Mapping definition. d. For Digital Matrix channels, static Channel Mapping or dynamic Channel Mapping

definition.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

181

Channel Mapping Troubleshooting Workflow

5.

If you are using dynamic Channel Mapping a recording may fail if there is an attempt to record more calls simultaneously that there are assigned channels. Depending on your system you can:

• • •
6. 7.

Buy more channels. Ensure in your dynamic mappings that all sources are mapped to more than one Logger. For QA recording, try to change your rule percentage/timing.

Verify that all ACDs (Hunt Groups), IVRs, and Pickup Groups participating in the call are mapped to Logger channels. Check that the mapping details match the Logger configuration in the Logger summary file (\NTLOGGER\CONFIG\summary.txt), for example, total versus interaction-based recording, the Logger summation support definition (Summed or Co-PABX/Stereo), and so on.

8.

Use the Functional Tester to verify that the channel definitions on the Logger match the devices that were mapped to the channels, for example, summation and in the mapping and in the Logger Channel Mapping import file. See the High Density Logger Troubleshooting Flow and the Channel Mapping Guide for Importing Mapping Definitions. Depending on the integration, check that the mapping is according to the extension number, MAC #, Device Unique Identifier (DUI) or IP address. Guide.
NOTE: It may take up to 10 minutes for the RCM to apply the configuration when there are many mappings and Logger channels.
Chapter 7: Channel Mapping Troubleshooting Flow 182

9.

10. Verify that the mapping configuration was updated to the CLS. See the Channel Mapping

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Channel Mapping Troubleshooting Workflow

11. Check the RCM logs to verify that the mapping configuration was successfully updated to the

Loggers:

• •

Check that the mappings have been applied, and that all the relevant change commands have been sent to the Loggers. Search for ERROR/WARN/Exception messages during the execution of the configuration changes. Otherwise, the only way you will find the errors is when the mapping does not work, or specific changes are not applied, and you may lose recordings as a result.

Troubleshooting an Out of Memory Error
An Out of Memory error appears in System Administrator while defining Channel Mapping (that is, Internet Explorer is connected to the Applications Server). The System Administrator’s memory consumption increases after many configuration changes.
NOTE: The out of memory error does not necessarily indicate a problem with Channel Mapping and can also be caused by other parts of System Administrator.

To troubleshoot an Out of Memory error: 1. 2. 3.

On the Applications Server, in Task Manager, click the Processes tab, and check the memory usage for NICE System Administrator. On the client workstation, in Task Manager, click the processes tab, and check the memory usage for iexplore.exe. Check the performance counters:
a. In the Run window enter perfmon. b. Click Add. c. For each counter (Processor, Memory, Private Bytes, Threads, Handles,% CPU, Page

Faults / Sec), do the following:

• • • •

From the Performance Object list, select the Object. If enabled, select All instances. Select Select counters from list. Select a counter and then click Add.

d. Click Close. e. Click View Report and monitor the results and check for irregularities. 4.

On the Applications Server, create a memory dump file:
a. Navigate to <NICE Perform installation folder>\NICE Applications\Tools. b. Double-click NICE Services Configuration Manager.exe.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

183

Channel Mapping Troubleshooting Workflow

c. Click Dump

.

d. Click Yes. The Browse For Folder window appears. e. Select the folder that will contain the memory dump files and click OK. Memory dump

files are created.
5. 6. 7.

On the client workstation, create a dump file using the Windows ADPlus utility. For more information regarding ADPlus, see http://support.microsoft.com/kb/286350. Restart the Applications Server. Re-launch Internet Explorer.

Troubleshooting Two Users Updating Channel Mapping Concurrently
Prior to NICE Perform Release 3.1, in the unlikely event of two users simultaneously updating the Channel Mapping configuration, the changes saved by the second user did not take into account the first user’s changes, resulting in a potential conflict. In NICE Perform Release 3.1, a message appears informing the second user that their changes will be discarded. The Channel Mapping configuration will reload, and the second user may then make the required Channel Mapping configuration changes. This is called an atomic save.

Troubleshooting Saving in a Multi Data Hub Environment
In a Multi Data Hub environment, some of the information in nice_admin is replicated. If replication is down, you will be unable to save in System Administrator.
NOTE: This affects all the branches in System Administrator, not only Channel Mapping.

The atomic save (described in Troubleshooting Two Users Updating Channel Mapping Concurrently on page 184) in a Multi Data Hub environment relies on the Primary Data Hub being up.

Adding a New Agent
You may need to add a new agent.
To add a new agent: 1.

If required, for TDM Loggers, connect the audio input. For further information, see the NiceLog High Density Logger Installation and Configuration Guide.
Chapter 7: Channel Mapping Troubleshooting Flow 184

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Channel Mapping Troubleshooting Workflow

2.

In System Administrator > Channel Mapping (for further information, see the relevant Integration Guide):
a. Define Logger channels b. Define sources c. Map channels to sources d. Update the Interactions Center

3.

In the Users Administrator:
a. Add the new agent. See Adding a New User in the Users Administrator Guide. b. Add the agent to the relevant group. See Adding Group Members in the Users

Administrator Guide.
c. If required, give the user privileges. See Profile Privileges in the Users Administrator

Guide.
4.

In the Rules Manager, ensure that the new agent is covered by any relevant rules, for example, if the agent will be recorded according to a Scheduler rule, or whether the interactions of the agent’s group will be archived according to Storage rule. Make a test call:
a. Monitor the test call. b. If the test call does not appear in the Monitor, there may be a CTI Server to Driver

5.

problem, Driver to CLS problem or Interactions Center process problem. See the Recording Troubleshooting Flow.
c. If the test call appears in Monitor, try to ROD the test call. d. If the test call is not recorded, check the mapping.

Other Troubleshooting Issues
If you edit the configuration files:

• • • •

Ensure that you use a plain text editor such as Notepad without any options.
DO NOT use text wrap as this will corrupt your configuration file. DO NOT save the files in Unicode as this will cause the System Administrator to crash on

startup.
In any event, any change of configuration files should be done ONLY in consultation with NICE Customer Support!!!!

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

185

Open Case Procedures

Open Case Procedures
This section tells you what information to provide if you need to send a problem to NICE Customer Support for the following:

• •

Open Case Procedures for System Administrator Parameters (CLS) Open Case Procedures for the RCM

Open Case Procedures for System Administrator Parameters (CLS)
Data to be provided in Interactions Center Parameters (CLS Plug-ins) case reports, when contacting Customer Support:

Details regarding System configuration:

• • • • • • • • • • • •

Exact Interactions Center release, service pack, and installed Updates Any special site configuration, for example, Multi Data Hubs

Problem description: Expected behavior Actual behavior Actions taken Is the problem related to a specific configuration (such as a specific integration or mapping type) or does it effect all configurations? Does the problem reoccur after restarting the System Administrator or browser? Is there a work around? Are there also problems with additional parameters, such as Integrations parameters, Loggers parameters, DB Server parameters.

Provide specific examples and details Complete log files, in debug level from the server (located on the computer where the Interactions Center sits, under NICE Systems > Applications> ServerBin)

• • • • •

NiceApplications.CLSPlugin.log.*.txt

A backup copy of the nice_admin database (located on the computer where the database is installed) Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide a screenshot of an error if one is displayed. Provide Performance Counters of SysAdmin process:

• •

Private Bytes Threads
Chapter 7: Channel Mapping Troubleshooting Flow 186

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedures

• • • • • • •

Handles % CPU Page Faults/Sec All performance counters under the .Net Memory Category.

In case of Out Of Memory issues, please provide dump of SysAdmin process. In case of Online Configuration Update issues, please also provide full Interactions Center Logs. Provide any other relevant data, for example, memory usage or screen shots.

Open Case Procedures for the RCM
Data to be provided in RCM case reports, when contacting customer support:

Details regarding System configuration:

• • •

Exact Interactions Center release, service pack and NICE Updates (Update Packs) installed Details of all Loggers configured to each Interactions Center: version, capture type, number of channels, recording method, GMT and local time of the Logger. Mapping details for each Logger attached to the Interactions Center:

• • • • • • • • • • • •

Total/Interaction Based Recording method of Logger channels (for example, Passive VoIP, DMCC, Nortel DMS, Digital Matrix) Type of Sources and their number Static/Dynamic Mapping Connection between Logger and Interactions Center (LAN/WAN). If WAN, please indicate Bandwidth and Latency. N+1 chain details (when relevant)

Details from CTI plug-in on the drivers and the type of interfaces (for example, Avaya TSAP, Genesys, CISCO ICM etc.). Site's SBC/Gateway topology (when relevant). Number of ScreenAgents reporting to the Interactions Center (when relevant) Call Rate on the Interactions Center Machine Work hours of agents mapped to the Interactions Center, including peak hours (for example, hours of shifts, 24/7) Problem description:

Expected behavior
Chapter 7: Channel Mapping Troubleshooting Flow 187

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedures

• • • • • •

Actual behavior Actions taken

Provide specific examples and call details (for example, Interaction ID, Segment ID, extension number, Agent number, time of the call, relevant DB Records for the call). If the issue occurs in specific call scenarios, describe the scenario. Does the issue occur in a specific type of call? (for example, only internal calls) Complete log files, in debug level (located on the Interactions Center computer, under NICE
Systems > Interactions Center > Log):

• • • • • •

Call Server.log.*.txt RCM.log.*.txt

Complete log files, in debug level, of drivers attached to the Interactions Center. Complete log files of Loggers (including capture logs) attached to the Interactions Center. All RCM configuration backup files found under NICE Systems > Interactions Center >
Loggers

Files found under NICE Systems > Interactions Center > bin

• • • •

RCM.log.Persistency.soap RCM.exe.config

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide the following database tables in CSV format (query from before the time the problem occurred, preferably overlapping with the time in the provided logs):

nice_interactions

• • • • • • • • • • •

tblInteraction tblRecording tblParticipant tblException

nice_admin Channel Mapping tables

The following Performance Counters of the RCM.Exe process: Private Bytes Threads Handles % CPU
Chapter 7: Channel Mapping Troubleshooting Flow 188

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedures

• • •

Page Faults / Sec All Performance Counters under the .Net Memory Category.

Provide any other relevant data, for example, the users included, memory usage, screen shots.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

189

Troubleshooting Scenarios

Troubleshooting Scenarios
Troubleshooting techniques are provided for the following issues which can occur when your system is up and running. Error Message appears when Updating Configuration A New or Modified Mapping Definition Does Not Work New Channel Mapping Definitions are not Saved Channel Mapping does not Work! Channel Mapping in System Administrator Differs from what is in Effect Channels Definitions: Logger does not Appear Channels Definitions: Input Channels do not Appear Source Definitions: Cannot define Virtual Extensions Source Definitions: Physical Switch does not Appear Dynamic Mapping: Screen Logger does not Appear Dynamic Mapping: Virtual Extensions Deleting or Changing Logger, Switch, or CLS Definitions Procedure for Reassigning a Logger to a Different CLS

Error Message appears when Updating Configuration
Scenario:

You click Update Configuration and receive an error message, similar to the one below, that update was unsuccessful.

Solution 1: 1. 2. 3.

Check the Network connection between the Applications Server and the Interactions Center. Check the status of the CLS Monitor service on the Interactions Center. Click Update Configuration again.

Solution 2:

Only if Solution 1 fails, do the following:
Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 190

Troubleshooting Scenarios

1.

Restart the following:

• • •
2.

RCM TRS Service In Media Provider Controller Manager (MPCM - FLM) environments, restart the Call Server

Click Update Configuration again.

A New or Modified Mapping Definition Does Not Work
Scenario:

You add and/or modify Channel Mapping definitions. Then you click Update Configurations. One or more of the new or modified mapping definitions does not work.
IMPORTANT Some configurations require that you restart the RCM to complete the update process. For a complete list, see the Channel Mapping Guide. Solution: 1. 2. 3.

Stop the CLS service. Delete all RCM persistency files located under the Interactions Center/Loggers folder. Restart the CLS service.

New Channel Mapping Definitions are not Saved
If you click Save and your changes do not appear to be saved, the following occurred: Another user was updating Channel Mapping definitions at the same time that you were and clicked Save first.
What to do: 1. 2.

Refresh the screen to see the new changes. Recreate your changes.

See also Channel Mapping does not Work! on page 191.

Channel Mapping does not Work!
When you define Channel Mapping and save your definitions, they are saved only in the interface. To apply them to the system, you must update the CLS. For instructions, see the Channel Mapping Guide. See also New Channel Mapping Definitions are not Saved on page 191.
Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 191

Troubleshooting Scenarios

Channel Mapping in System Administrator Differs from what is in Effect
Scenario:

You open the System Administrator application and view the Channel Mapping definitions. They appear to be different from what you know to be currently running.
Cause:

Changes were made to the Channel Mapping definitions in the System Administrator and saved. However, Update Configuration was not applied, so therefore the Interactions Center was not updated and the new Channel Mapping definitions were not applied.

Channels Definitions: Logger does not Appear
Scenario 1

Loggers in the Channels Definition branch are identified by their Logger ID, not by the Logger name that appears in the Organization tree. Verify that you are searching for the Logger ID. Logger IDs are defined during Logger installation.
To determine the Logger ID: 1. 2.

In the System Administrator, expand the Loggers branch and select the Logger. The Logger ID appears in the General tab of the Logger definition.

Scenario 2

If the Logger is attached to more than one CLS, then it will not appear in the Channel Definition window. If the Logger is not attached to a CLS, then attach it now.
To attach a Logger to a CLS: 1. 2. 3. 4. 5. 6.

In the System Administrator, click the Loggers branch and define the Logger. Click the Channels tab to check that input channels are defined. Click the CLS branch and select a CLS. In the Logger tab, select the new Logger. Go back to the Channel Mapping branch and click the Channels definition branch. Select the new Logger. The Update Channels Details window opens. Loggers are identified here by the Logger ID, not by the Logger name you supplied. For Screen Loggers, you cannot edit channel details. However, you must open their definition and click Save.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

192

Troubleshooting Scenarios

Channels Definitions: Input Channels do not Appear
To display input channels: 1. 2. 3. 4. 5. 6. 7.

In the System Administrator, expand the Loggers branch and select the Logger. Click the Channels tab and move the input channels to the Configured Input Channels list. Save the Logger definition. Open the CLS servers branch, detach the Logger. (All mapping in the Channel Mappings branch will be removed.) Click the Channel Mapping branch and expand one of the branches. This step is required to update the database. Open the CLS servers branch and reattach the Logger. Return to the Channel Mapping branch and continue.

Source Definitions: Cannot define Virtual Extensions
When working with Virtual Extensions, the Location of both the Logger and the Connection Manager must be identified using the same method: either by IP Address or by Host Name.

Source Definitions: Physical Switch does not Appear
Scenario:

A Physical Switch is present in the CTI Integrations, but does not appear in the Sources definition branch.
Solution: To attach a Physical Switch: 1.

Define the Physical Switch on a CTI Interface:
a. Expand the CTI Integrations branch and click the CTI Interfaces branch. b. From the Actions menu, select New CTI Interface. c. Define the new Interface with the Physical Switch.

2.

Attach the Interface to a Connection Manager:
a. Expand the CTI Integrations branch and then expand the Connection Manager branch.

Select a Connection Manager.
b. Click the Interfaces tab and attach the Interface (physical switch). 3.

Attach the CLS to the Driver.
a. Expand the CTI Integrations branch and then expand the Drivers branch. Select a Driver. b. In the General tab, expand the Attach CLS section.
Chapter 7: Channel Mapping Troubleshooting Flow 193

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Scenarios

c. Attach the CLS to the Driver. 4.

Attach the Driver to the Connection Manager.
a. Expand the CTI Integrations branch and then expand the Drivers branch. Select a Driver. b. Click the Interfaces tab and select the Physical Switch/Connection Manager.

Dynamic Mapping: Screen Logger does not Appear
Scenario:

A Screen Logger in your Data Hub does not appear in the Dynamic Mapping branch.
Solution:

Screen Loggers do not need to be defined in the Sources branch. However, you must open their definition and click Save.
To add a Screen Logger: 1. 2. 3. 4.

Expand the Channel Mapping branch and click the Channels definition branch. Click the Screen Logger to expand its definition. Click Save. Return to the Dynamic Mapping branch.

Dynamic Mapping: Virtual Extensions
If you receive an error message when attaching channels pools, verify that the number of Virtual Extensions is greater than or equal to the number of Channels in the Channels pool. Confirm that the Location of both the Logger and the Connection Manager are identified using the same method: either by IP Address or by Host Name.

Deleting or Changing Logger, Switch, or CLS Definitions
Channel Mappings are dependant upon the Logger, Switch, and CLS definitions made in the System Administrator. Changing or deleting attachments from a switch to a CLS or from a CLS to a Logger, affect the recordings once the RCM is restarted, in the following ways.
If you... Delete a Logger definition -orDetach a Logger from a CLS Delete a switch definition -orDetach a CLS from a driver Then... All channel definitions and mapping configurations for that Logger are deleted. Source definitions remain. All Source definitions and mapping configurations for that switch are deleted. Channel definitions remain.
Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 194

Troubleshooting Scenarios

If you... Delete a CLS definition

Then...

• • •

All mapping configurations for that CLS are deleted. All Channel definitions for Loggers that were attached to the CLS are deleted. All Source definitions for switches to which the CLS was attached remain.

Attach a Logger to a second CLS after the Logger’s channels have been defined and mapped.

The Logger’s Channel Mapping channel definitions and mapping configurations are deleted.

To update Channel Mapping definitions, see the Channel Mapping Guide.

Procedure for Reassigning a Logger to a Different CLS
If you move a Logger from one Interactions Center to another, you must change its assignment in the System Administrator. After removing the Logger from its current CLS, you must update the Channel Mapping configuration and wait 10 minutes for the update to be complete. Then you can attach the Logger to the new CLS and continue. These steps are outlined in the following procedure.
To attach a Logger to a different CLS: 1. 2. 3.

In the System Administrator Organization tree, expand a Data Hub and select CLS Definitions > CLS Server. Select the CLS Server to which the Logger is currently attached and click the Loggers tab. Move the Logger from the Attached Loggers list to the Available Loggers list. The following message appears.

4. 5. 6.

Click Yes. Then click Save

.

Select CLS Definitions > Channel Mapping. Then click Update Configurations. Wait 10 minutes for the CLS to be completely updated!

IMPORTANT Wait 10 minutes before continuing with this procedure!
Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 195

Troubleshooting Scenarios

7. 8.

Select CLS Definitions > CLS Server. Then select the new CLS server and click the Loggers tab. Move the Logger from the Available Loggers list to the Attached Loggers list. Then click
Save

.

9.

Select CLS Definitions > Channel Mapping.

10. Define Channel Mapping for the Logger following the regular Channel Mapping procedures.

Chapter 7: Channel Mapping Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

196

8
Applications Troubleshooting Flow
This section contains troubleshooting workflow for the NICE Perform Application Server, including solving Desktop problems and troubleshooting Browser problems using Logs. A list of the services, config and log files is also provided.

Contents
Workflow........................................................................................................................198 Gathering Information ..................................................................................................199 Checking the Basics ....................................................................................................200 Troubleshooting a Communications Failure..............................................................202 Troubleshooting Communications Failure with Importer or Reporter ....................203 Checking the Applications Server Services...............................................................205 Checking the Installation Files ....................................................................................208 Checking the Log Files ................................................................................................209 Solving Desktop Problems ..........................................................................................217 Troubleshooting Browser Problems Using Logs ......................................................237 Solving ActiveX Control Error on the Applications Server ......................................241 Optimizing Client Server Performance .......................................................................242 Optimizing Client Server Performance on Computers without Internet..................244 Solving Email Failure ...................................................................................................246 Troubleshooting Crystal Reports ................................................................................250 Using the Process Explorer Utility..............................................................................251 Using the Desktop Tester.............................................................................................252

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

197

Workflow

Workflow
If you experience a problem with the Applications Server follow the steps below to isolate the problem.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

198

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

199

Checking the Basics

Checking the Basics
For Applications Server problems, check the following:
1. Troubleshoot a Communication Failure (see Troubleshooting a Communications Failure

on page 202)
2. Troubleshoot a Communication Failure between the Applications Server and the Importer

and/or Reporter Packages when using Language Pack (see Troubleshooting Communications Failure with Importer or Reporter on page 203)
3. Check the Applications Server Services (see Checking the Applications Server Services

on page 205)

• •

Use NICE Services Configuration Manager (see Using NICE Services Configuration Manager on page 206) Solve Application Services User Rights Problem (see Solving Application Services User Rights Problem on page 207)

4. Check the Installation Files (see Checking the Installation Files on page 208) 5. Check the Logs (see Checking the Log Files on page 209) 6. Solve Desktop Problems

• • • • • • • • • • • •

Verify the .NET Framework Versions (see Verifying the .NET Framework Versions on page 217) Solve Set Security Wizard Continues to Appear After Installation (see Set Security Wizard Continues to Appear After Installation on page 218) Solve Set Security Wizard Hangs or Displays an Error (see Solving Set Security Wizard Hangs or Displays an Error on page 221) Set Security Manually (see Set Security Manually on page 222) Set Security Manually Using Caspol (see Set Security Manually Using Caspol on page 224) Solve Post Installation HTTP Error (see Solving Post Installation HTTP Error on page 224) Verify the Applications are Not Loaded under the Internet Zone (see Verifying the Applications are Not Loaded under the Internet Zone on page 228) Configure the NICE Perform as a Trusted Web Site (see Configuring NICE Perform as a Trusted Web Site on page 228) Check NICE Perform Initialization Errors (see Checking NICE Perform Initialization Errors on page 229) Solve Service Unavailable Error (see Solving Service Unavailable Error on page 230) Disable Custom Errors (see Disabling Custom Errors on page 233) Solve ActiveX Control Error (see Solving ActiveX Control Error on page 233)
Chapter 8: Applications Troubleshooting Flow

200

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Checking the Basics

Troubleshoot Application Access (Power Management Issue) (see Troubleshooting Application Access (Power Management Issue) on page 234) Explorer Log (on the Client Machine) (see Explorer Log (on the Client Machine) on page 237) Desktop Client Logs (on the Client Machine) (see Desktop Client Logs (on the Client Machine) on page 237) IIS - WWW Log File (on the Applications Server) (see IIS - WWW Log File (on the Applications Server) on page 238) IIS - HTTPERR Log File (on the Applications Server) (see IIS - HTTPERR Log File (on the Applications Server) on page 239) Application Event Log File (on the Applications Server) (see Application Event Viewer (on the Applications Server) on page 239)

7. Troubleshoot Browser Problems Using Logs

• • • • •

8. Solve ActiveX Control Error on the Applications Server (see Solving ActiveX Control Error

on the Applications Server on page 241)
9. Solving Email Failure (see Solving Email Failure on page 246) 10. Troubleshoot Crystal Reports (see Troubleshooting Crystal Reports on page 250) 11. Optimizing Client Server Performance (see Optimizing Client Server Performance

on page 242)
12. Optimizing Client Server Performance on Computers without Internet (see Optimizing Client

Server Performance on Computers without Internet on page 244)
WARNING Manually modifying registry settings can cause your system to malfunction. This procedure should only be done by qualified personnel.

IMPORTANT Back up your config files before editing them. If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file. In any event, any change of configuration files should be done ONLY in consultation with NICE Support!

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

201

Troubleshooting a Communications Failure

Troubleshooting a Communications Failure
If you ping the server and it responds, but the communications failure persists, you may have a problem with the fully qualified domain name.
To identify the problem:

Ping the server, using its fully qualified name: <ServerName>.<Domain Name>. In order to resolve this, the FQDN should either be resolved by the DNS (requires assistance of IT) or added to the hosts file (C:\WINDOWS\system32\drivers\etc\hosts).

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

202

Troubleshooting Communications Failure with Importer or Reporter

Troubleshooting Communications Failure with Importer or Reporter
NOTE: This procedure applies only to NICE Perform systems that use the Language Pack.

If your NICE Perform system uses a Language Pack, there may be a conflict between the Applications Server and the Importer and/or Reporter Packages features. To prevent such conflicts, you must configure the Applications Server to apply the language settings to the default user profile.
To configure the language settings: 1. In the Start menu, navigate to Settings > Control Panel > Regional and Language Options.

The Regional and Language Options window appears.
2. Click the Advanced tab.
Figure 8-1 Regional and Language Options Window - Advanced Tab

3. In the Default user account settings area, select Apply all settings to the current user account and to the default user profile.

When you select the checkbox, the Change Default User Settings window appears.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

203

Troubleshooting Communications Failure with Importer or Reporter

Figure 8-2

Change Default User Settings Window

4. Click OK to accept the changes.

The Regional and Language Options window reappears.
5. Click OK. NOTE: If, after changing the language settings, you open the Regional and Language Options window again, the checkbox appears not selected. However, the language settings have been changed and are in effect despite the fact that the checkbox is cleared. 6. Restart your computer.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

204

Checking the Applications Server Services

Checking the Applications Server Services
The following services run on the Applications Server:

• •

NICE System Administrator: The main Applications Server service that controls the login

process to NICE applications. If this service is not running, users will be unable to login.
NICE Business Analyzer: Runs the Business Analyzer application (the main query

application for searching and playing back interactions) and the queries in the DB. There are some sub-services under this service.

NICE Locate Service: Relevant to SP3 and SP4 only. Runs under the NICE Business

Analyzer service, and finds the actual location of the audio in the preparation for playback (Loggers, Storage Center, or Media Library).

NICE Rule Engine: Reads all defined rules from the database, ensures that the rules run at the defined times, and notifies the relevant unit that needs to run the rule (Storage Center, Audio Analysis, and soon) with details of what to do. NICE Rules Manager Host: Supports the Rule Manager application where users create rules. NICE Monitor Server: Handles all ROD and monitor requests, and communicates with the

• • • • •

Interactions Center (directly to the Call Server).
NICE Audit Trail Service: Runs the Audit Trail, logging all user and server operations and

functions to the relevant database (nice_audit).
NICE Coaching Server: Manages all Coaching activity that is created by supervisors and evaluators that is delivered to Agents (via "My Universe"). NICE MyUniverse: Manages the My Universe application, and is responsible for collecting

the required information and displaying the relevant components, based on settings defined by them, and/or their managers.

• •

NICE Media Server: Relevant to SP3 only. Fetches media from the Loggers and Storage

Center. See the Storage Center Troubleshooting Flow.
NICE Playback Resource Manager: Relevant to SP3 and SP4 only. Manages the state of

each of the ICB (Instant Callback) resources, for example, ports configured for ICB, and so on, and displays an error if no resources available. See the Playback Telephony Server Troubleshooting Flow.

NICE Playback Administration: Relevant to SP4 and 3.1. Builds the play list. In SP4, replaces the NICE Media Server. In 3.1 this service incorporates the NICE Playback Resource Manager service and NICE Locate service functionality. NICE Playback Streaming: Relevant to SP4 and 3.1 only. Ensures that the media is delivered to the NICE Player (according to the configuration in the System Administrator). NICE Evaluations Form Server: Creates evaluation forms and the evaluations themselves,

• •

as well as runs the Form Designer and Evaluator. If the NICE Reporter is running on the Applications Server, the following services will also be running:

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

205

Checking the Applications Server Services

• • •

NICE Reporter Engine: Executes reports and allows the Reporter application to run. NICE Reporter Scheduler: Searches the DB for any active report schedules and, at the

scheduled times, sends a request to the Reporter Engine to run the report(s).
Crystal Report Application Server: Must be running on the Reporter server. This is a third-party program that NICE Perform uses as its underlying reporting technology. NOTE: The NICE Reporter usually runs on the Data Mart. To solve services problems:

• •

Use NICE Services Configuration Manager (see Using NICE Services Configuration Manager on page 206) Solve Application Services User Rights Problem (see Solving Application Services User Rights Problem on page 207)

Using NICE Services Configuration Manager
The NICE Services Configuration Manager enables you to manage the services on the Applications Server.

To configure the NICE Services Configuration Manager to run on startup: 1. Copy a shortcut from Applications\Tools\Nice Services Configuration Manage\NICE Services Configuration Manager.exe to C:\Documents and Settings\Administrator\Start Menu\Programs\Startup. 2. Right-click the Shortcut to Nice Services Configuration Manager shortcut link and select

Properties.
3. From the Run drop-down list, select Minimize. 4. Click OK.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

206

Checking the Applications Server Services

Solving Application Services User Rights Problem
A service may not start if the user does not have Log on as a service rights.
To solve the problem: 1. To check which user is running the service, use the NICE Services Configuration Manager or right-click My Computer > Manage > Services and Applications > Services, select Properties and click the Log On tab.

2. Open Control Panel > Administrative Tools > Local Security Policy. 3. Expand Local Policies > User Rights Assignment > Log on as a service. 4. Verify that user used to run the service has this permission.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

207

Checking the Installation Files

Checking the Installation Files
The files are typically installed on the D:\ partition of the Applications Server, located under D:\Program Files\Nice Systems\Applications. Sub-folders include:

• • • • • • • • •

ServerBin: Contains binaries and DLL files for server-based applications. ClientBin: Contains the binaries and DLL files needed by clients. Assemblies: Contains COM wrappers for many applications. Desktop: Contains, among other things, ASPX files for the login page. NICE Database Suite: Contains SQL scripts used by the applications. NICE_Help: Contains help files for the web applications suite. Tools: Contains tools used by installers and NICE customer support. Client Side Applications: Contains all installations files for client side applications (Player,

ROD, setsecurity.exe and so on).
KeyStorageManager: Contains the Key Storage Manager Web service assemblies and log

files.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

208

Checking the Log Files

Checking the Log Files
The configuration and log files for each application are found in either (or both) the ServerBin, and ClientBin folder, together with the binary files.

• • •

Set the Log Level in the Config File (see Setting the Log Level in the Config File on page 210) Set the Log Level in System Administrator (Setting the Log Level in System Administrator on page 209) Check the Logs (see Checking the Logs on page 214)

Setting the Log Level in System Administrator
To set the log level in System Administrator: 1. In System Administrator, verify that you are working in Technician mode: In the Organization tree, click Organization. Then select the Technician Mode checkbox and click Save. 2. In the Organization tree, select Applications > Configurations > General. Click the Logs Level tab.

3. Set the Master Level and File Level to Debug. 4. After troubleshooting ensure that you reset these values.

WARNING Failure to do so may cause system problems due to a performance overhead.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

209

Checking the Log Files

Setting the Log Level in the Config File
Each of the components on the Applications Server has config files in the following folder:
<Applications Server Installation Path>\ServerBin\.

Each config file includes a <log4net> section with several loggers:

Each Logger includes several appendices, that is, the types of log file output:

Component Application Messages Audio Analysis branch (System Administrator) Audio Analysis Search Controller Audit Trail Branches Business Analyzer Coaching

SA* Y Y

Configuration File NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.ApplicationMessages"> NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.AudioAnalysis">

Y Y Y

NiceApplications.AASearchController.Host.exe.config NiceApplications.AuditTrail.Host.exe.config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.Branches"> NiceApplication.UnifiedQuery.host.exe.config

Y

NiceApplications.CoachingTool.Host.exe.config NiceApplications.CoachingTool.Host.vshost.exe.config

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

210

Checking the Log Files

Component Encryption Plugin Evaluation Forms FTF Query Importer Insight Manager (Investigations) Interactions Server Plugin IP Phone Tester License Manager Lists Editor Locate Service
Relevant to SP3 and SP4 only

SA* Y Y Y Y Y

Configuration File NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.EncryptionPlugin"> NiceApplications.EvaluationForms.Host.exe.config NiceApplications.FTFQuery.Host.exe.config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.Importer"> NiceApplications.Investigations.Host.exe.config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.CLSPlugin">

Y Y Y

IPPhoneTester.exe.config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.LicenseManager"> NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.ListsEditor"> NiceApplications.UnifiedQuery.LocateService.Startup.exe.config

Location Service Locator Loggers Plugin Media Server
Relevant to SP3 only

NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.LocationService"> NiceApplications.UnifiedQuery.LocateService.LocateServiceHost. exe.config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.LoggersPlugin"> NiceApplications.Playback.MediaServer.Startup.exe.config NiceApplications.Monitor.Server.ServerStartup.exe.config NiceApplications.Monitor.Server.ServerStartup.exe.config Y NiceApplications.MyUniverse.Host.exe.config NiceApplications.MyUniverse.Server.ServerStartup.exe.config NiceApplications.Playback.Administration.Startup.exe.config

Monitor MyUniverse Playback Administration
Relevant to SP4 and 3.1 only

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

211

Checking the Log Files

Component Playback Resource Manager
Relevant to SP3 and SP4 only

SA*

Configuration File NiceApplications.Playback.ResourceManager.RMServerStartup.e xe.config

Playback Streaming Relevant to SP4 and 3.1 only. Playback branch (System Administrator) Retention Service Rule Engine Rules Manager Storage Center branch (System Administrator) Summary Page System Administrator System Administrator SNMP Users Administrator Y Y Y Y Y Y

NiceApplications.Playback.Streaming.Encoder.exe.config NiceApplications.Playback.Streaming.Startup.exe.config

NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.PlaybackPlugin"> NiceApplications.RetentionService.Host.exe.config NiceApplications.RuleEngine.Service.exe.config NiceApplications.RulesManager.Host.exe.config NiceApplications.SystemAdministrator.StorageCenterPlugins. config NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.SummaryPage"> NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.SystemAdministrator"> NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.SystemAdminSNMP"> Y NiceApplications.SystemAdministrator.Host.exe.config <logger name="NiceApplications.UsersAdministrator">

IMPORTANT Backup your config files before editing them. If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file. In any event, any change of configuration files should be done ONLY in consultation with NICE Support!!!!.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

212

Checking the Log Files

To set the log level in the config file: 1. Navigate to <Applications Server Installation Path>\ServerBin\. 2. In Notepad, open the relevant .config file. See Log File Config Files (see Setting the Log

Level in System Administrator on page 209).
3. Expand the <log4net> section. 4. Expand the required logger name section.

5. Set the debug level, by changing the values on both Level and Threshold to be defined as ALL. 6. Save and close the config file. There is no need to restart the Applications Server. 7. After troubleshooting ensure that you reset these values.

WARNING Failure to do so may cause system problems due to performance overhead.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

213

Checking the Log Files

Checking the Logs
The server log files are found in the <Applications Server Installation Path>\ServerBin\ folder unless specified otherwise.
Component Analytics Application Messages Audio Analysis branch (System Administrator) Audio Analysis Search Controller Audit Trail Branches Business Analyzer Coaching Encryption branch (System Administrator) Evaluation Forms FTF Query Importer Insight Manager (Investigations) Interactions Center - CLS branch (System Administrator) IP Phone Tester Log File NiceApplications.ClearSightApp.txt NiceApplications.ApplicationMessages.log NiceApplications.AAPlugin.log

AASearchController.log

AuditTrailLog.txt AuditTrailDefaultLog.txt NiceApplications.SystemAdministrator.Branches.log NBA.log NBADefault.log CoachingHostLog.txt NiceApplications.EncryptionPlugin.log

EvaluationFormsLog.txt NiceApplications.FTFOpenCalls.Log NiceAppliactions.FTFQuery.Log NiceApplications.Importer.log NiceApplications.InvestigationsApp.txt

NiceApplications.CLSPlugin.log

XmlAppsHost.log XmlAppDefaultLog.txt
Chapter 8: Applications Troubleshooting Flow 214

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Checking the Log Files

Component License Manager List Editor Locate Service Relevant to SP3 and SP4 only. Location Service Locator Loggers branch (System Administrator) Media Server
Relevant to SP3 only

Log File NiceApplications.LicenseManager.log NiceApplications.ListsEditor.log LocateDefaultLog.txt LocateLog.txt

NiceApplications.LocationService.log Locator.log LoggersPlugin.log

MediaServer.txt

Monitor MyUniverse MyUniverse Playback Administration
Relevant to SP4 and 3.1 only

MonitorLog.txt monitorDefaultLog.txt MyUniverseApp.txt MyUniverseBasicPlugin.txt PlaybackAdministration.txt NiceApplications.PlaybackAdministration.SystemFramework.log PlaybackRM.txt (3.1 only) LocateLog.txt (3.1 only) ResourceManagerDefaultLog.txt ResourceManagerLog.txt

Playback Resource Manager
Relevant to SP3 and SP4 only

Playback Organizer Playback Streaming
Relevant to SP4 and 3.1 only

NiceApplications.Workflow.Log NiceApplications.PlaybackStreaming.SystemFramework.log PlaybackStreaming.txt

Playback branch (System Administrator)

NiceApplications.PlaybackPlugin.log

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

215

Checking the Log Files

Component Reporter

Log File ReporterBin\ReporterManager.txt ReporterBin\ReporterEngine.txt ReporterBin\ReporterSchedulerLog.txt The Reporter application also includes logs within the application itself. The log provides information regarding the execution of reports. You can also export it to csv format. To view the Reporter logs: 1. In Reporter, click Administration. 2. Click Log. 3. Click Export to save the log files as a csv file.

Retention Service Rule Engine

RetentionService.log RuleEngine.ActionQueue.txt RuleEngine.SystemFramework.txt RuleEngine.txt

Rules Manager Summary Page System Administrator System Administrator SNMP Users Admin

NiceApplications.RulesManager.Host.exe.log NiceApplications.SummaryPage.log NiceApplications.SystemAdministrator.Host.exe.log NiceApplications.SystemAdministrator.SystemFramework.log NiceApplications.SystemAdministrator.SNMP.log

NiceApplications.UsersAdmin.log

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

216

Solving Desktop Problems

Solving Desktop Problems
There are several types of Desktop problems:

• • • • • • • • • • • • •

Verify the .NET Framework Versions (see Verifying the .NET Framework Versions on page 217) Solve Set Security Wizard Continues to Appear After Installation (see Set Security Wizard Continues to Appear After Installation on page 218) Solve Set Security Wizard Hangs or Displays an Error (Solving Set Security Wizard Hangs or Displays an Error on page 221) Set Security Manually (see Set Security Manually on page 222) Set Security Manually Using Caspol (see Set Security Manually Using Caspol on page 224) Solve Post Installation HTTP Error (see Solving Post Installation HTTP Error on page 224) Verify the Applications are Not Loaded under the Internet Zone (see Verifying the Applications are Not Loaded under the Internet Zone on page 228) Configure the NICE Perform as a Trusted Web Site (see Configuring NICE Perform as a Trusted Web Site on page 228) Check NICE Perform Initialization Errors (see Checking NICE Perform Initialization Errors on page 229) Solve Service Unavailable Error (see Solving Service Unavailable Error on page 230) Enable Custom Errors (see Disabling Custom Errors on page 233) Solve ActiveX Control Error (see Solving ActiveX Control Error on page 233) Troubleshoot Application Access (Power Management Issue) (see Troubleshooting Application Access (Power Management Issue) on page 234)

Verifying the .NET Framework Versions
Verify that the correct version of .NET Framework is installed:

• • •

SP 3: .NET 1.1 or .NET 2.0 or .NET 3.0 or .NET 3.5 SP 4 and Release 3.1: .NET 2.0 or .NET 3.0 or .NET 3.5 .NET 1.1:

Ensure that the following Updates are installed:

• • • • • •

.NET Framework 1.1 dotnetfx.exe, version: 1.1.4322.573 .NET Framework 1.1 hotfix NDP1.1sp1-KB867460-X86.exe, version 1.0.871.2738 .NET Framework 2.0 dotnetfx.exe, version: 2.0.50727.42 .NET Framework 2.0 hotfix NDP20-KB914460-X86.exe, version 1.0.36.79

.NET 2.0 and SP1:

.NET 3.0 and 3.5
Chapter 8: Applications Troubleshooting Flow 217

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Solving Desktop Problems

Set Security Wizard Continues to Appear After Installation
After installation, when opening applications in the browser the Set Security page appears even though the SetSecurity.exe application was run.

Problem Occurs on One Workstation
If this happens on one workstation this may be because:

• •

.NET Framework is not installed. Applications are running under the internet zone

If .NET framework is installed, check if the Applications work when running Desktop Tester (this runs the Applications without using the browser). For more information, see Using the Desktop Tester on page 252. If the Applications work when running Desktop Tester but do not work in the browser, possible causes include:

• • •

SetSecurity.exe was run with the wrong URL There is a problem with the browser download cache not been refreshed. The browser is running with .NET 1.1. (from Release 3.1)

To solve the problem for one workstation: 1. If you see the following error when running SetSeurity.exe, install the .NET framework by typing http://ServerName/Nice in Internet Explorer. You will be prompted to download

.NET.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

218

Solving Desktop Problems

2. Check if the Applications are running under the internet zone. See Verifying the Applications

are Not Loaded under the Internet Zone on page 228.
3. On the Workstation, run SetSecurity.exe with the correct URL. Verify that the URL you

entered is the same one used in the browser.
4. Clear the browser download cache: a.

Close all open Internet Explorer Browser windows. If these are left open, an error may occur when clearing the cache.

b. Right-click on the Internet Explorer icon, click Properties (in Internet Explorer 7.0 click Internet Properties and then click Delete), click Delete Files and then click OK. -or-

Open the Run window and run gacutil.exe -cdl.
c.

Verify that Download cache deleted successfully appears. If an error occurs, ensure that all browser windows are closed, and repeat Step b.

TIP: gacutil.exe can be found in either of the following locations: C:\WINDOWS\Microsoft.NET\Framework\<.NET version> C:\Program Files\NICE Systems\Applications\Tools You can drag-and-drop the file into the Run window. d. In Internet Explorer enter the URL again. Verify that you are entering the server name correctly. (URL format is: http://ServerName/Nice). 5. Verify/set the security manually. See Set Security Manually on page 222 or Set Security

Manually Using Caspol on page 224.
6. From Release 3.1: a.

Verify that .NET 2.0 is installed on the machine.
Chapter 8: Applications Troubleshooting Flow 219

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Solving Desktop Problems

b. Open C:\Program Files\Internet Explorer\iexplore.exe.config and check that the <supportedRuntime version="v2.0.50727" /> is not commented out. <configuration> <startup> <!--<supportedRuntime version="v2.0.50727" /> <supportedRuntime version="v1.1.4322" /> </startup> </configuration>

NOTE: The iexplore.exe.config file typically does not exist. This is not a problem. c.

Either update the file and remove the commenting, or delete the file.

Problem Occurs on All Workstations
If this happens on all workstations, the .customconfig Mime Type is not defined, that is, the installation failed to define it.
To solve the problem for all workstations: 1. On the Applications Server, from the command line run inetmgr. 2. Right-click on the local computer and select Properties. 3. Click MIME Types. 4. Click New. 5. Create a new MIME type with: a.

Extension: .customconfig

b. Mime type: .customconfig (in 2008 = customconfig/customconfig)

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

220

Solving Desktop Problems

6. Click OK.

7. Click OK. 8. No restart is required.

Solving Set Security Wizard Hangs or Displays an Error
When running the SetSecurityApp.exe application, it hangs or displays a Policy level Enterprise cannot be saved error message. The user logged in to Windows does not have the necessary permissions required to run the SetSecurityApp.exe application, that is, the user who is logged in to the machine belongs to the Users group in Windows.
To solve the problem: 1. Log in to Windows as an Administrator user, that is, a user who belongs to the Administrators group in Windows. 2. Run the SetSecurityApp.exe application.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

221

Solving Desktop Problems

Set Security Manually
The Set Security utility sets all NICE Perform downloaded assemblies to receive full trust. When you are unable to set security automatically, you can set it manually by following the procedures below.
To set security manually: 1. From the Start menu, select Settings > Control Panel > Administrative Tools > Microsoft .NET Framework Configuration.

IMPORTANT If there is more than one version of .NET Framework installed on your computer, be careful to select the relevant .NET Framework version. 2. In the tree, navigate to the following branch: Runtime Security Policy > Machine > Code Groups > All_Code > NICE Perform Code Group. 3. Right-click NICE Perform Code Group and click New.

4. In the Name field, enter a relevant name.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

222

Solving Desktop Problems

5. Click Next.

6. From the Choose the condition type for this code group drop-down list, select Site. 7. In the Site name field, enter a relevant name. 8. Click Next.

9. Leave the Use existing permission set drop-down list as FullTrust.
Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

223

Solving Desktop Problems

10. Click Next. 11. Click Finish.

Set Security Manually Using Caspol
An additional option for configuring the security setting is to run a script that uses Caspol.exe (Code Access Security Policy). Ensure that you use the version of Caspol that matches your version of .NET.
To set security manually using Caspol:

Create the following script and run it in the command window:
C:\WINDOWS\Microsoft.NET\Framework\<.NET Version>\Caspol.exe -machine -pp off -addgroup "All Code" -site <ComputerName>FullTrust -name "NICE CEM Code Group" -d "Allow full trust privileges to NICE CEM applications",

replacing <.NET Version> with the version of .NET that is installed on the client desktop, and the <ComputerName> with the NICE Perform server name in the format that is used to access it (Host Name/ IP /FQDN).

NOTE: If there is more than one version of .NET Framework is installed on your computer, run this script for each version.

Solving Post Installation HTTP Error
When trying to open application after the installation, a Server Error in 'NiceApplication/Desktop/WebPage' Application HTTP error appears. Possible causes:

ASP.NET is not installed/allowed on the machine

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

224

Solving Desktop Problems

The machine is not configured to work with ASP.NET 2.0 (from NICE Perform Release 3.1)

To solve the problem: 1. On the Workstation, check that ASP.NET 2.0 is installed by: a.

From the Start Menu, select Settings > Control Panel > Add Remove Programs. Double-click Application Server.

b. Click Add/Remove Windows Components. c.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

225

Solving Desktop Problems

d. Check that the ASP.Net subcomponent is selected.

e. f.

Click OK. Click Next. Select Start > Settings > Control Panel>Administrative Tools>Internet Information Services (IIS) Manager.

2. From NICE Perform Release 3.1, verify that ASP.NET 2.0 is configured on the IIS: a.

b. In the tree, right-click Web Sites > Default Web Site, and select Properties. c.

Click the ASP.NET tab.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

226

Solving Desktop Problems

d. Ensure that the ASP.NET version is set to 2.0.50727. If not, select 2.0.50727.

e. f.

Click OK. If the ASP.NET tab is missing from the properties window, or the ASP.NET version drop-down list does not include version 2.0.50727, the ASP.NET for .NET Framework 2.0 is not configured. In this case, register ASP.NET 2.0 on the IIS:

• •

Open the command window (Select Start > Run… then enter cmd in the Open field and click OK). In the command window, navigate to the .Net framework 2.0 directory, located under windows installation dir in the following path:
C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727

Type aspnet_regiis.exe -i and press <Enter>. The system will install the ASP.NET 2.0 and register it on the IIS.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

227

Solving Desktop Problems

Wait until the command finishes, go back to the IIS manager and make sure that the ASP.NET tab is added and version 2.0 has been selected on the combo box (as mentioned above).

Verifying the Applications are Not Loaded under the Internet Zone
Verify that the NICE Perform application is not loaded under the Internet zone on your Internet Explorer window.
To verify the applications are not loaded under the Internet zone: 1. Log in to NICE Perform. 2. Verify that Local Intranet or Trusted appears in the right side of the status bar (and not

Internet).

Configuring NICE Perform as a Trusted Web Site
It is recommended to define your NICE Perform Applications Server as a trusted site.
To solve the problem: 1. In Internet Explorer, from the Tools menu, select Internet Options. 2. Click the Security tab. 3. Select Trusted Sites and click Sites.

4. In the Add this Web site to the zone field, enter your server name in the following format: http://ServerName 5. Click Add. 6. Clear the Require Server verification checkbox. 7. Click OK.
Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 228

Solving Desktop Problems

8. Restart your computer, open Internet Explorer window, and enter the URL again. Verify that you are entering the server name correctly. (URL format is: http://Server Name/Nice).

Checking NICE Perform Initialization Errors
A NICE Perform initialization error appears when accessing NICE Perform. Possible problems:

• • • •

Solving Applications Server Configuration Problem (see Solving Applications Server Configuration Problem on page 229) Troubleshooting Remote Name Could Not be Resolved Error (see Troubleshooting Remote Name Could Not be Resolved Error on page 230) Troubleshooting Unable to Connect to the Remote Server Error (see Troubleshooting Unable to Connect to the Remote Server Error on page 230) Troubleshooting Serialization Problem (see Troubleshooting Serialization Problem on page 230)

Solving Applications Server Configuration Problem
If there is a problem accessing NICE Perform from the workstations, but the problem does not occur when accessing NICE Perform from the Applications Server, the Applications Server host name may not have been configured correctly in the System Administrator.
To solve the problem: 1. In System Administrator, verify that you are working in Technician mode: In the Organization tree, click Organization. Then select the Technician Mode checkbox and click Save. 2. In the Organization tree, select Applications > Applications Server, and select the relevant

Applications Server.
3. Verify that the ServerName appears for each Host Name/IP Address.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

229

Solving Desktop Problems

Troubleshooting Remote Name Could Not be Resolved Error
If a <Remote Name> Could Not be Resolved Error occurs when you are accessing the URL with a hostname/IP address, this may be a DNS problem. Remoting uses DNS names for communication and there is a problem resolving the DNS name mentioned in the error.
To solve the problem:

Check for a communication failure. See Troubleshooting a Communications Failure on page 202.

Troubleshooting Unable to Connect to the Remote Server Error
The problem occurs when the NICE System Administrator service is either stopped or has failed to initialize.
To solve the problem:

Verify that the NICE System Administrator service is running. If the service fails to start, check the system admin log file (NiceApplications.SystemAdministrator.Host.exe.log) for errors.

Troubleshooting Serialization Problem
The problem occurs when there is a problem with a Microsoft Windows update installed on the machine.
To solve the problem: 1. Check for a .Net Framework HF problem. See Verifying the .NET Framework Versions

on page 217.
2. Check for Windows Update problems. See the latest NICE Technical Note on Microsoft

Security Patches (issued monthly).

Solving Service Unavailable Error
When trying to open application a Service Unavailable error appears. Possible problems:

IIS AppPool Problem (see IIS AppPool Problem on page 231)

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

230

Solving Desktop Problems

Non-Paged Pool Memory Leak Problem (see Non-Paged Pool Memory Leak Problem on page 233)

IIS AppPool Problem
The problem occurs when there is a problem with the IIS AppPool.
To solve the problem: 1. Identify the problem: a.

In the command line, enter inetmgr to open IIS Manager. In the command line, enter eventvwr to open Event Viewer. when the AppPool identity is set to a specific user name, and the password for this user was updated in Windows, without been updated in the AppPool. Continue with Step 3 to resolve the problem.

b. Verify that DefaultAppPool is running. c.

d. Identify logon issues by looking for W3SVC Warnings under System. This may occur

e.

Look for W3SVC Errors. Unhandled exceptions from an application running under the pool can cause it to lock. Continue with Step 4. Start Internet Information Services (IIS) Manager.

2. Start the DefaultAppPool: a.

b. Expand Internet Information Services, expand <Server Name>, expand Application Pools, and then expand DefaultAppPool. c.

Right-click DefaultAppPool, and then click Start. Right-click DefaultAppPool, and select Properties.

3. Update the DefaultAppPool logon credentials: a.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

231

Solving Desktop Problems

b. Click the Identity tab.

c.

In the Configurable area, update the logon credentials. Right-click DefaultAppPool, and then click Stop. Right-click DefaultAppPool, and then click Start. Right-click DefaultAppPool, and select Properties.

d. Click OK. e. f.

4. Modify the DefaultAppPool health: a.

b. Click the Health tab.

c.

Clear Enable rapid-fail protection (this should only be used for an interim period until the source of the problem is identified).

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

232

Solving Desktop Problems

Non-Paged Pool Memory Leak Problem
A non-paged pool memory leak can cause a connection to the service to be denied.
To identify the problem: 1. Check the errors in the IIS HTTPERR log file to identify if the denial of service was caused by

a memory leak (See IIS - HTTPERR Log File (on the Applications Server) on page 239).
2. Open Task Manager. 3. Click the Processes tab. 4. In the View menu, select Select Columns. 5. Select Non-paged Pool. 6. Click OK. 7. Click on NP Pool to sort by non-paged pool. 8. Look for unusually high values. This indicates a leak. 9. Identify the application causing the memory leak and take appropriate action.

Disabling Custom Errors
Custom errors should be not be displayed by default. If custom errors are switched on a Server encountered an internal error appears.
To solve the problem: 1. In Notepad, open the relevant service .config file (according to the exceptions source). See

Setting the Log Level in System Administrator on page 209.
2. Search for the following: <system.runtime.remoting> <customErrors mode="On" /> </system.runtime.remoting> 3. Change the value to Off. 4. Recreate the problem and check the error returned.

Solving ActiveX Control Error
If you get an ActiveX Control error on the Desktop, set the appropriate options ActiveX options. For details on how to set the ActiveX Control options for the Applications Server, see Solving ActiveX Control Error on the Applications Server on page 241.
To solve the problem: 1. In Internet Explorer, from the Tools menu, select Internet Options.
Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 233

Solving Desktop Problems

2. Click the Security tab. 3. Select Local Intranet and click Custom Level. 4. Scroll down to ActiveX controls and plug-ins and select options as follows:

• •

Download signed ActiveX Controls - Prompt Script ActiveX controls marked safe for scripting - Enable

5. Click OK.

Troubleshooting Application Access (Power Management Issue)
If the Network Interface Card (NIC) of the Applications Server is set to power save mode, it can turn off the power on your Applications Server network card, prohibiting workstations from accessing the server. To avoid this situation, the power save mode on the NIC of the Applications Server must be turned off. Depending on your NIC, power save mode can be located in two different places. You must check both locations. If power save mode appears in both locations, you must turn it off twice!
To turn off Power save mode: 1. On the Applications Server, select Start > Settings > Network and Dial-up Connections. 2. Double-click the Local Area Connection icon.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

234

Solving Desktop Problems

3. Click Properties.

4. In the General tab, click Configure, and then click the Power Management tab.

5. Click the Power Management tab and clear all checkboxes. (This is the first location to

check. Some NICs might not have this tab.)

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

235

Solving Desktop Problems

6. Click the Advanced tab. Browse through the Property list and determine if there is a property

for power management. Disable it. (This is the second location to check. Some NICs do not have a property for power management listed here.)

7. Click OK.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

236

Troubleshooting Browser Problems Using Logs

Troubleshooting Browser Problems Using Logs
You can troubleshoot browser problems using various log files:

• • • •

Explorer Log (on the Client Machine). Desktop Client Logs (on the Client Machine) IIS - WWW Log File (on the Applications Server) Application Event Log File. See Application Event Viewer (on the Applications Server)

Explorer Log (on the Client Machine)
The Explorer log is on the client machine and requires a registry key to be updated. To troubleshoot client downloading and execution issues, enable the Internet Explorer logging mechanism.
To enable smart client downloading and security logging: 1. Open the Registry Editor. (Select Start > Run. In the Run window, enter regedit and click OK.) 2. Locate and select the following registry key: HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework 3. Select Edit > New > DWORD Value. 4. Name the new DWORD DebugIEHost. Assign it any non-zero value. 5. Select Edit > New > String Value. 6. Name the new String IEHostLogFile. Assign it the full path (including the file name) of the file in which you want to record the debug trace (for example, C:\Temp\IEDebug.log) 7. Open the browser and enter the requested NICE Perform URL. 8. After the Set Security page appears (or any other error page which is not the login page), check

the browser log file.

Desktop Client Logs (on the Client Machine)
To view the desktop client logs:

View the Desktop Client logs at C:\ Documents and Settings\[User]\ Local Settings\Application Data\NICE.
NOTE: This path may contain hidden folders.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

237

Troubleshooting Browser Problems Using Logs

Component Coaching

Log File C:\Documents and Settings\[user]\Application Data\NICE\CoachingPackageClient.txt and CoachingNugget.txt This file contains errors and info. The debug level cannot be changed for the client log file.

Evaluation Forms MyUniverse

C:\Documents and Settings\[user]\Application Data\NICE\EvalFormsDesignerLog.txt The debug level cannot be changed for the client log file. C:\Documents and Settings\[user]\Application Data\NICE\MyUniverseClient.log. MyUniverse client logs are only written if the Desktop Tester is activated. The debug level cannot be changed for the client log file.

System Administrator

C:\Documents and Settings\[user]\Application Data\NICE\SystemAdminClientLog.txt, LoginUtilsLog.txt and Autolock.log. This file contains errors only. The debug level cannot be changed for the client log file.

User Admin Desktop General

UsersAdminClient.log DesktopLog.txt ClientComponents.txt, ClientMessages.log

IIS - WWW Log File (on the Applications Server)
The Internet Information Services (IIS) World Wide Web Publishing Service (W3SVC), sometimes referred to as the WWW Service, manages the HTTP protocol and HTTP performance counters. This log file records the W3SVC errors, for example, crashes, recycle, ping failures, and so on.
To view the IIS www logs:

View the IIS www logs at C:\WINDOWS\system32\LogFiles\W3SVC1.

To verify the log location: 1. On the Applications Server, open Internet Information Services Manager (from the command line run inetmgr or from the Start menu, select Settings > Control Panel > Administrative Tools > Internet Information Services Manager).

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

238

Troubleshooting Browser Problems Using Logs

2. Right-click on the Default Web Site, select Properties and click the Web Site tab.

3. Verify that logging is enabled, by checking that Enable logging is selected. 4. To check the log file location, click Properties. 5. Under Log file directory, check the directory where log files are saved.

IIS - HTTPERR Log File (on the Applications Server)
This file enables you to diagnose why requests are rejected by HTTP.sys. Each entry includes URL and a brief description of the error. A list of the most common errors follows:

• • • • •

Connection_Dropped AppOffLine BadRequest QueueFull Connection_Abandoned_By_AppPool

To view the IIS HTTPERR logs:

View the IIS HTTPERR logs at C:\WINDOWS\system32\LogFiles\HTTPERR.

Application Event Viewer (on the Applications Server)
Check the Applications Server event logs using the Event Viewer.
To check the event log: 1. In the command line, enter eventvwr to open Event Viewer -or-

Right-click My Computer, click Manage, expand System Tools > Event Viewer.
Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 239

Troubleshooting Browser Problems Using Logs

-or-

Select Start > Settings > Control Panel > Administrative Tools > Event Viewer.
2. Check for the following types of errors:

• • • •

W3SVC worker process errors, for example, ASP.NET reports an unhealthy status Deadlock detected. .NET runtime errors, for example, FatalExecutionEngineException. ASP.NET errors, for example, side-by-side not supported within single AppPool and ASP.NET state server communication errors. To check for a problem with the IIS AppPool, identify logon issues by looking for W3SVC Warnings under System .

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

240

Solving ActiveX Control Error on the Applications Server

Solving ActiveX Control Error on the Applications Server
If you get an ActiveX Control error on the Applications Server, set the appropriate options ActiveX options. For details on how to set the ActiveX Control options for the Desktop, see Solving ActiveX Control Error on the Applications Server.
To solve the problem: 1. In Internet Explorer, from the Tools menu, select Internet Options. 2. Click the Security tab. 3. Select Local Intranet and click Custom Level. 4. Scroll down to ActiveX controls and plug-ins and select options as follows:

• •

Download signed ActiveX Controls - Prompt Run ActiveX Controls and Plug-ins - Enable

5. Click OK. To solve the problem using the Registry: WARNING Manually modifying registry settings can cause your system to malfunction. This procedure should only be done by qualified personnel. 1. Open a registry editor and navigate to the following key: HKEY_CURRENT_USER\Software\Microsot\Windows\CurrentVersion\Internet Settings\Zones\1

(Zone 1 is the Local Intranet Zone.)
2. Verify the settings as follows:

Setting Download signed ActiveX Controls Run ActiveX Controls and Plug-ins 3. Click OK.

Value Key 1001 = 1 Key 1200 = 0

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

241

Optimizing Client Server Performance

Optimizing Client Server Performance
You can increase the speed at which the pages of the NICE Perform Applications Suite appear in your Internet Explorer window.
To optimize client server: 1. In your Internet Explorer, navigate to Tools > Internet Options.

2. On the General tab, in the Temporary Internet files area, click Settings.

3. In the Settings window:

In the Check for newer versions of stored pages area, select Automatically.
Chapter 8: Applications Troubleshooting Flow 242

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Optimizing Client Server Performance

In the Temporary Internet files folder area, in the Amount of disk space to use field, enter 100.

4. Click OK. 5. Click the Advanced tab.

6. Browse to the Security section and clear Empty Temporary Internet Files folder when browser is closed. 7. Click OK.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

243

Optimizing Client Server Performance on Computers without Internet

Optimizing Client Server Performance on Computers without Internet
When opening the NICE Perform Login page without internet connection available, the client server will not be able to reach the VeriSign server to validate the NICE Perform digital signature. In this event, there will be a failure timeout. This failure may take several minutes. In order to work around this failure and to reduce failure timeout to a minimum, you can modify the security settings. You can do this by performing one of the following:

• •

Redirecting the Client Request to the Local Server Modifying the Security Settings in the Internet Explorer

Although both options have a security impact, redirecting the client request to the local server has less of a security impact as only validation of VeriSign certificates is skipped. Other certificates are validated.

Redirecting the Client Request to the Local Server
You can optimize the client server performance in computers without internet connection by redirecting the client request to the local server in one of two ways:

• •

By modifying the DNS Server By modifying the hosts file

To redirect the client request to the local server:

Modify the DNS Server or hosts file to include the following IP addresses and server names.
IP Address 127.0.0.1 127.0.0.1 Server Name crl.verisign.com CSC3-2004crl.verisign.com

NOTE: The server names may change if VeriSign updates their server names.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

244

Optimizing Client Server Performance on Computers without Internet

Modifying the Security Settings in the Internet Explorer
You can optimize the client server performance in computers without internet connection by modifying the security section in the Internet Explorer.
To modify the security section in the Internet Explorer: 1. In your Internet Explorer, navigate to Tools > Internet Options. 2. Click the Advanced tab.

3. Browse to the Security section and clear Check for publisher's certificate revocation. NOTE: Clearing Check for publisher's certificate revocation has a security impact as the revocation phase is skipped and is not recommended. 4. Click OK to close the Internet Options window.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

245

Solving Email Failure

Solving Email Failure
To solve an email problem:

• •

Verify the SMTP Service Installation and Configuration Troubleshoot SMTP

Verifying the SMTP Service Installation and Configuration
If emails are not sent, you need to verify that the SMTP service is installed, configured correctly, and running on the NICE Perform Applications Server.
To solve the problem: 1. Verify that the SMTP service is installed on the NICE Perform Applications Server. 2. From the Start menu, select Run. In the Run window, enter inetmgr. 3. Verify that the SMTP service is running.

4. Verify that the SMTP service is configured properly:

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

246

Solving Email Failure

a.

Right-click Default SMTP Virtual Server and select Properties.

b. In the General tab, in the IP address drop-down list, confirm that (All Unassigned) is

selected.
c.

Click the Access tab. Then click Relay and select All except the list below.

d. Click OK.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

247

Solving Email Failure

e.

Click the Delivery tab. Then click Advanced.

f.

Verify that:

• •

The Fully-qualified domain name is defined correctly. The Smart host text field displays the name of the SMTP server of your organization, in the format MyExchange.Domain.com.

g. Click OK.

Troubleshooting SMTP
Check that the SMTP logging is enabled and review the logs.
To troubleshoot SMTP: 1. In the Event Viewer, look for SMTP service messages under the Application Log and for

relevant messages under the Security Log.
2. Navigate to the following folder and verify that it does not contain any email messages: C:\Inetpub\mailroot\Queue. 3. If there are email messages in this folder, this is due to either the SMTP configuration, or an

antivirus issue (known issue). To remedy the antivirus issue, for example, for Norton AntiVirus:
a.

From the Start menu, select Programs > Symantec Client Security > Symantec Antivirus > Configure > Internet Email Auto Protect.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

248

Solving Email Failure

b. Clear the Enable Internet E-mail Auto-Protect checkbox.

4. From the Start menu, select Run. In the Run window, enter inetmgr. 5. Right-click Default SMTP Virtual Server and select Properties.

6. On the General tab, ensure that Enable logging is selected. 7. Click Properties. 8. Specify the settings that you want to use. You can specify settings to include log time period

and the location and the name of the log file.
9. Click OK, and then click OK. 10. Check the logs (the default location is C:\WINDOWS\System32\LogFiles\SMTPSVC\).

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

249

Troubleshooting Crystal Reports

Troubleshooting Crystal Reports
If Crystal Reports crashes you may need to update the Data Execution Prevention settings.
To troubleshoot Crystal Reports: 1. In the command line, enter eventvwr to open Event Viewer. 2. Check for errors. 3. Select My Computer > Properties. 4. Click the Advanced tab. 5. In the Performance area, click Settings. 6. Click the Data Execution Prevention tab.

7. Select Turn on DEP for all programs and services except those I select. 8. Select crystalras.exe. 9. Click OK to close the Performance Options window. 10. Click OK to close the System Properties window.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

250

Using the Process Explorer Utility

Using the Process Explorer Utility
To enable diagnosis of performance issues, the NICE Perform installation includes the Process Explore utility (located in the Tools directory). This utility displays the full image path, command line, environment variables, parent process, security access token, opened handles, loaded DLLs, and mapped files.

To run the Process Explorer:

Double-click procexp.exe.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

251

Using the Desktop Tester

Using the Desktop Tester
The Desktop Tester is used to run the Applications without the browser, enabling you to narrow down a troubleshooting problem to a browser/proxy issue.
To use the Desktop Tester: 1. (Optional) To use the Desktop Tester on a computer other than the Applications Server: a.

Copy the <Applications Server Installation Path>\ClientBin\ folder from the Applications Server to the computer. Modify the TesterPort to <HostName>:<SysAdminPort(default 62070)>, for example, Teddy:62070.

b. Open DeskTopTester.exe.config using Notepad. c.

d. Save the config file.

2. Double-click DeskTopTester.exe. Verify that the Applications run.

Chapter 8: Applications Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

252

9
Databases Troubleshooting Flow
This chapter describes some of the most common problems that may arise during and after installing the NICE Perform databases, and offers probable solutions.
IMPORTANT These procedures must be perform only by authorized personnel. For full assistance, contact NICE Systems Customer Services.

Contents
General Databases Troubleshooting Flow .................................................................255 General Considerations ...............................................................................................256 Checking the Windows Event Viewer .........................................................................256 Checking the SQL Server Error Logs and Events .....................................................256 Checking SQL Jobs......................................................................................................257 Checking the Connection to the NICE Perform Database Server ............................257 Checking the Database Connections .........................................................................257 Checking for Missing Databases ................................................................................258 Checking Free Space in the Database ........................................................................259 Verifying the Log on as a Service User Rights Assignment.....................................260 Checking the SQL Server Installation Folder.............................................................263 Monitoring Performance Objects and Counters........................................................265 Checking for Performance Problems .........................................................................266 Scanning the Disk for Errors .......................................................................................267 Checking Replication in Multi Data-Hub Environments............................................268 Changing the Database Passwords in SQL Authentication .....................................269

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

253

Checking the Database Backup ..................................................................................271 Increasing the Size of a Database...............................................................................273 Checking Performance Using the NICE Performance Collector ..............................274 Defining Autogrowth ....................................................................................................276 Using the NICE DBSpace Utility ..................................................................................278 Analyzing the Database Using NICE DBAgnose .......................................................284 General Data Mart Troubleshooting Flow...................................................................287 Data Mart Troubleshooting Issues ..............................................................................288

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

254

General Databases Troubleshooting Flow

General Databases Troubleshooting Flow
If you experience a problem with databases, the following workflow lists the steps to isolate the problem. Each step in the workflow is a section in this chapter.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

255

General Considerations

General Considerations
Before attempting to troubleshoot any database problems, it is advisable to consider the issues listed below. These will better enable you to analyze your problems and determine if the issues are indeed related to the database.

Is this a new installation or an upgrade of an existing system?

• • • • • •

For new installations, when was the installation performed? How long after the installation did the problems appear? For upgrades, when was the upgrade performed? Did the problems arise before or after the upgrade?

Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Are the circumstances similar? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Checking the Windows Event Viewer
In is often recommended to look for errors in the Windows Event Viewer. I/O errors are the most common when dealing with read/write errors to and from the database.
NOTE: These error messages are also displayed in the SQL Server Log files, as described in the next section.

Checking the SQL Server Error Logs and Events
You can use the Log File Viewer to check the SQL Server Agent logs, SQL Server error logs and Windows event logs.
To check the SQL Server Error Logs and Events: 1. Connect to the SQL Server Management Studio. 2. Navigate to your SQL Server 2005 instance. 3. Navigate to Management > SQL Server Logs. 4. Double click on any of the log files. 5. Use the Export, Search, and Filter options as required.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

256

Checking SQL Jobs

Checking SQL Jobs
Use the SQL Server to check that jobs are configured correctly and that they run properly.
To check SQL jobs: 1. In the SQL Server Management Studio, navigate to SQL Server Agent > Jobs. 2. Verify that the job durations are reasonable (may vary from job to job). 3. Verify that numerous jobs have not been configured to run at the same time. This is especially

relevant for sites in which numerous jobs run during off-hours.
4. Verify the job outcome, as a job may be successful even if internal steps have failed. 5. Check the job history to verify if all previous jobs have run successfully. Failed jobs are

identified by an error icon

.

Checking the Connection to the NICE Perform Database Server
Check the connection to the Database Server making sure you use the name configured on the NICE Applications Server.
To check the connection to the NICE Database Server: 1. On the Applications Server, ping the Database Server using the host name and IP address. -or2. On the Applications Server: a.

Navigate to D:\Program Files\NiceSystems\Applications\ServerBin.

b. In the NiceApplications.SystemAdministrator.Host.exe.config file, search for SystemFramework.DataAccess.ConnectionString value=<DB Server Name> .

This is the Database Server name defined in the application.
c.

Ping the Database Server using this name.

Checking the Database Connections
Use the SQL Server to check the database connections listed below.
To check the database connections: 1. In the SQL Server Management Studio navigate to Management > Activity Monitor. 2. In the right-click menu select View Process.
Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 257

Checking for Missing Databases

3. Check the Activity Monitor as follows: a.

Scroll to the Blocked By and Blocking columns. Any non-zero value in these two columns indicates an error (a deadlock). excessive connections.

b. Each row in this list represents a database connection. Verify that no database has 4. Filter the Activity Monitor for blocked processes as follows: a.

Click the Filter button. Click Apply filter, and then click OK.

b. Specify a filter with Blocking Type = Blocked. c.

Checking for Missing Databases
You can check for missing databases using either the System Administrator or the SQL Server.
To check for missing databases using the System Administrator: NOTE: It is recommended not to work in Technical Mode in order to prevent making unintentional changes to the database configuration. 1. In the System Administrator, navigate to Master Site > Database Servers > Databases and

verify that all five mandatory databases are selected:

• • • • •

nice_admin nice_audit nice_ca nice_interactions nice_rule

2. In addition, verify that all the optional databases that are supposed to be installed at your site

are also selected.
To check for missing databases using the SQL Server: 1. Connect to the SQL Management Studio. 2. In the Object Explorer, navigate to Databases and verify that the mandatory and optional

databases are listed and that the tables for each one contains data.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

258

Checking Free Space in the Database

Checking Free Space in the Database
There are several ways to check the free space in the NICE Perform databases.
To check free database space: 1. Check the NMS alarms. -or2. Connect to the SQL Server Management Studio. a.

Navigate to Databases and select the relevant database. In the General page check Space available.
NOTE: The space available relates to the free space of the entire database, and does not distinguish between the sizes of the Data files and the Log files.

b. From the right-click menu select Properties. c.

-or3. Right-click the database and select Reports > Standard reports > Disk usage. The usage is

displayed graphically and indicates both the Data files and the Log files separately.
-or4. Use the Nice DBSpace utility to check the database space (see Using the NICE DBSpace

Utility on page 278).
-or5. In the NICE System Administrator, navigate to Master Site > Database Servers > Databases and select each database to see the used and free space percentages. NOTE: It is recommended not to work in Technical Mode in order to prevent making unintentional changes to the database configuration.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

259

Verifying the Log on as a Service User Rights Assignment

Figure 9-1

NICE System Administrator

-or6. Use Nice DBSpace to check the database space.

Verifying the Log on as a Service User Rights Assignment
If a service does not start, make sure your user has the Log on as a service User Right Assignment. There are two steps:

• •

Verifying the Service Account (see page 260) Verifying Domain Users Assigned to the User Rights Assignments (see page 261)

Verifying the Service Account
To verify the service account: 1. In the Computer Management window, open the Services window, and select the service that

you cannot access.
2. From the right-click menu select Properties.

The Properties window appears.
3. Click the Log On tab.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

260

Verifying the Log on as a Service User Rights Assignment

Figure 9-2

MSSQLSERVER Properties Window

4. The domain account and the domain user appear in the This Account field.

Verifying Domain Users Assigned to the User Rights Assignments
To verify domain users assigned to the User Rights Assignments: 1. In the Control Panel navigate to Administrative Tools > Local Security Policy > User Right Assignments, and select Log on as a service.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

261

Verifying the Log on as a Service User Rights Assignment

Figure 9-3

Local Security Settings Window

2. From the right-click menu select Properties.

The Log on as a service Properties window appears.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

262

Checking the SQL Server Installation Folder

Figure 9-4

Log on as a service Properties Window

Domain User for NICE Services

Domain User for SQL Services

3. All domain users defined with Log on as a service policy are listed. Verify that the domain

user that appears in Figure 9-2 appears in the list.

Checking the SQL Server Installation Folder
If there are SQL I/O problems, verify if the SQL software may be installed on a compressed folder.
To check the SQL Server installation folder: 1. Run the NICE DBAgnose utility. Verify that OK appears in the status column of the Compressed Files row, as in Figure 9-5.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

263

Checking the SQL Server Installation Folder

Figure 9-5

NICE DBAgnose

-or2. In Windows Explorer: a.

Navigate to the SQL Server installation folder. Click the General tab.
Advanced Attributes Window

b. In the right-click menu, select Properties. c.

d. Click Advanced.
Figure 9-6

e.

Ensure that Compress contents to save disk space is not selected.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

264

Monitoring Performance Objects and Counters

Monitoring Performance Objects and Counters
You can use the Performance Monitor to check the activity on the SQL Server.
To monitor performance objects and counters: 1. In the Run window enter perfmon and click OK.

The Performance window appears.
2. Click the Add button

.

3. For each object listed in the table below, do the following:

Object
Processor Memory PhysicalDisk

Counters
%Processor Time Pages/sec %Disk Time Avg Disk Queue Length Avg Disk Write Queue Length Avg Disk Read Queue Length Disk Read/sec Disk Write/sec

SQL Server:BufferManager SQL Server:MemoryManager

Buffer catch hit ratio Page life expectancy SQL Cache Memory Total Server Memory

a.

From the Performance Object drop-down list, select the object listed in the table. the table above.

b. Select Select counters from list and select the counter corresponding to that object in c.

Select All instances if it is enabled. The object you selected is added to the Performance chart.

d. Click Add.

4. Click Close. 5. Click the View Report button

.

6. Check the results for irregularities.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

265

Checking for Performance Problems

Checking for Performance Problems
You can check for performance problems in several ways.
To check for performance problems: 1. Use SQL Profiler to run a trace: a.

In the Start menu, select Programs > Microsoft SQL Server 2005 > Performance Tools > SQL Server Profiler. Connect to the computer on which the SQL Server is installed. The Trace Properties dialog box appears.

b. In the File menu, click New Trace. c.

d. In the Use The Template name drop-down list, select TSQL. e. f.

Click the Events Selection tab. Select Show all events and Show all columns.

g. Scroll down to the TSQL. h. Ensure that the following events are selected:

• • •
i.

SQL:BatchStarting SQL:StmtCompleted. SQL:StmtStarting

For each of these event classes, click Column Filters. The Edit Filter window appears.
Figure 9-7 Edit Filter Window

j.

Select the Duration filter.
Chapter 9: Databases Troubleshooting Flow 266

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Scanning the Disk for Errors

k.

Select Greater than or equal to and enter a value. A value of 1000 milliseconds, for example, would cause the filter to show only those queries that take longer than one second to run. Remain in the Events Selection tab of the Trace Properties window, and scroll up to Security Audit.

l.

m. Clear Audit Login and Audit Logout. n. Click Run to run the trace.

The SQL Server Profiler enables you to view the trace data. Only information satisfying the filter is captured.
-or2. To check consistency and integrity for different tables and databases, in the SQL Query

Analyzer, run the following commands:
a.

To check database consistency, run the following query using the following syntax: dbcc checkdb (database name).

b. To check the tables, run the following query using the following syntax: dbcc checktable (table name). c.

To check table index fragmentation, run the following query using the following syntax: dbcc showcontig (<table name>) with all_indexes. Verify that the value for Scan Density [Best Count:Actual Count] is greater than 60%.

-or3. Use Nice Performance Collector to check the performance. See Checking Performance

Using the NICE Performance Collector on page 274.

Scanning the Disk for Errors
To scan the disk for errors: 1. On the SQL Server, run a disk scan on the drives where the data and log files are located. 2. Select the drive, and from the right-click menu select Properties. 3. Click the Tools tab, and then click Check Now.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

267

Checking Replication in Multi Data-Hub Environments

Checking Replication in Multi Data-Hub Environments
Use the following methods to check replication in Multi Data Hub environments.
NOTE:

• •

If you want to make any changes to the database schema of replicated or un-replicated tables, you must first shut down the replication for that site. When running the replication, the created scripts and log files are saved in Documents and Settings\[System Admin User]\Application Data.

To check replication in multi data hub environments: 1. In the NICE System Administrator, verify that the replication is defined using only the

machine names.
-or2. Check the connectivity between the Primary and Secondary Data Hubs. -or3. Connect to the SQL Server Management Studio. In the Object Explorer, navigate to Replication and check if there are components with an error. -or4. In the NICE System Administrator: a.

Navigate to Master Site > Database Servers > Databases and click the Replication tab.

b. Verify that no errors appear in the Status column. -or5. If the Primary and Secondary Data Hubs have different database structures, the replication will

fail at those specific points. To ensure that the replicated data is identical, run tests on all data hubs to ensure that additional information (business data, any external additions such as PS solutions) can be accessed from different sites.
-or6. Verify that the same version of the NICE Perform Database and the same updates are installed on all sites. Check the following tables in the nice_admin database: a.

The database version appears in the tblDatabaseDetails table.
Figure 9-8 tblDatabaseDetails

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

268

Changing the Database Passwords in SQL Authentication

b. The version of the updates appears in the tblDatabaseHotFix table. NOTE: Only those updates that were installed using an Update Pack appear in the tblDatabaseHotFix table; updates that were installed manually do not appear. -or7. When a Multi Data Hub environment is upgraded, the replication is automatically removed

during the procedures. After the upgrade has been completed, create the replication again. Then test that the replication is functioning properly.

Changing the Database Passwords in SQL Authentication
It is generally recommended to change the user passwords after installation. You can change the passwords after the DBSetup installation, after completing the entire site installation, or in a live production system. Since passwords are kept in several places, you must update the password using the System Administrator to ensure that the passwords are updated in all these locations. There are two passwords:

nice sa - this password the same privileges as the Microsoft SQL Server sa user, and is used

for the following tasks:

• • • •

Changing passwords Replication Linking servers (such as Data Mart)

The nice sa password must be identical across all SQL Servers.
nice - this password is used as a connection string for all NICE applications to the database. You can use a different nice password for each SQL Server.

The nice and nice sa passwords are stored in encryption in the tblDbPassword table in the nice_admin database. The nice password is also stored in encryption in the System Administrator configuration file.
Figure 9-9 System Administrator Configuration File

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

269

Changing the Database Passwords in SQL Authentication

Changing the nice sa Login Password
IMPORTANT

• •

You must work in Technician Mode to change the password. You must remove all replication before changing the password.

To change the nice sa login password: 1. In the System Administrator, under Master Site select Database Servers. 2. From the Actions menu, select SQL Server Security.

The SQL Server Security window for nice sa login appears.
Figure 9-10 SQL Server Security Window for nice sa Login

Make Sure You are in the Security Window for

nice sa Login

3. Enter and confirm the new password. 4. To use a blank password, select Blank Password. 5. Click OK.

The password is changed on all SQL Servers in both the Primary and the Secondary Data Hubs.

Changing the nice Login Password
IMPORTANT

• •

You must work in Technician Mode to change the password. You must remove all replication before changing the password.

To change the nice login password: 1. In the System Administrator, under Master Site > Database Servers, select the database

server on which you want to change the password.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

270

Checking the Database Backup

2. From the Actions menu, select SQL Server Security.

The SQL Server Security window for nice login appears.
Figure 9-11 SQL Server Security Window for nice Login

Make Sure You are in the Security Window for

Nice Login

3. Enter and confirm the new password. 4. To use a blank password, select Blank Password. 5. Click OK.

The password is changed on this SQL Server only.
NOTE:

• •

It may take up to one minute for the configuration file to be updated; do not use the new password during this time. If your site includes the Data Mart server, you must recreate links. For details see the NICE Perform System Administrator’s Guide.

Checking the Database Backup
You can check the database backup in System Administrator.
To check the database backup: 1. In System Administrator, navigate to Master Site > Database Servers: a.

Select the desired database and click the Backup tab.

b. Verify that all backup jobs are predefined and that there no errors in the Status column. -or2. Connect to the SQL Server Management Studio: a.

Navigate to SQL Server Agent > Jobs. that the last database backup job has failed.
Chapter 9: Databases Troubleshooting Flow 271

b. If an error icon appears in the Differential Backup job or the Full Backup job, it indicates

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Checking the Database Backup

-or3. Check if the source of the problem is the cassette as follows: a.

Check that there is a DAT/AIT cassette in the tape drive. Check that the cassette is not corrupted.

b. Check that the cassette is not in write-protect mode. c. -or4. Connect to the SQL Server Management Studio: a.

Navigate to SQL Server Agent > Jobs.

b. Select the desired job and select View History in the right-click menu. -or5. Run the job again: a.

Right-click the job and select Start Job.

b. Wait until the job finishes and check if the backup succeeded. -or6. Check if the source of the problem is the tape driver:

Open the operating system backup program and try to backup a file to the tape drive. If you succeed, the tape driver is working properly and the reason for the failure is not the tape.

-or7. Connect to the SQL Server Management Studio: a.

Select the nice_admin database. Configure the destination to be a local file on the hard disk and try to backup the database.

b. In the right-click menu select Tasks > Back Up. c.

If the backup succeeds, then the problem is not in the database.
-or-

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

272

Increasing the Size of a Database

8. Verify that the total database space used does not exceed the cassette capacity, as indicated in

the following table:
Cassette Type DAT DDS2 DAT DDS3 DAT DDS4 AIT1 AIT2 DAT72 DVD Panasonic 9576 Capacity 4 GB 12 GB 20 GB 35 GB 50 GB 72 GB 9.4 GB (double-sided)

Increasing the Size of a Database
To increase the size of a database: 1. Connect to the SQL Server Management Studio. 2. Under Databases, select the database for which you want to increase the size. 3. From the right-click menu, select Properties.

The Database Properties window appears.
4. Select the Files page.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

273

Checking Performance Using the NICE Performance Collector

Figure 9-12

Database Properties Window - Files Page

5. In the Initial Size column, click inside the cell for either the data file or the log file. 6. Enter the desired size for the database. 7. Click OK.

Checking Performance Using the NICE Performance Collector
This tool can also be found on the Tier 4 Team Site. Ensure that you are using the newest version which also supports SQL Server 2005. This tool collects SQL Profiler and Performance Monitor traces.
To check performance using the NICE Performance Collector: 1. On the NICE Database Server, navigate to Program Files\Nice Systems\Applications\Tools\Performance Collector. 2. Double-click Nice Performance Collector.exe. 3. The Nice Performance Collector window appears.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

274

Checking Performance Using the NICE Performance Collector

Figure 9-13

NICE Performance Collector

4. In the SQL Server field, enter the SQL Server name or the IP Address. 5. Select the authentication method you are using. 6. Click Get Data/Log Drive.

The program automatically detects the location of the data and log files and enters them in the respective fields.
7. In the Destination Folder field, define a location for the files which the Nice Performance

Collector creates.
8. In the schedule area, define a start and stop time. 9. Click Start.

Two files are created, one for counters and one for SQL Profiler.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

275

Defining Autogrowth

Defining Autogrowth
To define autogrowth: 1. Open the Microsoft SQL Server Management Studio. 2. Under Databases, select the database for which you want to define the autogrowth.
Figure 9-14 Microsoft SQL Server Management Studio

3. In the right-click menu select Properties.

The Database Properties window appears.
4. In the Select a Page area, select Files.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

276

Defining Autogrowth

Figure 9-15

Database Properties Window

Autogrowth Parameters Buttons

5. In the Autogrowth column, select either the Data file or the Log file and click the corresponding Autogrowth Parameters button. 6. The Change Autogrowth window appears.
Figure 9-16 Change Autogrowth Window

7. To enable autogrowth, select Enable Autogrowth. -or-

To disable autogrowth, clear Enable Autogrowth.
Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 277

Using the NICE DBSpace Utility

8. If you enabled autogrowth: a.

Under File Growth select either In Percent or In Megabytes and select the desired parameter from the corresponding drop-down box.

b. Under Maximum File Size select either Restricted File Growth and choose the desired parameter from the corresponding drop-down box, or select Unrestricted File Growth. 9. Click OK.

The parameters you defined appear in the Autogrowth column of the Properties window.

Using the NICE DBSpace Utility
The DBSpace utility enables you to verify database space usage and to monitor its status. DBSpace offers the following features:

• • • • •

Indicates databases that have less than 20% space available. Shows the used and free disk space for each database. Shows the used and free disk space for data and log files. Shows all tables in each database and their current status. Predicts the approximate number of days that remain until the tables reach full capacity.

IMPORTANT You must run the DBSpace utility locally. Therefore, before running the DBSpace utility, you must copy it to the NICE Perform server(s) on which the NICE Perform databases are installed.

To run the DBSpace utility: 1. Copy the DBSpace utility from the NICE Perform Applications Server to any convenient

location on the NICE Perform server.
2. Extract the files from the Nice DBspace.zip file. 3. In the folder containing the extracted files, double-click Nice DBspace.exe.

The Nice DBSpace window appears.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

278

Using the NICE DBSpace Utility

Figure 9-17

Nice DBSpace Window - Data/Log Tab
Low Space Databases Area

SQL Server Connection Area

Database Status Area

4. In the SQL Server field, enter the SQL Server name or the IP Address, or select the SQL

Server from the drop-down list. Select either Use Windows Authentication or Use SQL Authentication. If you selected SQL Authentication, enter your SQL Server Login name and the SQL Server password in the Login and Password fields respectively.
NOTE: For SQL Authentication only: By default, the sa Login password is blank. It is highly recommended to define a unique sa Login password for your site. However, if you did not define a unique sa Login password, leave the Password field blank. 5. Click the Connect button.

The Database Status area is populated.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

279

Using the NICE DBSpace Utility

6. In the Data/Log tab, select either Data to see the status of the SQL Server data files or Log to

see the status of the SQL Server log files. The DBSpace utility displays the following: Area
Low Space Databases

Status
Lists all databases in which the amount of free space is less than 20%, and the percentage of the remaining free space. Displays the amount of used space (displayed in blue) and free space (displayed in green) on the drive on which either the Data files or the Log files are located. Displays the amount of used space (displayed in blue) and free space (displayed in green) in each database in the NICE Perform Database Suite. The status of each database is displayed both as a table and a graph. TIP: Use the scroll bar under the table to display the portion of the table not visible.

Drive Status Area

Databases Status Area

NOTE: The DBSpace utility does not display the status of the databases in real time. To refresh the display, click the Connect button. 7. Click the Tables tab.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

280

Using the NICE DBSpace Utility

Figure 9-18

Nice DBSpace Window - Tables Tab

8. From the Databases drop-down list, select the desired database.

The Tables list displays all the tables in that database and the status of each table. To display fragmentation details, double-click the desired table. The fragmentation details appear in the Fragmentation area.
9. Click the Predict tab.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

281

Using the NICE DBSpace Utility

Figure 9-19

Nice DBSpace Window - Predict Tab

10. The Predict table gives an estimate of how much time remains until each of the listed tables

reaches full capacity. The calculations are based on the parameters listed at the top of each column.
11. Click the Activity tab.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

282

Using the NICE DBSpace Utility

Figure 9-20

Nice DBSpace Window - Activity Tab

12. In the Refresh Rate drop-down box select the desired number of seconds and click the Start

button. To see the most relevant connections of any of the databases, click the desired database. To see all the connections of that database click the Show All button.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

283

Analyzing the Database Using NICE DBAgnose

Analyzing the Database Using NICE DBAgnose
The DBAgnose utility checks your NICE Perform Database Suite and SQL Server configurations, and offers an analysis of many of the potentially serious database problems. By running the DBAgnose utility you ensure that your database and the SQL Server are configured properly.
IMPORTANT You must run the DBAgnose utility locally. Therefore, before running the DBAgnose utility, you must copy it to the NICE Perform server(s) on which the SQL Server is installed.

To run the DBAgnose utility: 1. Copy the DBAgnose utility from the NICE Perform Applications Server to any convenient

location on the NICE Perform server.
2. Open the Nice DBAgnose folder you just copied and double-click Nice DBAgnose.exe.

The Nice DBAgnose window appears.
Figure 9-21 Nice DBAgnose Window

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

284

Analyzing the Database Using NICE DBAgnose

3. In the SQL Server field, enter the SQL Server name or the IP Address, or select the SQL

Server from the drop-down list. Select either NT Authentication or SQL Authentication. If you selected SQL Authentication, enter your SQL Server Login name and the SQL Server password in the Login name and Password fields respectively.
NOTE: For SQL Authentication only: By default, the sa Login password is blank. It is highly recommended to define a unique sa Login password for your site. However, if you did not define a unique sa Login password, leave the Password field blank. 4. Click the Diagnose button.

The DBAgnose utility checks the SQL Server configurations.
Figure 9-22 Nice DBAgnose Window

5. The DBAgnose utility displays the status of each SQL Server configuration. IMPORTANT Some of the displayed configurations may not be applicable to the database migration, and you may be able to successfully conclude the migration procedures despite some warning or even error messages. For details or assistance contact NICE Systems Customer Services.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

285

Analyzing the Database Using NICE DBAgnose

NOTE: You can save the results by clicking the Save Results button. You can then print them out or email them to NICE Systems Customer Services. TIP: For a detailed description of each configuration, place your cursor at the beginning of the row; a pop-up window appears displaying the details of that configuration.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

286

General Data Mart Troubleshooting Flow

General Data Mart Troubleshooting Flow
If you experience a problem with the Data Mart, the following workflow lists the steps to isolate the problem. Each step in the workflow is a section in this chapter.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

287

Data Mart Troubleshooting Issues

Data Mart Troubleshooting Issues
Data Mart Troubleshooting covers the following topics:

• • • • •

Checking the Database Size (see page 288) Recreating Links (see page 288) Checking the Data Mart Database Version (see page 289) Checking Connectivity Between the Database and Data Mart Servers (see page 289) Checking a Data Mart Job Failure (see page 290)

Checking the Database Size
Check to see if the size of the database is sufficient. If necessary, increase the database size. For details, see Checking Free Space in the Database on page 259.

Recreating Links
To recreate links to the Data Mart: 1. In the System Administrator, expand the Master Site > Data Marts and select the required

Data Mart server.
2. Click the Sites tab.
Figure 9-23 Data Marts - Sites Tab

3. Click Recreate Links.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

288

Data Mart Troubleshooting Issues

Checking the Data Mart Database Version
Ensure that the database version is the same on all sites.
To check the Data Mart database version:

Verify that the database version in the tblDatabaseDetails table of each database is identical.
Figure 9-24 tblDatabaseDetails

Checking Connectivity Between the Database and Data Mart Servers
The Data Mart adds the Database Server as a linked server. It is crucial that these two servers can communicate with one another.
To check connectivity between the Database and Data Mart servers: 1. In the SQL Server Management Studio expand the Security > Linked Servers branch.
Figure 9-25 Linked Servers Branch

2. From the Data Mart server, ping the Database Server.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

289

Data Mart Troubleshooting Issues

Checking a Data Mart Job Failure
To check a Data Mart job failure: 1. Check for error messages in Windows Event Viewer. -or2. Check for error messages in the SQL log files. -or3. Check the job history to see which step failed and why. This isolates the problematic domain

or problematic communication with the database.
a.

In System Administrator, expand the Master Site > Data Marts and select the required Data Mart server. In the SQL Jobs area, select the required job. The job steps are detailed in the History area.
Figure 9-26 Monitor Tab

b. Click the Monitor tab. c.

Chapter 9: Databases Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

290

10
Customer Feedback Installation Troubleshooting Flow
This section provides a review of some information that will aid the installer in understanding the Customer Feedback workflow. Also provided is a workflow recommending how to proceed when troubleshooting a Customer Feedback system.

Contents
General Information .....................................................................................................292 Troubleshooting Customer Feedback ........................................................................294 Troubleshooting Flow Expanded..............................................................................295 Troubleshooting - General Issues......................................................................295 Troubleshooting - DTMF Routing ......................................................................296 Troubleshooting - CTI Info or Business Data Routing .......................................298 Surveys and Interactions are not Properly Correlated .............................................301

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

291

General Information

General Information
The following information is made available to help the installer troubleshoot the NICE Customer Feedback system. For additional tips, see Troubleshooting Customer Feedback on page 294. For NICE Customer Feedback to function properly, certain information must be made available from the PBX.
Binding between call and survey

To bind a call to a survey, the Telephony Services must receive a call event from the CLS (CAPI). In order to receive the events, the Feedback extensions must be monitored by the drivers. In most cases, the Telephony Services uses the Compound ID of the call to identify its previous segments (segments that have the same Compound ID). It then binds the survey to one of segments, choosing either the first or last segment, according to the Feedback configuration in the System Administrator. For Avaya Automatic Routing, the Telephony Services cannot use the Compound ID since the Feedback call and the previous calls have different Compound ID values. In this case, the Telephony Services uses the Avaya UCID value to bind the calls. It is important that the PBX reports the UCID to the driver and that the CLS reports the UCID in its events table.
Dynamic loading of surveys

There are two main ways to dynamically load a survey:

By DTMF - the DTMF is sent by the switch to the Feedback extension immediately after the call to the Feedback extension is established. For a sample of the System Administrator definition, see Figure 10-1 on page 293.
-or-

By Call Data (DNIS, Phone Number, VDN or Business Data) - the call data is received in events reported by the CLS. If the Feedback extensions are not monitored by the drivers, events will not be received and surveys cannot be loaded. For a sample of the System Administrator definition, see Figure 10-2 on page 293.

Environment Setup

The following should exist in your NICE Perform environment:

Playback Server extensions which are allocated for Feedback must be monitored by the driver. Use the Driver Tester for verification. The important thing to check is that events such as Start and End Call from agent extensions as well as from playback extensions are monitored. For Avaya Automatic Routing, the PBX should be configured such that the UCID information is enabled. This can also be verified using the Driver Tester. If the PBX is sending the information correctly, the CLS will have to save it in the database. For this to happen, a value must be defined for the for the vcPBXUniversalCallInteractionId field in the Business Data tab of the CLS branch of the System Administrator application. The VDN architecture should be known and Telephony Services extension should be configured to auto-login as described in the Release Notes.
Chapter 10: Customer Feedback Installation Troubleshooting Flow 292

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

General Information

Configuration in the System Administrator

The following are sample configurations in the System Administrator application:

By DTMF - in the Ports tab of the Telephony Services > Server > Feedback branch there must be at least one port defined with an extension number that is configured as Feedback.
Figure 10-1 System Administrator - Feedback Branch, Ports Tab

By Call Data - in the Surveys tab of the Telephony Services > Server > Feedback branch, surveys must be attached to a DTMF for Avaya Automatic Routing or the DNIS number for Manual Transfer.
Figure 10-2 System Administrator - Feedback Branch, Surveys Tab

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

293

Troubleshooting Customer Feedback

Troubleshooting Customer Feedback
In the event that you need to troubleshoot your Customer Feedback system, use the following flowchart and accompanying tables to locate the source of your problem and promptly correct it. Details appear in Troubleshooting Flow Expanded on page 295.
Figure 10-3 Customer Feedback Troubleshooting Flowchart

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

294

Troubleshooting Customer Feedback

Troubleshooting Flow Expanded
In the event that you need to troubleshoot your Customer Feedback system, the troubleshooting flowchart is explained in the following tables. See the accompanying flowchart to locate the source of your problem and promptly correct it. The flowchart appears in Troubleshooting Customer Feedback on page 294. Troubleshooting - General Issues on page 295 Troubleshooting - DTMF Routing on page 296 Troubleshooting - CTI Info or Business Data Routing on page 298

Troubleshooting - General Issues
Table 10-1: Troubleshooting - General Issues # 1 Problem Surveys are not assigned to an agent in the Business Analyzer Feedback. Symptom The survey was done correctly. In the Database table tblPbsNifSurveyResult there are values in the iCompundID and iInteractionID, however the iUserID field is set to -99. Possible Cause & Solution Make sure there is an agent defined in the Users Administrator, and that the agent identity (agent ID or extension number) matches the parameters of the call that preceded the survey. In the following line, taken from the log, the agent should be either associated with extension 40005 or agent ID 46149:
INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46149’ IsInteractionInitiator=False PhoneNum=40005 UniqueDeviceID=

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

295

Troubleshooting Customer Feedback

Table 10-1: Troubleshooting - General Issues (Continued) # 2 Problem An error message “Prompt file is missing” is played at the beginning of, or during a survey. Surveys can be viewed in the Business Analyzer Feedback, but they are not attached to any interactions. Symptom The call will be disconnected by the Telephony Services and there will be a new record for the survey in the database. The value of fltSurveyScore field depends on the total weight of the questions answered. If the total weight is above the minimum weight, the actual score will be saved. Otherwise, the value will be -1. A record exists in the database table tblPbsNIfSurveyResult. The iCompoundID field contains a value but the nvcUCID, iInteractionID and iUserID are empty. Furthermore, a search for interaction in the vwCallInteraction with the iCompoundID value returns an interaction which has a blank vcPBXUniversalCallInteractionID. Possible Cause & Solution A prompt file is missing. A new prompt file must be recorded using the Prompt Recorder.

3

The UCID is not reported to the driver by the PBX. The PBX should be configured to pass this value.

Troubleshooting - DTMF Routing
Table 10-2: Troubleshooting - DTMF Routing # 4 Problem Surveys can be viewed in the Business Analyzer Feedback, but they are not attached to any interactions. Symptom A record of the call appears in the database table tblPbsNifSurveyResult. However the fields nvcDNIS, nvcUCID, iCompoundID, iInteractionId, iUserID and iSwitchID have empty values. The following messages can be found in the Telephony Services log:
INFO ; [CFBasicActivity.OnHasNextQuestion] – Line[1] Quit questions loop … WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI

Possible Cause & Solution The Feedback extensions are not monitored by the driver. In this case no CAPI event is received by the Telephony Services. The Feedback extensions should be added to the monitored devices list in the Integrations branch of the System Administrator application.

interaction was found

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

296

Troubleshooting Customer Feedback

Table 10-2: Troubleshooting - DTMF Routing (Continued) # 5 Problem Surveys can be viewed in the Business Analyzer Feedback, but they are not attached to any interactions. Symptom A record of the call appears in the database table tblPbsNifSurveyResult. However, the fields nvcDNIS, nvcUCID, iCompoundID, iInteractionId, iUserID and iSwitchID have empty values. The following messages can be found in the Telephony Services log at the start of the survey:
INFO ; [CapiServer.HandleEvent] – Received event:

Possible Cause & Solution Feedback ports are configured in the Telephony Services branch of the System Administrator with an incorrect extension number.

CLS_START_CALL_EVENT
INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46150’ IsInteractionInitiator=False PhoneNum=40008 UniqueDeviceID=

The following messages can be found in the Telephony Services log at the end of the survey:
INFO ; [CFBasicActivity.OnHasNextQuestion] – Line[1] Quit questions loop … WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI

interaction was found

The phone number in the log file (in this case 40008) is not the extension number configured in the System Administrator for the line (in this case 1). 6 Survey is not loaded. Feedback call is immediately disconnected. The call is routed to the Telephony Services but the server immediately hangs up the call. There is no record of the call in the database. In the log file the following lines will appear:
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] Get survey ID by load information [Load by DTMF] INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] DTMF

The Telephony Services cannot find the received DTMF in the System Administrator’s configuration. In Avaya Automatic Routing, the received DTMF is the VDN that initiates the Feedback call. A DTMF routing should be added to the Survey Assignments tab of the Feedback branch in the System Administrator with the received DTMF sequence.
297

sequence 48016
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] – Line[2] No survey

ID was found

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting Customer Feedback

Troubleshooting - CTI Info or Business Data Routing
Table 10-3: Troubleshooting - CTI Info or Business Data Routing # 7 Problem Survey is not loaded. Feedback call is disconnected after more than 15 seconds. Symptom After the call was transferred to the Feedback extension, there is silence for 15-20 seconds and then the line is disconnected. No data was saved in the database. In the log, CAPI information is not received:
INFO ; [CFMainWF.OnIsCtiInfoLoadType] – Line[1] Survey load type is

Possible Cause & Solution The Feedback extensions are not monitored by the driver. In this case a CAPI event is not received by the Telephony Services. The Feedback extensions should be added to the monitored devices list in the Integrations branch of the System Administrator.

CTI_INFORMATION
… INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start

call, 15 seconds left
INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start call, 14 seconds left … INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start call, 1 seconds left WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI

interaction was found

8

Survey is not loaded. Feedback call is disconnected after more than 15 seconds.

After the call was transferred to the Feedback extension, there is silence for 15-20 seconds and then the line is disconnected. No data was saved in the database. In the log, CAPI information is received, but not used. In the log below, the CAPI reports calls for extension 40005, while the Telephony Services configuration specifies that extension 40008 is used in this line (line 1):
INFO ; [CapiServer.HandleEvent] – Received event: CLS_START_CALL_EVENT INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46149’ IsInteractionInitiator=False PhoneNum=40005 UniqueDeviceID= … [continued on next page...]

If CAPI events are constantly received but for a different extension, the Feedback ports are configured in the Telephony Services plug-in with a wrong extension number.

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

298

Troubleshooting Customer Feedback

Table 10-3: Troubleshooting - CTI Info or Business Data Routing (Continued) # 8 Problem Symptom
[...continued from previous page] INFO; [CapiControl.AddInteraction] – Added interaction to hashtable [Extension 40005, SwitchId 1, CompoundId 300866, InteractionId 300869, DNIS 40005, UCID 00001062061213527184, Phone number, VDN, IsTransferInteraction False] … INFO ; [TelephonyManagerService.AttachWorkflowToExtension] – Line[1] Workflow instance added to extension 40008 (InstanceId=ccd70c25-7e60-4db9-a32a-1c02e2eb5fdd) … INFO ; [CFMainWF.OnIsCtiInfoLoadType] – Line[1] Survey load type is

Possible Cause & Solution

CTI_INFORMATION
… INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start

call, 15 seconds left
INFO ; [CapiServer.HandleEvent] – Received event: CLS_START_CALL_EVENT INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46149’ IsInteractionInitiator=False PhoneNum=40005 UniqueDeviceID= … INFO; [CapiControl.AddInteraction] – Added interaction to hashtable [Extension 40005, SwitchId 1, CompoundId 300866, InteractionId 300872, DNIS 40005, UCID 00001062061213527184, Phone number T6205#1, VDN, IsTransferInteraction True] … INFO ; [CapiServer.HandleEvent] – Received event: CLS_END_CALL_EVENT INFO ; [CapiServer.HandleEvent] – internal participant: agentId=’46149’ IsInteractionInitiator=False PhoneNum=40005 UniqueDeviceID= … INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start call, 14 seconds left … INFO ; [TelephonyManagerService.GetCapiInformation] – Line[1] Wait for CAPI start call, 1 seconds left WARN ; [TelephonyManagerService.GetCapiInformation] – Line[1] No CAPI

interaction was found

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

299

Troubleshooting Customer Feedback

Table 10-3: Troubleshooting - CTI Info or Business Data Routing (Continued) # 9 Problem Survey is not loaded. Feedback call is immediately disconnected. Symptom After the call is transferred to the Feedback extension it is immediately disconnected. No data was saved in the database. In the log below, the survey should be loaded by DNIS. A call, previously received from the CAPI, is found with DNIS 40005, but there is no routing configured in the System Administrator for this DNIS value:
INFO ; [TelephonyManagerService.GetCapiInformation] - Line[1] Wait for CAPI start call, 15 seconds left INFO ; [TelephonyManagerService.GetCapiInformation] - Line[1] Found CAPI

Possible Cause & Solution No survey is configured for the CTI information or the Business Data of the call. A CTI information or Business Data routing should be added to the Survey Assignments tab in the Feedback branch of the System Administrator with the received information.

interaction ID 300895
INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] Get survey ID by load information [Load by DNIS] INFO; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] interaction information Extension 40005, SwitchId 1, CompoundId 300889, InteractionId 300895, DNIS 40005, UCID 00001062561213527869, Phone number T6255#1, VDN, IsTransferInteraction True INFO ; [TelephonyManagerService.GetSurveyIdByLoadInfo] - Line[1] No survey ID

was found

10

After a survey ends another survey starts in the customer’s call.

In Manual Transfer, after the agent transferred the call and the survey was completed, the customer hears once again the survey prompts from the beginning. Also, in the database there are two surveys with the same CompoundID and nvcUCID.

Both Manual Transfer and Automatic Routing are used. The agent first transfers the customer to the Feedback extension. When the Feedback session ends, the VDN transfers the customer to another extension. Only one of the methods should be used. Either the Automatic Routing will be disabled, or the agent will stop using the Manual Transfer.

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

300

Surveys and Interactions are not Properly Correlated

Surveys and Interactions are not Properly Correlated
If you have correlation issues between your surveys and the agents’ interactions, this may be due to a faulty UCID. Check your switch configuration. See the Playback Server Installation Guide. Review the Customer Feedback Setup Guide.

Chapter 10: Customer Feedback Installation Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

301

Blank page for double-sided printing.

11
High Density Logger Troubleshooting Flow
This chapter contains troubleshooting tips and procedures for High Density Loggers.

Contents
General High Density Logger Troubleshooting Flow................................................304 Gathering Information ..................................................................................................305 Checking the Basics ....................................................................................................305 Verifying the Logger Configuration ............................................................................306 Generating a Build Content Report.............................................................................307 Checking Logger Initiation Failure..............................................................................308 Setting the Debug Level...............................................................................................309 Checking for Logger Configuration Errors ................................................................314 Checking Logger Activity ............................................................................................316 Checking Map Errors ...................................................................................................316 Checking RCM Issues ..................................................................................................317 Synchronizing the Time ...............................................................................................317 Testing the Logger COTS using the NICE Test Kit ....................................................318 Testing the NICE Boards using the NICE Test Kit .....................................................320 Testing the Logger using the NICE PST .....................................................................321 Shutting Down and Starting Up the Logger ...............................................................322 Shutting Down and Starting CLS ................................................................................323 Using the Nicelog Setup Tool ......................................................................................323 Other Troubleshooting Issues.....................................................................................325 Other Useful Tools........................................................................................................326

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

303

General High Density Logger Troubleshooting Flow

General High Density Logger Troubleshooting Flow
If you experience a problem with a High Density Logger, the following workflow lists the steps to isolate the problem. Each step in the workflow is a section in this chapter.
Figure 11-1 General High Density Logger Troubleshooting Flow

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

304

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Checking the Basics
Perform the following basic checks:
1. Check that the Logger services are up. 2. Reboot the Logger. See Shutting Down and Starting Up the Logger on page 322. 3. Check the connection to the power supply. 4. Check the network connection. 5. Verify that the TCP/IP properties correspond with the system administrator's site definition. 6. If the Logger is not responding in the SNMP traps, stop the CLS and then reboot the CLS and

Logger. See Shutting Down and Starting Up the Logger on page 322 and Shutting Down and Starting CLS on page 323.
7. Check any changes were made to the switch, for example, frame format, signaling type, switch

version, and so on.
8. Check the connection between the Logger and the NICE Interactions Center. 9. Check the cables between the Logger and the switch. 10. Use the NICE Test Kit to check the COTS components. See Testing the Logger COTS using

the NICE Test Kit on page 318.
11. Use the NICE Test Kit to check the NICE boards. See Testing the NICE Boards using the

NICE Test Kit on page 320.
12. Use the PST to check the Logger. See Testing the Logger using the NICE PST on page 321.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

305

Verifying the Logger Configuration

Verifying the Logger Configuration
Check that the Logger has been correctly configured in System Administrator.
To verify the Logger configuration: 1. Ensure that the Logger is up and running. 2. Verify that you are working in Technician mode. In the Organization tree, click Organization. Then select the Technician Mode checkbox, and click Save. 3. In the Organization tree, select the Logger Servers branch. 4. Select the relevant Logger and verify the: a.

IP address or Host Name Input Channels Global parameters

b. Input Trunks c.

d. Predefined Selective Recording e.

5. In the Organization tree, expand CLS Definitions > CLS Servers, and select a CLS server. 6. Click the Loggers tab. 7. Ensure that the Logger is attached to the CLS. 8. In the Organization tree, expand Logger Servers > NiceLog > <Logger name> > Recording Channels. 9. Verify the Audio Recording Channels setup.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

306

Generating a Build Content Report

Generating a Build Content Report
Use the NICE Test Kit Application to generate a build content report of the software and hardware. You can collect the following logs in the zip file together with the build content report:

• • • •

Logger Cfg Files Logger Log Files OS Event Log File MSInfo32 File

For further details, see Testing the Logger COTS using the NICE Test Kit on page 318, and the NICE Test & Recovery Kit User Manual.
To build a content report: 1. If you do not have the NICE Test Kit Application, you can download it from ExtraNICE > Services > Diagnostic Utilities. 2. In the NICE Test Kit, click Build Content Report.
Figure 11-2 NICE Test Kit Window

3. In the Options area, select the required log files, and in the Destination Folder field, enter a

Destination Folder for the Content Report.
Figure 11-3 Log File and Destination Folder Options

Select log files

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

307

Checking Logger Initiation Failure

4. Click Take Build Content. The build content is recorded. 5. To see the report, click View Build Content. 6. Close the Build Content Viewer window. 7. Click Save Build. 8. Click OK.

The build content is saved in a zip file.

Checking Logger Initiation Failure
In the event of a Logger initiation failure, ensure that all the new hardware has been installed.
To check Logger initiation failure: 1. In Computer Management, select Device Manager. 2. Look for errors. There is no specific error messages for Logger initiation failure. Different

error messages may appear according to which devices were installed.
Figure 11-4 Computer Management Window Displaying Errors

Errors

3. From the Action menu, run Scan for hardware changes.

The device identification process starts.
4. Ensure that all the hardware is installed.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

308

Setting the Debug Level

Setting the Debug Level
Switch the debug level to FATAL.
To set the debug level: 1. Open d:\NTlogger\logger\config\SysRC.dat in Notepad.
Figure 11-5 SysRC.dat File in Notepad

2. In [system -default], set the LogLevel2File and LogLevel2Screen to 1 instead of 3. 3. Reboot the Logger. See Shutting Down and Starting Up the Logger on page 322.

Checking the Logs
Bug Find and Log Analyzer are two tools used to check the logs of High Density Loggers:

• •

Using Bug Find to Check Logs on page 309 Using Log Analyzer to Check Logs on page 312

Using Bug Find to Check Logs
Bug Find automatically searches in the Logger log files for FATAL and BUG errors. Other files can be dragged and dropped into it and it will search in them as well. The logs are located in D:\NTLogger\logger\log. The log files are plain text. The directory includes:

Log files for monitoring the recording activity:

Logs all Logger software messages and errors.
Chapter 11: High Density Logger Troubleshooting Flow 309

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Setting the Debug Level

• • • • • •

Last ten files, up to 10 MB each. Named LogFile.DAT (current), then LogFile1.DAT, …, LogFile9.DAT (oldest).

Event log files: Logs important events and user actions on Logger. One file for each day of the month. Named EVELogFileN.DAT, where N=1..31

Figure 11-6 Log File

Figure 11-7 Event File

To check the logs: 1. Double-click D:\NTLogger\Logger\Testers\LoggerUtils\bugfind.exe to run Bug Find.

This tool can also be found on the Tier 4 Team Site.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

310

Setting the Debug Level

Figure 11-8 Bug Find

2. In the Search String field, enter - logger. This is part of the string --logger is up and running- indicates that the Logger started up.
Figure 11-9 Bug Find - “Logger is up and running” String

3. For a FATAL error, check which component name is listed after FATAL:. The example below

shows a DBM fatal error start, indicating a problem with the database.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

311

Setting the Debug Level

Figure 11-10

Bug Find - Fatal Error

Using Log Analyzer to Check Logs
Log Analyzer searches multiple log files for any string, such as an Extension, Agent ID, or IP address.
To use Log Analyzer to check logs: 1. Double-click D:\Program Files\Common Files\Nice\LogService\Bin\LogAnalyzer.exe

to run Log Analzyer.
Figure 11-11 Log Analyzer

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

312

Setting the Debug Level

2. In the Search String field, enter the string you want to find in the logs. 3. Add log files to Log Analyzer in the following ways:

• •

Drag and drop log files into the Log Analyzer window. Click Add , and browse to each log file you want to add.

4. Click Search to start searching the logs.

Log Analyzer searches the logs, and the Search button changes to Stop.
Figure 11-12 Log Analyzer Searching Logs

5. Click Stop to stop the search, or Exit to close Log Analyzer.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

313

Checking for Logger Configuration Errors

Checking for Logger Configuration Errors
Line errors can be seen in the Functional Tester. A warning error (on one or more lines) on the Logger may indicate an issue with the Logger configuration. This tool should only be used in consultation with NICE Support!!!! Error thresholds are set which indicate how and when an error is reported.
To view the error thresholds: 1. To view the error threshold, open d:\NTLogger\logger\config\AudHW.cfg and search for: [NATI2_ERRORS_PARAMS] / [ETAI2_ERRORS_PARAMS] / [ETAI3_ERRORS_PARAMS]/ [ALI4_ERRORS_PARAMS] 2. There are 2 parameters per error:

• •

The number of times that the board checks the line and finds the error. The total number of times that the board checks the line, that is, the number of checks in window of checks.

EXAMPLE: 11 ; Min Errors: Max Frame Length - Error 0 - LOS (Loss of sync) - in order to report this error, the error must occur every time the line is checked. 2 4 ; Min Errors: Max Frame Length - Error 2 - UBAL (PBX terminal balance error) - in order to report this error, the error must occur at least twice in 4 checks.

To view the errors: 1. The error message format is:

Board ALI4 ETAI2 ETAI3 BTAI2 NATI2

Error ALI4_CheckAllBoards: Board has real error, <Board, Stream, Error Index, Error Count, Frame Length> ETAI2_CheckAllBoards: Board has real error, <Board, Line, Error Index, Error Count, Frame Length> real error, <Board, Stream/Line, Error Index, Error Count, Frame Length> BTAI2_CheckAllBoards: Board has real error, <Board, Line, Error Index, Error Count, Frame Length> NATI2_CheckAllBoards: Board has real error, <Board, Stream, Error Index, Error Count, Frame Length>

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

314

Checking for Logger Configuration Errors

Where:
Parameter Board Stream/Line Error Index Description Board ID. Stream/line one the board. The type specific error ID: ALI4: 0 – NOS ETAI2: 0 – FER, 1 – MLOS, 2 – PRA, 3 – FLOS, 4 – AIS, 5 – NOS ETAI3: 0 – FER, 1 – MLOS, 2 – RS, 3 – XS, 4 – PRA, 5 – FLOS, 6 – AIS, 7 – NOS BTAI2: 0 – FLOS, 1 – NOS NATI2: 0 – LOS, 1 – NOS, 2 – UBAL, 3 – LSIG, 4 – MFAS, 5 – INDICATOR TIME OUT, 6 – SYNCWINKFLASH Error Count Frame Length The number of times the board checked the line and an error occurred in the current window of checks. The length of the frame in which the errors were counted.

2. For example, error message 4 20 1 1 1 indicates a critical problem that is reported even if

there is only one error out of one check:

• • • • •

Board = 4 Line on Board = 20 Error Index = the first error in the config file Error Count = 1 Frame Length = 1

This error may indicate a signal that is too low and require an amplifier or active coupler. A real error is reported when the error rate is greater than or equal to that specified by the parameters:
WARN: NATI2_CheckAllBoards: Board has real error

This is followed by another error when the error rate drops below that specified by the parameters:
WARN: NATI2_CheckAllBoards: Board error was reset

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

315

Checking Logger Activity

When the Capture driver does not find all the hardware boards the following log message appears:
System_Init: Failed to find all boards. <0,0>.

From Logger 9.06 SP5, a warning message appears while waiting for all boards to be found:
System_Init: Wating for all boards to be found by Pnp. This may take several minutes... <SafeCounter,0> SafeCounter indicates the number of tries to find the boards. The maximum number of retries

is 60 with an interval of 10 seconds, that is, the plug and play manager has 10 minutes to locate the NICE boards.

Checking Logger Activity
If there is no activity on the Logger, use the Functional Tester to check the mapping and input type. This is not usually a Resource Coordination Module (RCM) issue.
IMPORTANT This tool should only be used in consultation with NICE Support.

Checking Map Errors
The following may cause an error when trying to create a map on the logger (Digital Matrix configurations):

• • •

The board may be configured as Co-PaBX and there is no summation mate in the map. The summation mate trunk is not configured on the board. There are not enough channels and the Logger is mapped with static inputs.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

316

Checking RCM Issues

Checking RCM Issues
RCM communicates with different components, such as Loggers, ScreenAgents and other CLS components.
To check the RCM Logs:

On the NICE Interactions Center, open the RCM process (Resource Manager console), or review the RCM.log.xxx.txt files in D:\Program files\NICE Systems\NICE CLS\Bin\Log.

To check the Call Server Logs:

On the NICE Interactions Center, the CallServer.log.xxx.txt logs are located in a \Log folder in the installation path. The following RCM problems may occur:

If the RCM reports that there are no resources to allocate (indicated as an Unmapped exception 15 in the Call Server logs), check the following:

• • •

For Total and Interaction-based recording, check the mappings in System Administrator. For Interaction-based recording, also check that you have enough channels in your channel pool when recording simultaneous calls.

If the RCM cannot connect to a Logger, check if the Logger is assigned to the CLS Server in System Administrator and if the settings were saved.

Synchronizing the Time
Each computer in the Logging System must be accurately time synchronized to ensure correct operation. Time synchronization of Logging System components is achieved using the Windows Time service known as W32Time that uses the Network Time Protocol (NTP). Open a command prompt and type: w32tm /resync.
NOTE: Loggers clock should not be modified while the Logger is running. Modify the time with NTP only.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

317

Testing the Logger COTS using the NICE Test Kit

Testing the Logger COTS using the NICE Test Kit
The NICE Test Kit is provided on a USB flash drive and contains several utilities used to perform diagnostic testing of COTS (Commercial Off The Shelf) hardware. This enables you to determine if a component is faulty, and thus avoid replacing properly functioning components. The NICE Test Kit includes tests for PC peripherals, hard disks and backup devices. It also contains System Recovery Tools that enable you to restore the NICE system on your Logger/Recorder. You can also detect the devices on the Logger/Recorder and create a Build Content Report. The NICE Test Kit on the USB can be ordered as a spare part. You can also download the NICE Test Kit Application from ExtraNICE. This can be used without the USB and offers limited functionality. The NICE Test Kit Application enables you to detect hardware devices on your Logger (SBC/MB, NICE Cards, Hard Disk Devices and Backup Devices). It also enables you to create a hardware build content report and collect the following files.

• • • •

Logger Cfg Files Logger Log Files OS Event Log File MSInfo32 File

To test the Logger: 1. If you do not have the NICE Test Kit, you can download it from ExtraNICE > Services > Diagnostic Utilities. 2. For further details, see the NICE Test and Recovery Kit User Manual.

Tests and Log File Locations in a Windows Environment
The table below lists the location of tests and log files in a Windows Environment.
Table 11-1: Test and Log File Locations in a Windows Environment Test/Operation Detected Device List Build Content Reports Test Description List of detected H.W devices. Report of the Build content (Software and Hardware) Update the NICE Test Kit Checking the CPU Log File Location No log file. Root:\Reports\*.zip & *.html

Live Update (Web) PC Peripherals - CPU

No log file. Root:\Programs\BurnInTest\BIT_log*.log

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

318

Testing the Logger COTS using the NICE Test Kit

Table 11-1: Test and Log File Locations in a Windows Environment (Continued) Test/Operation PC Peripherals Network PC Peripherals Memory PC Peripherals – VGA 2D HDD Fujitsu (SCSI) HDD Maxtor (IDE & SATA) HDD Seagate Enterprise (SCSI) HDD Seagate Seatools (IDE & SATA) Test Description Checking the Network Checking the Memory Checking the VGA 2D Checking Fujitsu SCSI Hard Drives Checking Maxtor IDE & SATA Hard Drives Checking Seagate SCSI Hard Drives Checking Seagate IDE & SATA Hard Drives – Requires DotNet2 software Checking Western Digital IDE, SATA & SCSI Hard Drives Checking undefined manufacturers SCSI Hard Drives Checking PC peripherals for various NICE EIS products Checking PC peripherals for various NICE Security products Checking SONY AIT IDE Backup Devices. Firmware Upgrade for SONY IDE & SCSI Backup Test DVD Creator Application Backup DVD Read & Verify Creating a Test DVD Checking DVD Drives Log File Location Root:\Programs\BurnInTest\BIT_log*.log Root:\Programs\BurnInTest\BIT_log*.log Root:\Programs\BurnInTest\BIT_log*.log Root:\Programs\HDD Fujitsu Diag SDIAG for Windows Root:\Programs\HDD Maxtor - NICE Disk Tool for Windows Root:\Programs\HDD Seagate SeaTools Enterprise for Windows Root:\Programs\SeaTools for Windows

HDD Western Digital (IDE, SATA & SCSI) HDD Other (SCSI)

Root:\Programs\HDD Western Digital Diagnostic for Windows\WinDlg.htm Root:\Programs\HDD Seagate SeaTools Enterprise for Windows Root:\Programs\BurnInTest\BIT_log*.log

NICE EIS Products

NICE VIM Products

Root:\Programs\BurnInTest\BIT_log*.log

Backup Sony AIT & DDS Tape Tool

Root:\Programs\SONY Tape Tool for Windows\S.N.dat S.N indicates the tape serial number No log file. Root:\Programs\BurnInTest\BIT_log*.log

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

319

Testing the NICE Boards using the NICE Test Kit

Tests and Log File Locations in a DOS Environment
The table below lists the location of tests and log files in a DOS Environment.
Table 11-2: Test and Log File Locations in a DOS Environment Test / Operation Backup SONY AIT & DDS Test (SCSI) HDD Seagate and Maxtor Test (IDE and SATA) HDD Western Digital Test (SATA) HDD Hitachi-IBM Test (IDE, SATA and SCSI) Test Description Checking SONY AIT & DDS SCSI Backup Devices Checking Seagate & Maxtor IDE & SATA Hard Drives. Checking Western Digital SATA Hard Drives Checking IBM or Hitachi IDE, SATA or SCSI Hard Drives (DOS) Memory test Performing System Recovery over IP Log File Location Root:\Programs\SONY\NTAU\*.ana Root:\Programs\SONY\Ship\*.abc Root:\Programs\SEAGATE

No log file. Root:\Programs\DFT\*.BLZ Location depends on the user. The log files can only be read by the manufacturer. No log file. No log file.

Memory Diagnostic Tools System Recovery Tools

Testing the NICE Boards using the NICE Test Kit
You can use the NICE Boards Diagnostics Utility (NBDU) to analyze the operation of the NICE boards within the recording platform, and diagnose whether a board is defected. The NBDU allows Field Engineers to easily test the hardware boards in the field and get an immediate indication of its functioning. The NBDU requires the use of the NICE Boards Diagnostics Device (NBDD) self test accessory which must be purchased separately. The following NICE boards can be tested:

• • • •

ALI IV NATI II ADIF IV ETAI III

For further details, see the NICE Test & Recovery Kit User Manual.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

320

Testing the Logger using the NICE PST

Testing the Logger using the NICE PST
The NICE Portable Service Tester (PST) is a system level tester that improves hardware failure diagnostics on site. The PST enables the field support engineer to test audio Loggers. See the NICE PST User Manual. The following tests are available:

• • • • • • •

Activity: Checks for the channel input signal. Threshold: Tests the channel signal threshold. Monitor Telephone Side: Checks the signal on the telephone side (for live monitoring). Monitor PBX Side: Checks the signal on the PBX side (for live monitoring). Record and Play: Records the channel for a few seconds of signal, and then plays it back. Compression: Compresses and records a call, and then plays it back. Manual Archiving: (If archiving device is connected). Archives a call and retrieves it from

the backup device.

Specifications
• • • •
Suitable for use with the following Loggers: NiceLog Logger, High Density Logger, NiceCall Focus, ICU and MIRRA IV. Compatible with the following PABXs: Analog, E1, T1, BT, Nortel, Norstar, Lucent, Siemens, Nec, Philips, Ericsson, Aspect, Alcatel. (The PST is ready to add any new PABXs). Testing time per channel of between 5 and 60 seconds. Supports up to two NATI boards, two ALI boards, one ETAI board and one BTAI board (future use).

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

321

Shutting Down and Starting Up the Logger

Shutting Down and Starting Up the Logger
Follow the procedures below to shut down and start up the Voice Logger processes correctly, BEFORE you restart or shutdown the Windows operating system.
To shut down and start up the Logger: 1. There are multiple Voice Logger modules running in the Windows Taskbar.
Figure 11-13 Multiple Voice Loggers Running in the Windows Taskbar

Multiple Loggers running

2. From the Start menu, select Shutdown Logger. All of the modules, except for the VL Snmp

Agent, will shutdown. You can either close the VL Snmp Agent manually, or ignore it. If you are not rebooting the Operating System, do not close the SNMP Agent.
-or-

Stop the NiceLog service.
3. To start the Voice Logger either:

• • •

Reboot the Windows operating system and let the system startup automatically. Select Start > Programs > NiceLog Server > Monitor LOG or Start > Programs > Startup > Monitor LOG. Start the NiceLog service.

The Voice Logger starts up and all of the Voice Logger modules appear in the Windows Taskbar. When the Voice Logger is completely started up, it plays the 7-note couplet "Shave and a Haircut, Two Bits."
NOTE: If you close the VL Snmp Agent, but do not reboot the Windows operating system, then you must restart the Windows SNMP.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

322

Shutting Down and Starting CLS

Shutting Down and Starting CLS
To shut down the Interactions Center: 1. On the Interactions Center, double-click the -or-

icon in the task bar, and then in the Interaction Server Service Manager, click Stop NICE CLS. Stop the NICE CLS service.

2. Shut down Windows normally. To start the Interactions Center: 1. Power the computer on and log in via the console or PC Anywhere using the admin account.

This should launch all services, startup apps, the CLS Dispatch process, and all CLS daughter processes. If auto-login is configured or the CLS is running as a service, even a Windows login is unnecessary.
2. To start the NICE CLS service, either use the Interactions Server Service Manager or start the

NICE CLS service.

Using the Nicelog Setup Tool
The Nicelog Setup Tool is installed on the Loggers. This enables you to make setting changes. This tool should only be used in consultation with NICE Customer Support!!!!
To run the Nicelog Setup Tool: 1. In D:\NTLogger\Logger\Utilities, double-click SetupTl.exe. The options vary according to

the boards installed.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

323

Using the Nicelog Setup Tool

Figure 11-14

Nicelog Setup Tool

2. Click GENERAL.
Figure 11-15 GeneralForm Window

3. From the Pcm Mode list, select A_law or U_law, and then click OK. 4. To modify the ETAI settings, click ETAI.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

324

Other Troubleshooting Issues

Figure 11-16

ETAI3 Setup Window

5. Make the necessary changes (typically Frame Format) and click OK. 6. Click OTHERS, and select LMOP if you have a High Density Logger, or Alarm Board if you

have NiceCall Focus III or ICU.

Other Troubleshooting Issues

If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file. In any event, any change of configuration files should be done ONLY in consultation with NICE Support!!!! In the event of a change in configuration, restart the relevant service. Backup browser related errors that are viewed in the System Event Viewer logs may be due to one of the following reasons: Network outages Name resolution problems An overloaded master browser A master browser that is unavailable because, for example, it is shut down or has lost network connectivity Solutions for various additional troubleshooting and performance issues are detailed in the Channel Mapping Guide:

• • • • • • •

Channels Definitions: Logger does not Appear
Chapter 11: High Density Logger Troubleshooting Flow 325

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Other Useful Tools

• • • • • •

Channels Definitions: Input Channels do not Appear Source Definitions: Cannot define Virtual Extensions Source Definitions: Physical Switch does not Appear Dynamic Mapping: Screen Logger does not Appear Dynamic Mapping: Virtual Extensions Deleting or Changing Logger, Switch, or CLS Definitions

Other Useful Tools
Various hardware diagnostic utilities are available to test Logger hard drives and storage devices on ExtraNICE > Services > Diagnostic Utilities.

Perfmon is a Windows built-in tool for monitoring the utilization of operating system resources, as well as system availability. The Perfmon Counter Logs Creator helps in automatically creating counter logs for using with the Perfmon tool. See Perfmon Counter Logs Creator on the Tier 4 Team Site.
Figure 11-17 Perfmon Counter Logs Creator Window

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

326

Other Useful Tools

If you do not have the Perfmon Counter Logs Creator, follow the procedure below to collect the Performance Monitor logs:
a.

From the Start menu, select Run. In the new window (on the left hand side), under Console Root > Performance Logs & Alerts, right-click Counter Logs and select New Log Settings. In the new window, in the General tab, click Add Objects, and select the Use local computer counter settings option. In the Performance Objects list, add all the processes that are relevant to the Logger and click Close. The file is saved in the path specified in the General tab.

b. In the Open field, enter perfmon and click OK. c.

d. In the new window, enter in a suitable filename and click OK. e.

f.

Chapter 11: High Density Logger Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

327

Blank page for double-sided printing.

12
VoIP Troubleshooting Flow
This section contains troubleshooting tips and procedures for when you experience a problem with VoIP.

Contents
Gathering Information ..................................................................................................330 Isolating the VoIP Problem ..........................................................................................330 Troubleshooting VoIP Problems .................................................................................331 Troubleshooting the VoIP Logger LAN Connection ..................................................332 Troubleshooting N+1....................................................................................................347 Other Troubleshooting Issues.....................................................................................349

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

329

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Isolating the VoIP Problem
There are two VoIP recording modes:

• •

Passive Active:

• • • • • •

VRG VRA CISCO Active Avaya DMCC Nortel DMS Alcatel IPDR

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

330

Troubleshooting VoIP Problems

Troubleshooting VoIP Problems
1. If the problem is a system-level problem, that is, there is no recording at all:

• • • •
a.

Check that all services are up and running (Applications Server, Interactions Center). Check that the VoIP Loggers are up. Check the communication between servers and VoIP Loggers. Check that there is time synchronization between the various NICE servers. Verify the VoIP Logger configuration in System Administrator, check that the VoIP Logger was added to the CLS and that it is up (by selecting it and verifying that you see all the VoIP Logger's details). LAN Connection on page 332):

2. If the problem is a VoIP Logger-level problem:

b. Troubleshoot the VoIP Logger LAN Connection (see Troubleshooting the VoIP Logger

• • • •

Checking the Logger Network Cables on page 332 Troubleshooting VoIP Logger Connections on page 332 Troubleshooting Speed Synchronization Failure on page 332 Verifying the TCP/IP Configuration on page 333

3. Using Logs to Check that VoIP Logger is Configured to Receive RTP Packets 4. Checking the RTP Packets using the Sniffer 5. Troubleshooting Network and Environment Issues 6. Checking the CLS Exceptions 7. Checking Channel Mapping

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

331

Troubleshooting the VoIP Logger LAN Connection

Troubleshooting the VoIP Logger LAN Connection
This section provides procedures for:

• • • •

Checking the Logger Network Cables on page 332 Troubleshooting VoIP Logger Connections on page 332 Troubleshooting Speed Synchronization Failure on page 332 Verifying the TCP/IP Configuration on page 333

Checking the Logger Network Cables

Check that the following cables are connected:

• •

The network cable for communication with NICE Perform. The Sniffing network cable.

Troubleshooting VoIP Logger Connections

If you see that the physical connection between the Logger and the LAN is down (the LED on the NIC is off), you must check the hardware at the site.
To troubleshoot VoIP Logger connections:

Check the physical connection between the Logger or NICE Interactions Center and the switch or hub by checking the following hardware along the path:

• • • •

Network adapter Cable Switch/hub LAN outlet (if one exists)

Troubleshooting Speed Synchronization Failure

This procedure enables you to check and configure your speed synchronization. Optimally, the Logger's speed and duplex should be configured to 100 Full Duplex.
To troubleshoot your speed synchronization: 1. On the desktop, right-click My Network Places. From the drop-down menu, select Properties. 2. Right-click Local Area Connection and select Properties. 3. Click Configure. 4. Click the Advanced tab. 5. Select the Property called Link Speed & Duplex, or Connection Type, or Media Type.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

332

Troubleshooting the VoIP Logger LAN Connection

NOTE: This property name may vary, depending on the LAN chipset. 6. Change the value as follows:

• •

If your site is using a hub, set the Duplex Mode to 100 Half Duplex or 10 Half Duplex (depending on the hub at the site). If your site is using a network switch, set the Duplex Mode to 100 Full Duplex.
TIP: If the connection still fails, gradually reduce Duplex Mode to 100 Half Duplex, then 10 Full Duplex, then 10 Half Duplex.

7. Verify the connection speed by checking the speed on the switch/hub, or on the Network

Adapter LEDs.
8. If connectivity still fails: a.

Reset the switch/hub port. Interface Card configuration.

b. Check that the switch/hub configuration (speed and Duplex) matches the Network 9. If you still have a problem, contact NICE Customer Service for further assistance. Verifying the TCP/IP Configuration verify the TCP/IP configuration:

Check that the following TCP/IP properties correspond with your system administrator's site definitions:

• • • • •

IP address Subnet mask Default gateway DNS server address WINS server address

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

333

Troubleshooting the VoIP Logger LAN Connection

Using Logs to Check that the VoIP Logger is Configured to Receive RTP Packets
The VoIP Logger Capture log files are located in D:\NTLogger\VOIPCapture\Logs:

IPCapture.<#>.log

EXAMPLE:

You have 4 log files. Their names are shown as follows: IPCapture.1.log, IPCapture.2.log, IPCapture.3.log,IPCapture.4.log

• •

IPCapture.sav: Contains the first error in the current session, or if there are no errors in the

current session, the first error in the last session when an error occurred.
IPCapture.crash: Exists when the Logger or Capture process crashes, that is when the

session was not closed. The VoIP Logger log files are located in:
D:\NTLogger\Logger\Logs NOTE: If you open a support case, zip all the log files in the D:\NTLogger\VOIPCapture\Logs and D:\NTLogger\Logger\Logs folders. To use logs to check that VoIP Logger is configured to receive RTP packets: 1. Search for the commands that configure the channel for recording.
Figure 12-1 Commands

NOTE: The Channel number is different from the Logger channel and the channel mapping number. Dir=Both indicates recording in both directions - Incoming (Rx) and Outgoing (Tx). In the log in Figure 12-2 on page 335, Dir=0 indicates Incoming (Rx), and Dir=1 indicates Outgoing (Tx).

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

334

Troubleshooting the VoIP Logger LAN Connection

2. Verify that the IP address and port is correct. NOTE: The port number is important for Gateway-based recording. In the log file, the port number is the second parameter in the ADDR attribute, after the colon (:).

This is the port number

3. Check that the channel was configured successfully (per direction requested). The Logger now

starts to sniff for RTP packets from this IP address and port, in the direction specified.
Figure 12-2 Channel Configured Successfully

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

335

Troubleshooting the VoIP Logger LAN Connection

4. If the VoIP Logger is configured to start sniffing for RTP packets use the sniffer to check the

RTP packets. See Checking the RTP Packets using the Sniffer.
NOTE: The Interactions Center pauses and resumes channels and for selective recording this may cause a problem.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

336

Troubleshooting the VoIP Logger LAN Connection

Checking the RTP Packets using the Sniffer
For detailed information on how to use the sniffer see Using the Sniffer on page 340.
To check the RTP Packets using the Sniffer: 1. Once the sniffer is connected and running, generate the following information:

• •

If there is a RTP capture issue on Logger, or Skinny capture issue on Interactions Center, make some calls from the relevant IP phone/s. If there is a RAS capture issue on Interactions Center, wait enough time for "Keep Alive" messages to be sent from IP phones. Protocol (RTP, RAS and so on) IP addresses of source and destination Compression (G.711, G.729, G723_6.3, G723_5.3 or G.722)

2. Analyze the relevant packets and confirm that the settings are correct:

• • •

3. Check for packet loss to see if all the relevant packets have been sniffed and check for packet

delay. If there has been any packet loss or delay, possible reasons may include:

• • • •

A problem with any one of the switches between the VoIP Logger and the PBX. A hardware problem with the network adapter. A problem with the configuration of teaming on the network adapter. A problem with the network, environment or switch configuration. See Troubleshooting Network and Environment Issues on page 338.

4. Check if there is jitter. If there has been jitter, contact the site administrator to check the media

provider configuration.
5. Check if there are multiple instances of the same traffic. Possible reasons may include:

• • • • •

If the same stream arrives twice, the network configuration may be causing the Logger to receive the stream from two NICs. Contact the site administrator. If there are multiple streams, and the recording is Gateway-based, check if the port for the Gateway is defined. In Active recording, check if two different streams are been sent to the same socket. Check if the rx and tx streams are been sent to the same port. This may be a vendor system problem. In passive recording, if the call is between two devices on the same switch that is used for spanning, there will be duplicate streams. This is not a problem. Check if the VoIP Logger is overloaded. Check as follows:

6. If the RTP appears to be sniffed correctly but there are still no recordings: a.

In Task Manager, check for high CPU consumption.
Chapter 12: VoIP Troubleshooting Flow 337

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting the VoIP Logger LAN Connection

• •

In Task Manager, check the network utilization. Run the Performance Counter and monitor the crucial elements.
Performance Counter

Figure 12-3

b. For selective-recording, there may be a problem with the IP addresses received from the

Interactions Center. See Using Logs to Check that the VoIP Logger is Configured to Receive RTP Packets on page 334.
c.

Check the CLS exceptions. See Checking the CLS Exceptions on page 339.

Troubleshooting Network and Environment Issues
Contact the site administrator in order to troubleshoot network and environment issues.
To troubleshoot network and environment issues: 1. Check the following with the system administrator:

• • • • • •

If there were any destination port range changes. If any of the ports were moved. If there were any source port changes for the phones or gateways. If a switch is down. If a switch is in "error disable" mode, indicating a malfunction of the physical port. If there were any VLAN configuration changes.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

338

Troubleshooting the VoIP Logger LAN Connection

For active VoIP:

• •

Check that the NIC IP is routable from the source IP address. Verify that the port intervals configured in the Logger IP Tool are open on the firewall for UDP packets.

2. If audio is not being recorded, the network board may not be sniffing the right ports on the

switch because:

• • • •

Check if VoIP Logger can ping all of the relevant components (Gateway, Call Manager and IP phones). Check for switch limitations that prevent sniffing of the relevant components. If Conference Calls are not recorded, check if a Conference Resource is in use (Cisco CCM environments), and verify that it is mirrored. Check the switch port mirroring configuration. The switch configuration may not have been saved before disconnecting from the programming interface, in which case, all the relevant spanning settings will be lost.
Figure 12-4 Switch Port Mirroring Configuration

Checking the CLS Exceptions
Query the database to see the exceptions. Most of the time exceptions are connected to recordings and not directly to the interaction IDs. In order to see the connection between an exception and interaction ID, query both the exception details and the related recording (which includes the interaction ID). For more details, see the Interaction Data Reference Guide.
To check the CLS Exceptions: 1. Query: SELECT * FROM tblExceptionXX, tblRecordingXX WHERE tblExceptionXX.iRecordingID = tblRecordingXX.iRecordingID

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

339

Troubleshooting the VoIP Logger LAN Connection

2. Check over what period the exceptions occur. 3. Select the vcExceptionDetail field in tblException. 4. Sub-exception 12 20 indicates no VoIP audio. This may be due to:

A configuration problem. This is dependant on the VoIP integration. Check the configuration, the forwarding data passed to the Capture (RCM logs), the Capture logs, as well as the forwarding device configuration. For example, no audio is received if no or the wrong forwarding information is passed to Capture, or if the audio is not forwarded by the telephony switch/forwarding device. The RCM CheckForAudioDelay parameter specified in the System Administrator specifies the period, in milliseconds, for RPT sniffing that the RCM waits after starting a call before checking for audio availability on the Logger. If the RTP takes too long, the CLS will mark the call as not recorded for total recording, and for interactions-based recording it will send a stop command to the VoIP Logger. If this occurs try to increase the duration of this parameter.

5. Exception 38 occurs, for example, when the update is for an unmapped device or a closed

request, or because the update was not received by the RCM.

Checking Channel Mapping
The VoIP Logger may have been configured with the wrong configuration in channel mapping (for example, IP addresses instead of MAC address). For more information, see Channel Mapping Troubleshooting.

Using the Sniffer
There are many Sniffer applications available, some of which are freeware. Wireshark (previously called Ethereal) (www.wireshark.org) is easy to install and a convenient tool to use.
To use the sniffer: 1. Run the sniffing application on a PC connected to the port on the switch where the sniffing

should be performed, that is, either:

• •

On the VoIP Logger if the Logger software is not running. A different PC connected to the switch port.

2. From the Capture menu, select Interfaces.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

340

Troubleshooting the VoIP Logger LAN Connection

Figure 12-5

Capture Interfaces

3. To capture all the traffic on the network adapter, click Start next to the required network

adapter to start capturing.
4. To filter while capturing: a.

In the Wireshark Capture Interfaces window, click Options next to the required network adapter.
Figure 12-6 Wireshark Capture Options

b. Check that the IP address is correct.

c.

Ensure that Capture packets in promiscuous mode is selected. In the Capture File(s) area:

d. (Optional) In Capture Filter, type host x.x.x.x, where x.x.x.x is the required IP address. e.

• •

Click Browse and specify a folder and file name. Verify that the drive has sufficient space available, for example 2 GB for 20 files of 100 MB. Select Use multiple files.
Chapter 12: VoIP Troubleshooting Flow 341

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting the VoIP Logger LAN Connection

• •
f.

Select Next file every 100 megabytes. Select Ring buffer with 20 files.

Click Start.

g. Make a test call. 5. From the Capture option, select Stop to stop capturing. 6. If the files are too small or too few to cover the required period, extend the number and size of

the capture files.
7. To filter an offline capture file for RTP packets: a.

From the File menu, select Open to load a capture file (*.pcap).
Figure 12-7 Wireshark Decode As

b. Right-click on the list of packets and select Decode As.

c.

Select Decode. Click OK.

d. In the Transport tab, select UDP both port(s) as RTP. e.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

342

Troubleshooting the VoIP Logger LAN Connection

Figure 12-8

Wireshark Offline Capture Filter

8. To analyze a stream: a.

From the Statistics menu, select RTP > Show All Streams.
Figure 12-9 Wireshark RTP Streams

b. Check for packet loss in the Lost field. c.

For more detailed information, select the relevant stream and click Analyze.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

343

Troubleshooting the VoIP Logger LAN Connection

Figure 12-10

Stream Analysis

9. To save the voice (G.711) for playback in a third-party player, click Save payload. This

enables you to check whether voice quality issues are related to the network or to the NICE system.
10. Check for jitter. 11. It may sometimes be necessary to manually check a softphone stream (Wireshark may not

report the correct number of expected packets). Typically in this case the playback will not be clear. This requires you to calculate the expected number of packets for the duration of the call and compare it to the actual number of packets received.
Figure 12-11 Softphone Stream

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

344

Troubleshooting the VoIP Logger LAN Connection

a.

Note the start time of the call, for example, 12.03.45. From the start and end times, calculate the duration of the call. Calculate the size of the payload: UDP Packet Size in Bytes - 8 Byte Header - 12 Byte RTP Header)/2, that is, 180-8-12=160 bytes. Check the codec size/msec.
Codec G.711/G.722 G.729 G.723_6.3 G.723_5.3 Codec Size/msec 80 bytes per 10 msec 10 bytes per 10 msec 24 bytes per 30 msec 20 bytes per 30 msec

b. Note the end time of the call, for example, 12.04.02. c.

d. Check the size of the UDP packet (User Datagram Protocol), for example, 180 bytes. e. f.

g. Calculate the duration of the RTP packet, that is, Payload / (Codec Size/msec).

See the following examples:
Codec G.711/G.722 G.729 G.723_6.3 G.723_5.3 Payload Duration 160 bytes/(80 bytes/10 msec)=20 msec 20 bytes/(10 bytes/10 msec)=20 msec 24 bytes/(24 bytes/30 msec)= 30 msec 20 bytes/(20 bytes/30 msec)=30 msec

h. Calculate the number of packets per second: 1000/Duration of the RTP packet, for

example, 1000/20=50 packets per second. See the following examples:
Codec G.711/G.722 G.729 G.723_6.3 G.723_5.3 Packets per Second 1000/20 msec=50 packets per second 1000/20 msec =50 packets per second 1000/30 msec=33.3 packets per second 1000/30 msec=33.3 packets per second

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

345

Troubleshooting the VoIP Logger LAN Connection

i. j.

Calculate the expected number of packets: Duration of the Call in seconds x Number of Packets per second, for example, 13x50=650 packets. If there exists a deviation of 50 packets per minute or greater between the number of expected and actual packets, this could be due to a problem in the media provider, for example, the phone, PBX, etc.

EXAMPLE:

A USB headset can cause such a deviation, since soft phones tend to generate a time drift. If, after checking the media provider, there still exists a deviation of 50 packets per minute or greater between the number of expected and actual packets, open a case with NICE support.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

346

Troubleshooting N+1

Troubleshooting N+1
When a normal Logger goes down and the spare Logger takes over the sequence of events can be viewed in the log files.
Spare Logger

The spare Logger determines that something happened to the normal Logger and notes that the normal Logger failed to report heartbeats.
27/07/2008 12:31:42.507 WARN: AUD_NPlus1SW_CheckNormalLoggers: Logger failed to report heartbeats. <LoggerHostID,0> 1 0 [ThreadID=3952]

The spare Logger starts the switching process.
27/07/2008 12:31:42.507 INFO: SpareLoggerThreadProc: spare logger is in the process of switching state. <OverridingLoggerID,0> 1 0 [ThreadID=3952]

The spare Logger is going to take over the function of Logger number <X>, where <X>is the position of the normal Logger in the N+1 chain, and not the Logger number in System Administrator.
27/07/2008 12:31:42.507 INFO: EVTA1020 Nplus1: Spare logger identified Normal switching from logger no. 1 [ThreadID=3952] 27/07/2008 12:31:42.507 INFO: AUD_SetQuietBuffsToSwitchFromFax got num of buffers 16 0 [ThreadID=3952]

The current configuration on the spare Logger is queried.
27/07/2008 12:31:42.507 INFO: VoIPDrvr.VoIP_GetChannelConfiguration : Log Channel : 1 0 [ThreadID=3952] 27/07/2008 12:31:42.507 INFO: VoIPDrvr.VoIP_GetChannelConfiguration : Log Channel : 2 0 [ThreadID=3952] 27/07/2008 12:31:42.507 INFO: VoIPDrvr.VoIP_GetChannelConfiguration : Log Channel : 3 0 [ThreadID=3952]

The channels on the spare Logger are configured according to the masc view of the normal Logger.
27/07/2008 12:31:43.007 INFO: SetVoipDMascConnections: Finished configuring all channels. 0 0 [ThreadID=3952]

The spare Logger is now overriding Logger number <X>, and the switching process is over.
27/07/2008 12:31:43.007 INFO: EVTA1021 Nplus1: Spare logger is overriding logger no. 1 - 5629 [ThreadID=3952]

The spare Logger is emulating the normal Logger, but with all channels started.
27/07/2008 12:31:43.007 INFO: Spare starts recording audio of logger : 5629 [ThreadID=3952] 1

The channels that should not be recording are stopped. Note that this works as designed, and that no buffers were been recorded for the channels that are stopped.
27/07/2008 12:31:43.007 WARN: AudMgr Nplus1StopRecord stopping the channels. <Status,0> 0 0 [ThreadID=3952]
Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 347

Troubleshooting N+1

NOTE: In the event of a brutal failure in Active VoIP environments, the spare Logger may need resources that are still in use by the normal Logger, for example, in Avaya DMCC the Virtual Extensions may still view themselves as in use by the normal Logger. These resources are released after a timeout period. After this short period the channels that could not be configured initially will be recovered. Normal Logger

An error is reported, for example, after the sniffing network card is disconnected. This error appears both in the Logger log file, as well as the Capture log file. The time to report this error is now only 10 seconds (in previous versions it might take up to 2 minutes).
27/07/2008 12:32:12.291 ERR: NormalLoggerThreadProc: Received a capture critical error event 0 0 [ThreadID=4452]

The normal Logger tries to stop the channels and release the resources, and the spare Logger starts to take over immediately.
27/07/2008 12:32:12.291 INFO: EVTA1010 Nplus1: Normal logger switching audio to spare due to Manual Switch [ThreadID=4452] 27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc Audio has been switched 0 0 [ThreadID=4452] 27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc perform switching to spare 0 0 [ThreadID=4452] 27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc stopping the channels 0 0 [ThreadID=4452] 27/07/2008 12:32:12.291 WARN: AudMgr NormalLoggerThreadProc ExternalCapture!! 0 0 [ThreadID=4452] 27/07/2008 12:32:12.291 INFO: VoIPDrvr.VoIP_StopRecord : Log Channel : 1 0 [ThreadID=4452] 27/07/2008 12:32:12.291 INFO: EVTA0101 Record Stop On Channel [1,1] [ThreadID=4452] 27/07/2008 12:32:12.291 WARN: AudMgr Nplus1StopRecord stopping the channels. <Status,0> 0 0 [ThreadID=4452] 27/07/2008 12:32:12.978 INFO: EVTA0751 AUM_AudioImport stop on input channel 1 with token number 1 [ThreadID=4520] 27/07/2008 12:32:13.025 INFO: DMA: DM_RecordProcessOnChannel- Stop record 1 0 [ThreadID=4424]

IMPORTANT After the normal Logger resumes its normal functioning, it MUST be restarted. The spare Logger does not need to be restarted.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

348

Other Troubleshooting Issues

Other Troubleshooting Issues
mascm
The VoIP Logger saves the configuration received from the CLS in the mascmap file. The VoIP Logger reconfigures itself on restart from this file. If there is a discrepancy between the CLS and the Logger, the CLS sends a request to the Logger to update the mascmap. There is therefore no need to delete this file.

Configuring after Replacing a NIC
If you replace a NIC, run the IP Tool (D:\NTLogger\VoIPCapture\Bin) and reconfigure the NIC properties.

Replacing a CISCO Phone
The Device Unique Identifier (DUI) is built into the CISCO physical phone. If you replace such a phone, the channel mapping must be updated as this affects the mascmap. See the VoIP Logger Installation Guide.

Troubleshooting a Packet Size Problem
There may be a problem with the maximum packet size if the following error appears in the Capture logs:
PacketHandler::HandleReceivingCompleted. Completion operation failed. ErrorCode=14040

This indicates that the message is too long, that is, a message sent on a datagram socket was larger than the internal message buffer or some other network limit, or the buffer used to receive a datagram was smaller than the datagram itself. This may occur with Avaya DMCC integrations. The MaxPacketSize parameter is set to 320 by default. Avaya DMCC may use a packet duration of 60 msec. In G.711 this means a packet size of 480 bytes without the header, that is, over 500 bytes per packet.
To troubleshoot a packet size problem:

In the Registry, change the MaxPacketSize to 512.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

349

Other Troubleshooting Issues

Figure 12-12

change the MaxPacketSize

Troubleshooting RTP Timestamp
Depending on the environment, the RTP packets should be transmitted every x milliseconds. You can use Wireshark to check that the RTP packets are in fact been transmitted and sniffed every x milliseconds. For more information, see Using the Sniffer on page 340. An increase of 8 in the timestamp is equal to one millisecond.
Example 1:
Figure 12-13 Check the RTP Packets

In this example if the phone uses 30 millisecond packets the timestamp should be incremented by 8 x 30 =240. Instead the timestamp is incremented by only 8, indicating a problem. This causes the Capture to mishandle the packets.
Example 2:

After about a minute a call starts to drift and the IPCapture generates errors and starts beeping. In this case the problem was caused by a specific soft phone with USB headset.
Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

350

Other Troubleshooting Issues

1. Open the trace file. 2. Decode the IP address packets to RTP. 3. In the Filter field, filter the calls for the source IP address by typing in: ip.src = = <X>.<X>.<X>.<X> and rtp 4. The first few packets are usually the ring tone and not related to the voice RTP stream. 5. The packets that follow are the voice RTP stream. In this example, the first was captured at

10:22:06.625 and the last one was captures at 10:24:16:107. This gives a stream duration of 00:02:09.50 hours, that is, 129.5 seconds.
6. The timestamp of the first packet is 0 and the timestamp of the last packet is 1048960. In

8KHz packets this translates to 131.12 seconds ((1048960-0)/8000).
7. There is more than a second between these two calculation, that is, the phone sends 131.12

seconds of voice in 129.5 seconds.

Troubleshooting RDP Delay
When connecting to a machine using RDP (Remote Desktop Protocol), a serious delay could occur both on the client side (console “freeze”) and on the server side (processes hang). As a result, crucial processes (for example, IP Capture process on the VoIP Logger) face unexpected situations due to not receiving the necessary resources.
Cause

There are several different parameters and system configurations that could cause such delays. Some are related to the network configuration and others to the RDP version installed on either the client or the server.
1. The order of Network Providers in the registry could be incorrect. The key to be checked is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\NetworkProvider\Ord er. The default order is "RDPNP,LanmanWorkstation,WebClient". Note that in Vista clients

(prior to SP1) there is a bug and the order is not as stated above (when installing Vista SP1 from a clean installation this bug is fixed). RDPNP should be the first provider in the list (before the LanmanWorkstation).
2. In RDP version 6.0, a network query is launched during the connection attempt (which does

not happen in RDP version 5.x). If a WINS server is configured on the client, and this server does not respond, the query will time out. As a result, the server waits for a response from the client and this causes the symptoms as described above.
Solution

In general, the Remote Desktop Connection could affect both the client and the server, causing delays and process starvation. Verify the following to solve the delays:
1. The registry key stated above (HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\NetworkProvider\Ord er) is set to the default order ("RDPNP,LanmanWorkstation,WebClient").
Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 351

Other Troubleshooting Issues

2. If using RDP version 6.0 make sure all network configurations (for example, WINS servers)

are available.

Troubleshooting the “Chain Not Found” Error Message
NOTE: For more in depth information on normal and spare Loggers in a chain, see “Resiliency and N+1” in the System Administrator’s Guide.

The “Chain not found” error message appears when the normal Loggers in the chain do not have the same multicast IP address as the spare Logger. Each chain has a unique multicast IP address, and all the Loggers in the chain (spare and normal) share this same multicast IP address.
To troubleshoot the “Chain not found” error message: 1. In the System Administrator, navigate to Master Site > Logger Servers > NiceLog, and

check that the normal Loggers and the spare Logger are all added to the system.
2. Create a new chain with these Loggers. See “Resiliency and N+1” in the System

Administrator’s Guide.
3. If the error message still appears, complete the following: a.

Stop the NICE VoIP Logger Service.

b. Navigate to D:\NTLogger\Logger\Testers\LoggerUtils, and double-click SWNPlus1Configurator.exe.

The Software N+1 Configuration window appears.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

352

Other Troubleshooting Issues

Figure 12-14

Software N+1 Configuration Window

c.

In the Multicast Group IP field, enter the multicast IP address for the chain. This IP address is the same for each Logger in the chain.

d. In the Chain ID field, enter a Chain ID for the Logger. TIP: Note the Chain ID for each Logger so that you can reconstruct these IDs when you recreate the chain in Step 4 on page 353. e. f.

Click Save and Close. Restart the NICE VoIP Logger Service. sure that you enter the Chain ID correctly. The Chain ID of each normal Logger increases by 1 for each additional Logger in the chain. The spare Logger is the last Logger in the chain and has the highest Chain ID.

g. Repeat Step a through Step f for each Logger in the chain. When repeating Step d, make

4. Create a new chain with the Loggers you configured in Step 3 on page 352. See “Resiliency

and N+1” in the System Administrator’s Guide.

Troubleshooting Active VoIP Integrations
See the following guides:

• • •

VoIP Recording Agent (VRA) Installation Guide. VoIP Recording Gateway Installation Guide Avaya DMCC Integration Guide.
Chapter 12: VoIP Troubleshooting Flow

353

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Other Troubleshooting Issues

• • •

Cisco IP Phone-based Active Recording Integration Guide. Integration with Nortel CS1000 and Duplicate Media Stream over IP Guide. Integration with Alcatel-Lucent OXE Including IP DR-Link Guide.

Chapter 12: VoIP Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

354

13
Interactions Center Troubleshooting Flow
This section contains troubleshooting tips and procedures for when you experience a problem with the NICE Interactions Center.

Contents
Interactions Center Troubleshooting Workflow.........................................................356 Gathering Information ..................................................................................................366 Open Case Procedure ..................................................................................................366 Checking the Basics ....................................................................................................373 Checking the SNMP Traps ...........................................................................................374 Setting the Reporting Level .........................................................................................375 Checking the CLS Configuration ................................................................................377 Starting/Stopping the Interactions Center Services..................................................378 Checking the Status of Interactions Center Components ........................................379 Troubleshooting using the Interactions Center Logs ...............................................381 Examples of Troubleshooting using the Logs...........................................................383 Checking Ports .............................................................................................................408 Checking the CLS Exceptions.....................................................................................410 Fine-tuning the CLS Server Configuration Parameters ............................................ 411 Isolating a Screen Problem Further ............................................................................412 Shutting Down and Starting CLS ................................................................................413 Other Troubleshooting Issues.....................................................................................414 Other Useful Tools........................................................................................................415

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

355

Interactions Center Troubleshooting Workflow

Interactions Center Troubleshooting Workflow
If you experience a problem with the Interactions Center follow the steps below to isolate the problem.
Figure 13-1 Troubleshooting Workflow

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

356

Interactions-based Call Recording Data Flow

Interactions-based Call Recording Data Flow
Figure 13-2 shows an interactions-based call flow, starting from the point where a call starts, and details how the information appears in the log files. This enables you to monitor the call flow, and troubleshoot the problems that may arise during the recording process, up to the point when the call is inserted into the database. See Interactions-based Call Recording Data Flow Details on page 358 for a description of every step mentioned in Figure 13-2.
Figure 13-2 Interactions Center Call Flow

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

357

Interactions-based Call Recording Data Flow

Interactions-based Call Recording Data Flow Details
1. The CTI Server sends a CTI event to the Driver. 2. The Driver processes the event. 3. The Driver sends a Start Call event to the Call Server. 4. The Call Server checks and processes the Start Call event: a.

The Call Server verifies that the agent ID or extension is recognized and exists. This check is done by accessing an internal login table (Calls_LT) which holds all agent login information. QA/Negative/Selective rule.

b. The RecordingDecisions checks if the new calls match a recording request or a c.

The Call Server updates an internal table (Calls_CT) with the call details, that is, with the start time, agent details, extension number, and so on.

5. The Call Server sends a Start Record event request (including all participants) to the RCM

and per participant.
6. The RCM returns channel allocation information to the Call Server (Synchronous Answer)

including the Logger #, Logger channel and time information.
7. The RCM sends the Logger a Start Recording command. 8. The Call Server updates Calls_CT with the recording details, including the Logger #, Logger

channel, Logger start time, and so on.
9. The Call Server sends a Start Call event to the Client, for example, Monitor (the client must

be registered for these type of events according to the Agent or Station and switch).
10. The Logger records the call. 11. The Logger returns a Recording Succeeded/Failed event to the RCM. 12. The RCM returns an Asynchronous Response to the Call Server. In interactions-based

recording, the real call start time is returned here. The call start time is the time that RCM received a response from the Logger that it has started recording, normalized to the Logger local time.
13. The RCM sends an Audio Check command to the Logger 2.5 seconds (configurable) after the Asynchronous Response is sent to the Call Server. 14. The Call Server sends a Start Record event to the Client, for example, Monitor (the client

must be registered for these type of events according to the Agent or Station and switch).
15. For VoIP configurations only, the RCM waits a configured amount of time and performs an

audio check with the VoIP Logger. If the Logger has not received any audio packets from the beginning of the call, an asynchronous Record Failure event is sent to the Call Server.
16. If one of the media recordings fail: a.

The RCM reports a Record Failure event to the Call Server.

b. The Call Server asks RecordingDecisions if the second media should be stopped.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

358

Interactions-based Call Recording Data Flow

c.

The RecordingDecisions answers the Call Server as to whether the second media should be stopped.

d. The Call Server sends a Stop Record request to the RCM for the second media. 17. The CTI Server sends a CTI event to the Driver. 18. The Driver sends an End Call event to the Call Server. 19. The Call Server checks if the calls are recorded in its internal database. 20. The Call Server sends a Stop Record event request to the RCM. 21. The RCM returns the stop request information with the recording status of the call to the Call

Server.
22. The RCM sends a Stop Record event to the Logger. 23. The Logger stops recording the call. 24. The Call Server updates the call details including the stop time and removes this call from

Calls_CT.
25. The Call Server sends an Insert Interaction event request to the DBServer (this is based on

having a Start and Stop message for the same call Interaction ID).
26. The Call Server sends an Insert Contact request to the DBServer including data about nested

interactions.
27. The Call Server sends a Stop Call (with recording information) event to the Client. 28. The DBServer inserts the interaction into the SQL Database on the Database Server.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

359

Understanding the Total Call Recording Data Flow

Understanding the Total Call Recording Data Flow
Figure 13-3 shows a total call recording Data flow, starting from the point where a call starts, and details how the information appears in the log files. This enables you to monitor the call flow, and troubleshoot the problems that may arise during the recording process, up to the point when the call is inserted into the database. See Understanding the Total Call Recording Data Flow Details on page 361 for a description of every step mentioned in Figure 13-3.
Figure 13-3 Total Call Recording Data Flow

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

360

Understanding the Total Call Recording Data Flow

Understanding the Total Call Recording Data Flow Details
1. The CTI Server sends a CTI event to the Driver. 2. The Driver processes the event. 3. The Driver sends a Start Call event to the Call Server. 4. The Call Server checks and processes the Start Call event: a.

The Call Server verifies that the agent ID or extension is recognized and exists. This check is done by accessing an internal login table (Calls_LT) which holds all agent login information. QA/Negative/Selective rule.

b. The RecordingDecisions checks if the new calls match a recording request or a c.

The Call Server updates an internal table (Calls_CT) with the call details, that is, with the start time, agent details, extension number, and so on.

5. The Call Server sends a Start Record event request (including all participants) to the RCM

per media (voice and screen) and per participant.
6. The RCM returns channel allocation information to the Call Server (Synchronous Answer)

including the Logger #, Logger channel and time information. In total-recording environments, the call recording start time returned is the start time reported by the Driver, normalized to the Logger local time. This is because the Logger is already recording at the time the call starts.
7. The Call Server updates Calls_CT with the recording details, including the Logger #, Logger

channel, Logger start time, and so on.
8. The Call Server sends a Start Call event to the Client, for example, Monitor (the client must

be registered for these type of events according to the Agent or Station and switch).
9. The Logger records the call. 10. The Logger returns a Recording Succeeded/Failed event to the RCM. 11. The RCM returns an Asynchronous Response to the Call Server. 12. The RCM sends an Audio Check command to the Logger 2.5 seconds (configurable) after the Asynchronous Response is sent to the Call Server. 13. The Call Server sends a Start Record event to the Client, for example, Monitor (the client must

be registered for these type of events according to the Agent or Station and switch).
14. For VoIP configurations only, the RCM waits a configured amount of time and performs an

audio check with the VoIP Logger. If the Logger has not received any audio packets from the beginning of the call, an asynchronous Record Failure event is sent to the Call Server.
15. If one of the media recording fails: a.

The RCM reports a Record Failure request to the Call Server. The RecordingDecisions answers the Call Server as to whether the second media should be stopped.
Appendix 13: Interactions Center Troubleshooting Flow 361

b. The Call Server asks RecordingDecisions if the second media should be stopped. c.

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Understanding the Total Call Recording Data Flow

d. The Call Server sends a Stop Record request to the RCM for the second media. 16. The CTI Server sends a CTI event to the Driver. 17. The Driver sends an End Call event to the Call Server. 18. The Call Server checks if the calls are recorded in its internal database. 19. The Call Server sends a Stop Record event request to the RCM. 20. The RCM returns the stop request information with the recording status of the call to the Call

Server.
21. The Logger stops recording the call. 22. The Call Server updates the call details including the stop time and removes this call from

Calls_CT.
23. The Call Server sends an Insert Interaction event request to the DBServer (this is based on having a Start and Stop message for the same call Interaction ID). 24. The Call Server sends an Insert Contact request to the DBServer including data about nested

interactions.
25. The Call Server sends a Stop Call (with recording information) event to the Client. 26. The DBServer inserts the interaction into the SQL Database on the Database Server.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

362

Understanding the Screen Call Recording Data Flow

Understanding the Screen Call Recording Data Flow
Figure 13-4 shows a screen call recording Data flow, starting from the point where a call starts, and details how the information appears in the log files. This enables you to monitor the call flow, and troubleshoot the problems that may arise during the recording process, up to the point when the call is inserted into the database. See Interactions-based Call Recording Data Flow Details on page 358 for a description of every step mentioned in Figure 13-4.
Figure 13-4 Screen Call Recording Data Flow

Screen Call Recording Flow Details
1. The CTI Server sends a CTI event to the Driver. 2. The Driver processes the event. 3. The Driver sends a Start Call event to the Call Server. 4. The Call Server checks and processes the Start Call event: a.

The Call Server verifies that the agent ID or extension is recognized and exists. This check is done by accessing an internal login table (Calls_LT) which holds all agent login information. QA/Negative/Selective rule.
Appendix 13: Interactions Center Troubleshooting Flow 363

b. The RecordingDecisions checks if the new calls match a recording request or a

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Understanding the Screen Call Recording Data Flow

c.

The Call Server updates an internal table (Calls_CT) with the call details, that is, with the start time, agent details, extension number, and so on.

5. The Call Server sends a Start Record event request (including all participants) to the RCM

for voice and per participant.
6. The RCM returns channel allocation information to the Call Server (Synchronous Answer)

including the Logger #, Logger channel and time information.
7. The RCM sends the Logger a Start Recording command (voice). 8. The Call Server updates Calls_CT with the recording details, including the Logger #, Logger

channel, Logger start time, and so on.
9. Screen: The Call Server sends a Start Record event request to the RCM for screen media. 10. Screen: The RCM returns channel allocation information to the Call Server (Synchronous

Answer).
11. Screen: The RCM calls the Capture Control DLL which calls the ScreenAgent. 12. Screen: The ScreenAgent starts recording. 13. The Call Server sends a Start Call event to the Client, for example, Monitor (the client must

be registered for these type of events according to the Agent or Station and switch).
14. The Logger records the call. 15. The Logger returns a Recording Succeeded/Failed event to the RCM. 16. The RCM returns an Asynchronous Response to the Call Server. In interactions-based

recording, the real call start time is returned here. The call start time is the time that RCM received a response from the Logger that it has started recording, normalized to the Logger local time.
17. The RCM sends an Audio Check command to the Logger 2.5 seconds (configurable) after the Asynchronous Response is sent to the Call Server. 18. Screen: The ScreenAgent returns an Asynchronous Response to the RCM. 19. Screen: The RCM returns an Asynchronous Response to the Call Server. 20. The Call Server sends a Start Record event to the Client, for example, Monitor (the client

must be registered for these type of events according to the Agent or Station and switch).
21. For VoIP configurations only, the RCM waits a configured amount of time and performs an

audio check with the VoIP Logger. If the Logger has not received any audio packets from the beginning of the call, an asynchronous Record Failure event is sent to the Call Server.
22. Screen: If there is a problem, the ScreenAgent returns an Asynchronous Recording Error

to the RCM.
23. Screen: If the screen recording fails: a.

The RCM reports a Record Failure request to the Call Server. The RecordingDecisions answers the Call Server as to whether the voice should be stopped.
Appendix 13: Interactions Center Troubleshooting Flow 364

b. The Call Server asks RecordingDecisions if the voice should be stopped. c.

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Understanding the Screen Call Recording Data Flow

d. The Call Server sends a Stop Record request to the RCM for the voice. 24. The CTI Server sends a CTI event to the Driver. 25. The Driver sends an End Call event to the Call Server. 26. The Call Server checks if the calls are recorded in its internal database. 27. The Call Server sends a Stop Record event request to the RCM. 28. The RCM returns the stop request information with the recording status of the call to the Call

Server.
29. The RCM sends a Stop Record event to the Logger (voice and screen). 30. The Logger stops recording the call. 31. Screen: The Call Server sends a Stop Record request to the RCM for screen recording. 32. Screen: The RCM sends a Stop Record request to the ScreenAgent. NOTE: Screen: If a screen wrap-up time is defined, that is, the total amount of time (in seconds) it takes for the agent to wrap-up the interaction, the RCM waits for this period before sending the request. 33. Screen: The ScreenAgent returns an Asynchronous Response to the RCM. 34. The Call Server updates the call details including the stop time and removes this call from

Calls_CT.
35. The Call Server sends an Insert Interaction event request to the DBServer (this is based on having a Start and Stop message for the same call Interaction ID). NOTE: Screen: For screen recording with a wrap-up time defined, the Call Server does NOT wait before sending the Insert Interaction event, it simply uses the wrap-up time to calculate the call stop time. This means that if a screen recording problem occurs during the wrap-up time, there will be no indication of the problem in the database and no exceptions will be thrown. You will simply not be able to playback the screen recording to the end. If screen is still recording in the wrap-up time, and the agent answers a new call, the screen wrap-up recording will be stopped. The Call Server will update the stop time in the database with the actual wrap-up time. 36. The Call Server sends an Insert Contact request to the DBServer including data about nested

interactions.
37. The Call Server sends a Stop Call (with recording information) event to the Client. 38. The DBServer inserts the interaction into the SQL Database on the Database Server.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

365

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Open Case Procedure
This section tells you what information to provide if you need to send a problem to customer support at NICE for the following:

• • • • • •

RecordingDecisions on page 366 Call Server on page 367 DB Server on page 368 TRS on page 369 RCM on page 369 System Administrator Parameters (CLS Plug-ins) on page 371

RecordingDecisions
Data to be provided in RecordingDecisions case reports, when contacting customer support:

Details regarding System configuration:

• • • • • • • • • • •

Exact Interactions Center release and service pack Number of Interactions Centers on sites Any special site configuration, for example e.g., Multi Data Hubs, cluster Total/QA Number of rules configured Number of users on site

Problem description: Relevant rules defined Expected behavior Actual behavior Actions taken
Appendix 13: Interactions Center Troubleshooting Flow 366

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedure

• •

Provide specific examples and call details (for example, Interaction ID, Segment ID, Extension number, Agent number). Complete log files, in debug level (located on the computer where the Interactions Center sits, under NICE Systems > Interactions Center > Log)

• • • •

Call Server.log.*.txt NiceCls.RecordingDecisions.*.log.*.txt

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide the following database tables in CSV format:

nice_rule

• • • • • • • •

tblSchedule tblScheduleActivation tblRERuleondition tblReRuleConditionItem

nice_interactions Sched_Programs_Recording_Count nc_general

Provide any other relevant data, for example, the users included, memory usage, screen shots.

Call Server
Data to be provided in Call Server case reports, when contacting customer support:

Details regarding System configuration:

• • • • • • • • • •

Exact Interactions Center release and service pack Number of Interactions Centers on sites Any special site configuration, for example e.g., Multi Data Hubs, cluster Total/QA

Problem description: Expected behavior Actual behavior Actions taken

Provide specific examples and call details (for example, Interaction ID, Segment ID, Extension number, Agent number). Complete log files, in debug level (located on the computer where the Interactions Center sits, under NICE Systems > Interactions Center > Log)
Appendix 13: Interactions Center Troubleshooting Flow 367

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedure

• • •

Call Server.log.*.txt

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide database query results, regarding problematic time frame or problematic interaction ID/contact ID, from the following database tables:

• • • • • • • •

tblInteraction tblCallInteraction tblContact tblContactInteraction tblParticipant tblRecordign tblExceptionXX

Provide any other relevant data, for example, the users included, memory usage, screen shots.

DB Server
Data to be provided in DB Server case reports, when contacting customer support:

Details regarding System configuration:

• • • • • • • • • • •

Exact Interactions Center release and service pack Number of Interactions Centers on sites Any special site configuration, for example e.g., Multi Data Hubs, cluster Total/QA

Problem description: Expected behavior Actual behavior Actions taken

Provide specific examples and call details (for example, Interaction ID, Segment ID, Extension number, Agent number). Provide the number and size of persistent queue files. Complete log files, in debug level (located on the computer where the Interactions Center sits, under NICE Systems > Interactions Center > Log)

• •

DB Server.log.*.txt

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

368

Open Case Procedure

• •

In the case of an SQL time-out, provide a list of running times for all SQL jobs on the Interaction database. Provide any other relevant data, for example, the users included, memory usage, screen shots.

TRS
Data to be provided in TRS case reports, when contacting customer support:

Details regarding System configuration:

• • • • • • • • • •

Exact Interactions Center release and service pack Number of Interactions Centers on sites Any special site configuration, for example e.g., Multi Data Hubs, cluster Total/QA

Problem description: Expected behavior Actual behavior Actions taken

Provide specific examples and call details (for example, Interaction ID, Segment ID, Extension number, Agent number). Complete log files, in debug level (located on the computer where the Interactions Center sits, under NICE Systems > Interactions Center > Log)

• • • • •

TRS.log.*.txt

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide any other relevant data, for example, the users included, memory usage, screen shots. Provide the exact time of the recording that should or should not be inserted by the TRS. Provide the tblRecordingXX database query results, regarding the problematic time frame.

RCM
Data to be provided in RCM case reports, when contacting customer support:

Details regarding System configuration:

• • •

Exact Interactions Center release, service pack and NICE Updates installed. Details of all Loggers configured to each Interactions Center: version, capture type, number of channels, recording method, GMT and local time of the Logger. Mapping details for each Logger attached to the Interactions Center:

Total/Interaction Based
Appendix 13: Interactions Center Troubleshooting Flow 369

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedure

• • • • • • • • • • •

Recording method of Logger channels (for example, Passive VoIP, DMCC, Nortel DMS, Digital Matrix) Type of Sources and their number Static/Dynamic Mapping Connection between Logger and Interactions Center (LAN/WAN). If WAN, please indicate Bandwidth and Latency. N+1 chain details (when relevant)

Details from CTI plug-in on the drivers and the type of interfaces (for example, Avaya TSAP, Genesys, CISCO ICM etc). Site's SBC / Gateway topology (when relevant). Number of ScreenAgents reporting to the Interactions Center (when relevant) Call Rate on the Interactions Center Machine Work hours of agents mapped to the Interactions Center, including peak hours (for example, hours of shifts, 24/7) Problem description:

• • • • • •

Expected behavior Actual behavior Actions taken

Provide specific examples and call details (for example, Interaction ID, Segment ID, extension number, Agent number, time of the call, relevant DB Records for the call). If the issue occurs in specific call scenarios, describe the scenario. Does the issue occur in a specific type of call? (for example, only internal calls)

IMPORTANT Before sending log files for RCM, trigger RCM to dump the full internal state to the log file the logs, by running the RCM Information Dumper. See Running the RCM Information Dumper on page 376.

Complete log files, in debug level (located on the computer where the Interactions Center sits, under NICE Systems > Interactions Center > Log):

• • • • •

Call Server.log.*.txt RCM.log.*.txt

Complete log files, in debug level, of drivers attached to the Interactions Center. Complete log files of Loggers (including capture logs) attached to the Interactions Center. All RCM configuration backup files found under NICE Systems > Interactions Center >
Loggers
Appendix 13: Interactions Center Troubleshooting Flow 370

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedure

Files found under NICE Systems > Interactions Center -> bin

• • • •

RCM.log.Persistency.soap RCM.exe.config

Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide the following database tables in CSV format (query from before the time the problem occurred, preferably overlapping with the time in the provided logs):

nice_interactions

• • • • • • • • • • • • • •

tblInteraction tblRecording tblParticipant tblException

nice_admin Channel mapping tables

The following Performance Counters of the RCM.Exe process: Private Bytes Threads Handles % CPU Page Faults / Sec All Performance Counters under the .Net Memory Category.

Provide any other relevant data, for example, the users included, memory usage, screen shots.

System Administrator Parameters (CLS Plug-ins)
Data to be provided in Interactions Center Parameters (CLS Plug-ins) case reports, when contacting customer support:

Details regarding System configuration:

• • • • • •

Exact Interactions Center release, service pack, and installed Updates Any special site configuration, for example , Multi Data Hubs

Problem description: Expected behavior Actual behavior Actions taken
Appendix 13: Interactions Center Troubleshooting Flow 371

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Open Case Procedure

• • • • • •

Is the problem related to a specific configuration (such as a specific integration or mapping type) or does it effect all configurations? Does the problem reoccur after restarting the System Administrator or browser? Is there a work around? Are there also problems with additional parameters, such as Integrations parameters, Loggers parameters, DB Server parameters.

Provide specific examples and details Complete log files, in debug level from the server (located on the computer where the Interactions Center sits, under NICE Systems > Applications> ServerBin)

• • • • •

NiceApplications.CLSPlugin.log.*.txt

A backup copy of the nice_admin database (located on the computer where the database is installed) Please point to the relevant times in the logs when the problem can be seen in the logs. Quote the lines if you can. Provide a screenshot of an error if one is displayed. Provide Performance Counters of SysAdmin process:

• • • • • • • • •

Private Bytes Threads Handles % CPU Page Faults / Sec All performance counters under the .Net Memory Category.

In case of Out Of Memory issues, please provide dump of SysAdmin process. In case of Online Configuration Update issues, please also provide full Interactions Center Logs. Provide any other relevant data, for example, memory usage or screen shots.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

372

Checking the Basics

Checking the Basics
For Interactions Center problems check the following:
1. View the SNMP traps. See Checking the SNMP Traps on page 374. 2. Set the reporting File level to Debug. See Setting the Reporting Level on page 375. 3. Collect all the logs and other relevant files. See the Troubleshooting Guide, Collecting

Information Using the Log Collector.
4. Check the CLS configuration. See Checking the CLS Configuration on page 377. 5. If many calls are not recorded, restart the RCM process. See Starting/Stopping the

Interactions Center Services on page 378.
6. If connectivity or power was lost, or if restarting the RCM process does not solve the problem,

restart the NICE CLS services. See Starting/Stopping the Interactions Center Services on page 378.
7. Troubleshoot using the Interactions Center logs. See Troubleshooting using the Interactions

Center Logs on page 381 and Troubleshooting using the Interactions Center Logs on page 381.
8. Check the ports. See Checking Ports on page 408. 9. Check the CLS exceptions. See Checking the CLS Exceptions on page 410. 10. Fine-tuning the CLS Server configuration parameters. See Fine-tuning the CLS Server

Configuration Parameters on page 411.

NOTE: For NiceScreen Logger and ScreenAgent problems, see the NiceScreen Troubleshooting Flow.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

373

Checking the SNMP Traps

Checking the SNMP Traps
In the SNMP Console, view the events (polling failure messages and the SNMP traps) generated by the monitored servers. See the NMS User's Guide for a list of the trap messages and recommended actions.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

374

Setting the Reporting Level

Setting the Reporting Level
Set the File reporting level to Debug for the various modules and services.

To set the reporting level: 1. In System Administrator verify that you are working in Technician mode: From the Settings menu, select Technician Mode.

2. In the Organization tree, select CLS Definitions > CLS Server. Click the Report Level tab.

3. Expand the required component and set the reporting level.

NOTE: Currently the Screen reporting level category is not available, although it appears on the screen. Changing this field has no effect on the system. 4. Set the File to Debug.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

375

Running the RCM Information Dumper

Running the RCM Information Dumper
The RCM Information Dumper triggers the RCM to dump a full internal state to the log file. You must run the RCM Information Dumper, before sending RCM log files to NICE Customer Support.
To run the RCM Information Dumper: 1. Navigate to NICE Systems\Interactions Center\Tools\RCMInformationDumper.exe 2. Double-click RCMInformationDumper.exe.

The RCM Information Dumper window appears.
Figure 13-5 RCM Information Dumper

3. Select Resources Dump, and click DUMP. 4. Select Requests Manager Dump, and click DUMP.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

376

Checking the CLS Configuration

Checking the CLS Configuration
Verify that the CLS Servers are correctly defined in the System Administrator. For more details, see the System Administrator's Guide.
To check the configuration: 1. In System Administrator, verify that you are working in Technician mode: In the Organization tree, click Organization. Then select Technician Mode and click Save. 2. In the Organization tree, select CLS Definitions > CLS Server. Verify that the Interactions

Centers are correctly defined.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

377

Starting/Stopping the Interactions Center Services

Starting/Stopping the Interactions Center Services
The following services run on the Interactions Center:

• • •

NICE CLS NICE CLS Monitor NICE CLS TRS

To Start/Stop the NICE CLS Service: 1. Start/stop the NICE CLS service from Services. -or2. From the Start menu, select Programs> Nice Systems > Interactions Center > Tools > Nice Services Configuration Manager.

The NICE Services Configuration Manager window appears.
Figure 13-6 NICE Services Configuration Manager Window

3. Select the required service and click

to start the service, or click

to stop the service.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

378

Checking the Status of Interactions Center Components

Checking the Status of Interactions Center Components
You can use the NICE Services Configuration Manager utility to monitor the status of the services run on the Interactions Center. In addition, you should check the status of processes related to the NICE CLS component which can be viewed via the Windows Task Manager.

Checking Components with the NICE Services Configuration Manager
NOTE: For all-in-one sites, before proceeding ensure that the NICE Applications Suite’s instance of the NICE Services Configuration Manager tool is not already running. To check the components status with the NICE Services Configuration Manager:

From the Start menu, select Programs> Nice Systems > Interactions Center > Tools > Nice Services Configuration Manager. The NICE Services Configuration Manager window appears.
Figure 13-7 NICE Services Configuration Manager Window

Under the State column the status should state Running for the following:

• • •

NICE CLS NICE CLS Monitor NICE CLS TRS

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

379

Checking the Status of Interactions Center Components

Checking Processes Status
Figure 13-8 Windows Task Manager

To check the NICE CLS processes status: 1. Open the Windows Task Manager. 2. Click the Processes tab. 3. Ensure that the following processes appear:

• • • • •

CallsrvrHost.exe DBSrvr.exe DISPATCH.exe NiceCls.RecordingDecisions.Scheduler.exe RCM.exe

4. Check for information, such as excessive CPU usage, excessive memory usage, and memory

leaks.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

380

Troubleshooting using the Interactions Center Logs

Troubleshooting using the Interactions Center Logs
Each component writes events (based on the selected reporting level) into its own log file. Log files are located in D:\Program files\NICE Systems\Interactions Center\Log. Also see Examples of Troubleshooting using the Logs on page 383.
NOTE: The Technical CLS API Programmer's Reference Guide may be useful when analyzing the log files.
Figure 13-9 Interactions Center Log Files

There are log files for:

• • • • • • • • •

Call Server - Call Server.log.xxx.txt files CLS Monitor - CLS Monitor.log.xxx.txt files CLS Manager - CLSManager.log.xxx.txt files DBServer - DBServer.log.xxx.txt files Dispatch - Dispatch.log.xxx.txt files RCM - RCM.log.xxx.txt files NiceCls.RecordingDecisions.Scheduler NiceCls.RecordingDecisions.Scheduler.log.xxx.txt files

Snmp - snmp.log.xxx.txt files TRS - TRS.log.xxx.txt files

To troubleshoot using the logs: 1. Check the logs. From the logs it can be seen that an attempt was been made to record both voice and screen using a rule (CLS_INITIATOR_QA (16)). The Call Server sent RCM two

requests, one to record voice and one to record screen, but both failed.
Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 381

Troubleshooting using the Interactions Center Logs

2. Check the rules. If necessary delete and recreate the Scheduler rules. 3. The Logger is not mapped correctly (switch 2, trunk 202, trunk group 3), and therefore the

voice recording fails.
4. Check the mapping.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

382

Examples of Troubleshooting using the Logs

Examples of Troubleshooting using the Logs
Example 1
For example, exception exp016 is reported for some of the calls which were not recorded. The following logs appear:
Request to record voice:

01/08/2006 11:34:08.500 DEBUG [7928]: RCM Request data : RecordStartRequest: requestID=110331462; interactionID=110331462; callStartTimeInTicks=632900090480000000; clsDriverTimeDiff...=0; callStartTime=8/1/2006 6:04:08 AM; previousCallID=0; PBXCallID=14678; switchID=2; urid=0; type=33620224 (02010100); timeReceived=632900288485000000; timeUpdatedCounter=0; RequestInternalParticipant: deviceID=0; deviceType=0; station=38053; agentID=39268; type=REQUEST_INTERNAL_PARTICIPANT (02010000); recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0); timeDifference=0; recordingStartTim...=0; recordingStartTime=1/1/0001 12:00:00 AM; recordingStopTime...=0; recordingStopTime=1/1/0001 12:00:00 AM; recordingChannel=0; recordingLogger=0; requestInitiators=CLS_INITIATOR_QA (16); requestMediaType=VOICE_MEDIA (2); requestWrapUpTime=0; requestWrapUpTime...=0; summationType=SUMMATION_TYPE_SUMMED (3); participantID=1; ipPort=0; uniqueDeviceID=38053; ExternalParticipant: trunkNumber=202; trunkGroup=3; type=EXTERNAL_PARTICIPANT (03000000); recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0); timeDifference=0; recordingStartTim...=0; recordingStartTime=1/1/0001 12:00:00 AM; recordingStopTime...=0; recordingStopTime=1/1/0001 12:00:00 AM; recordingChannel=0; recordingLogger=0; requestInitiators=CLS_INITIATOR_QA (16); requestMediaType=VOICE_MEDIA (2); requestWrapUpTime=0; requestWrapUpTime...=0; summationType=SUMMATION_TYPE_SUMMED (3); participantID=2; ipPort=0; (LogWriter.LOGWrite2OldLog)
Request to record screen:

01/08/2006 11:34:08.531 DEBUG [7928]: RCM Request data : RecordStartRequest: requestID=109733342; interactionID=110331462; callStartTimeInTicks=632900090480000000; clsDriverTimeDiff...=0; callStartTime=8/1/2006 6:04:08 AM; previousCallID=0; PBXCallID=14678; switchID=2; urid=0; type=33620224 (02010100); timeReceived=632900288485000000; timeUpdatedCounter=0; RequestInternalParticipant: deviceID=0; deviceType=0; station=38053; agentID=39268; type=REQUEST_INTERNAL_PARTICIPANT (02010000); recordingFlags=CHANS_REC_FLAG_FULL_VOICE (4); chosenInitiator=0 (0); timeDifference=0; recordingStartTim...=0; recordingStartTime=1/1/0001 12:00:00 AM; recordingStopTime...=0; recordingStopTime=1/1/0001 12:00:00 AM; recordingChannel=0; recordingLogger=0; requestInitiators=CLS_INITIATOR_QA (16); requestMediaType=SCREEN_MEDIA (1); requestWrapUpTime=60; requestWrapUpTime...=60000; summationType=SUMMATION_TYPE_SUMMED (3); participantID=1; ipPort=0; (LogWriter.LOGWrite2OldLog)
Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 383

Examples of Troubleshooting using the Logs

Record voice response: 01/08/2006 11:34:08.593 DEBUG [7928]: RCM Start Record Request(Voice) response data : ResourceAlocatedResponse: requestID=110331462; interactionID=110331462; eventType=CHANS_EVENT_REC_FAIL (4); eventReason=CHANS_REC_VOICE_LOGGER_GENERAL_FAILURE (1402); urid=0; type=RESOURCE_ALLOCATED_RMO_RESPONSE (03010400); timeReceived=632900288485156250; timeUpdatedCounter=0; ResponseInternalParticipant: recordingLogger=10959703; recordingChannel=44;

Example 2

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

384

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

385

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

386

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

387

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

388

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

389

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

390

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

391

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

392

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

393

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

394

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

395

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

396

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

397

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

398

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

399

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

400

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

401

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

402

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

403

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

404

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

405

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

406

Examples of Troubleshooting using the Logs

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

407

Checking Ports

Checking Ports
Check that the required ports are open.
To check connectivity:

Check that the following ports are open.
Table 13-1: Required Ports Proto col TCP Port 62051 Destination (Server) Call Server on NICE Interactions Center Source(Client) ScreenSense Server Applications Server Playback Server CTI (drivers) TCP 62050 Call Server on NICE Interactions Center DB Server on NICE Interactions Center CAPI integrations CLS API Control of system recording. Passing requests to insert/update interactions and contacts to/in the database. Passing notifications for SNMP traps from CLS processes to CLS SNMP Agent. Handling recording requests from Call Server. Control ScreenAgent recording for NAT/PAT environments. The Call Server uses this port for keep alive with CAPI Client. Service .NET Remoting (HTTP) Description Control of system recording.

TCP

62055

Call Server on NICE Interactions Center

.NET Remoting (HTTP)

TCP

62059

CLS SNMP Agent (part of SNMP.exe service) on NICE Interactions Center RCM on NICE Interactions Center Screen Capture Control on NICE Interactions Center

All CLS processes on the NICE Interactions Center Call Server on NICE Interactions Center Agents desktops (screen recording)

.NET Remoting (HTTP) .NET Remoting (HTTP) .NET Remoting (HTTP)

TCP

62069

TCP

2010

TCP

62065

Call Server

CAPI Client

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

408

Checking Ports

Table 13-1: Required Ports (Continued) Proto col UDP Port 162 Destination (Server) NMS Source(Client) SNMP Service on Applications Server SNMP Service on NICE Interactions Center SNMP Service on Playback Server SNMP Service on Voice/VoIP Logger SNMP Service on NiceScreen Logger SNMP Service on Storage Center VoIP Recording Gateway Service SNMPv1 Description Unsolicited operations (traps) sent by the SNMP Service for alarming.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

409

Checking the CLS Exceptions

Checking the CLS Exceptions
Query the Database
Query the database to see the exceptions. Most of the time exceptions are connected to recordings and not directly to the interaction IDs. In order to see the connection between an exception and interaction ID, query both the exception details and the related recording (which includes the interaction ID). See the Recording Troubleshooting workflow for a list of CLS exceptions. For more details, see the Interaction Data Reference Guide.
To check the CLS Exceptions: 1. Query the tblInteractionCatalog table to check over what period the calls occur. 2. Get all the exceptions relating to interactions: SELECT * FROM tblExceptionXX, tblInteractionXX WHERE tblExceptionXX.iInteractionID = tblInteractionXX.iInteractionID 3. Get all exceptions relating to recordings: SELECT * FROM tblExceptionXX, tblRecordingXX WHERE tblExceptionXX.iRecordingID = tblRecordingXX.iRecordingID 4. Check the vcExceptionDetail field in tblException.

In the Business Analyzer
To check the CLS exceptions in Business Analyzer:

In Business Analyzer, select the interaction and click on the Exceptions tab, in the Details pane.
Figure 13-10 Business Analyzer Exceptions Tab

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

410

Fine-tuning the CLS Server Configuration Parameters

Fine-tuning the CLS Server Configuration Parameters
You can modify the CLS Server configuration parameters.
To fine tune the CLS Server configuration: 1. Verify that you are working in Technician mode: From the Settings menu, select Technician Mode.
Figure 13-11 Technician Mode

2. In the Organization tree, select CLS Definitions > CLS Server. 3. Select the required CLS Server, and click the Configuration tab.
Figure 13-12 CLS Server Configuration Tab

4. Modify the required parameter. For example, if you get an Internal Call Server Table is Full

error:
a. Click

next to the Call Server.

b. Increase the values for SegmentTableMaxEntries, LoginTableMaxEntries and CompoundTableMaxEntries.

See the Interactions Center Installation Guide for a complete list of CLS Server configuration parameters.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

411

Isolating a Screen Problem Further

Isolating a Screen Problem Further
Try to isolate the problem and identify the faulty component.
To isolate the problem further:

Try and isolate the problem by checking the logs to see:

• •

If the StartRecord command was called by the RCM. See the NiceScreen Troubleshooting Workflow. If the ScreenAgent received the StartRecord command. See the NiceScreen Troubleshooting Workflow. The following is a sample StartRecord in the ScreenAgent log: INFO: 02/01/08 10:56:31:015 IncCommandsHandler: StartRecording received from IC IpAddr(screen-env3:2010). capture handle(10000001), DestinationAddr(screen-env3:30100), DestinationResourceId(93); Context = (676)

If NiceScreen received the StartRecord command. See the NiceScreen Troubleshooting Workflow.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

412

Shutting Down and Starting CLS

Shutting Down and Starting CLS
To shut down the Interactions Center: 5. From the Start menu, select Programs> Nice Systems > Interactions Center > Tools > Nice Services Configuration Manager.

The NICE Services Configuration Manager window appears.
Figure 13-13 NICE Services Configuration Manager Window

6. Click -or-

.

Stop the NICE CLS service.
7. Shut down Windows normally. To start the Interactions Center: 1. Power up the machine and log in via the console or PCAnywhere using the admin account.

This should launch all services, startup apps, the CLS Dispatch process, and all CLS daughter processes. If auto-login is configured or the CLS is running as a service, even a Windows login is unnecessary.
2. To start the NICE CLS service, press

on the NICE Services Configuration Manager

window (see Figure 13-13),
-or3. Start the NICE CLS service.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

413

Other Troubleshooting Issues

Other Troubleshooting Issues

If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file.

IMPORTANT Any change of configuration files should be done ONLY in consultation with NICE Support.

• •

For NiceScreen Logger and ScreenAgent problems, see the NiceScreen Troubleshooting Workflow. During a NICE Perform installation of CLS 9.13.04.11, the following message appears after entering the CLS login credentials, “Input string was not in a correct format”. The installation fails. Set the regional settings of the CLS to English. The NICE Perform application receives messages with "NICE CLS Error Code 20" when trying to start ROD under monitor modules. The system stops recording audio or screens. Check that there is a sufficient number of TSAPI licenses. In the event of a change in configuration, restart the relevant service. When running RemonitorApplication.exe, an error appears “Failed To create remonitor event”. This utility is designed for use in an Avaya Integration only. If you are doing your mapping in System Administrator (and not manually using a file):

• • •

• • • •

In CLS Definitions > CLS Servers, select a CLS Server. Click the Configuration tab and click RCM. Validate that the Read Mapping from File parameter value is set to No.

The MaxHistoryLengthHours parameter can also be used (see Fine-tuning the CLS Server Configuration Parameters on page 411):

• • •

If the customer wants to insert older calls this parameter can be increased. When retrieving calls from a tape backup to a Logger and then using TRS to insert the calls into the database.

A problem may arise from a feature called "silent whisper". The whisper is a sound bite that is played to the agent to make them aware what type of call is coming through for example, sales, fraud, and so on, before they are connected to the customer. If this feature is not fully configured and there is silence for several seconds at the beginning of the call when the call is connected, that is, there is no sound bite configured, no audio packets are sent through to the NICE system and recording fails. If you ROD the call after the silenced part of the call, the call is recorded. In order to compensate this gap of silence change the CheckforAudioDelay RCM parameter to 7000 ms. For RTP sniffing, this is the period in milliseconds that the RCM waits after starting a call before checking for audio availability on the Logger.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

414

Other Useful Tools

Other Useful Tools
CAPI Tester. See Tier 4 for further information.

Appendix 13: Interactions Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

415

Blank page for double-sided printing.

14
NMS Troubleshooting Flow

Contents
Workflow........................................................................................................................418 Gathering Information ..................................................................................................419 Troubleshooting NMS...................................................................................................420 Client-Side Troubleshooting........................................................................................421 Server-Side Troubleshooting.......................................................................................427 Changing the System Administrator Server Name ...................................................445 Checking for a CAClient Serialization Issue ..............................................................445 Other Troubleshooting Issues.....................................................................................445 Using the NMSUtil Tool ................................................................................................446

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

417

Workflow

Workflow
If you experience a problem with the NMS follow the steps below to isolate the problem.
Figure 14-1 Troubleshooting Workflow

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

418

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were any changes done to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

419

Troubleshooting NMS

Troubleshooting NMS
NOTE: Some of the procedures include instructions that can also be executed using NMSUtil. This tool is available on the Tier 4 Team Site. See Using the NMSUtil Tool on page 446.

For NMS problems see the following:
1. Client-Side Troubleshooting: a.

Pinging the Monitored Server on page 421 Verifying that SNMP Agent is Sending Traps on page 425 Reinstalling the NICE Agents on page 426 (not recommended for production sites) Collecting the System Log Files on page 427 Checking the Trap Mechanism on page 435

b. Checking the SNMP Service Configuration on page 421 c.

d. Repairing the SNMP Service on page 425 e.

2. Server-Side Troubleshooting a.

b. Checking the NMS Services on page 428 c.

d. Checking the MIB File on page 439 3. Checking Polling Timeout Errors on page 440 4. Troubleshooting Scenarios

• • • • •

Scenario 1: Monitored Server Does Not Appear in the 2D Map on page 442 Scenario 2: Drilldown Fails in the 2D Map on page 442 Scenario 3: Only Some Agents are Reporting on page 442 Scenario 4: There are No Traps in the Console on page 442 Scenario 5: No Email Notifications Are Received on page 443

5. Changing the System Administrator Server Name on page 445 6. Checking for a CAClient Serialization Issue on page 445

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

420

Client-Side Troubleshooting

Client-Side Troubleshooting
In cases where specific agents or monitored servers do not send traps, the problem most likely is a client-side problem. See the following:

• • • •

Pinging the Monitored Server on page 421 Checking the SNMP Service Configuration on page 421 Verifying that SNMP Agent is Sending Traps on page 425 Repairing the SNMP Service on page 425

Pinging the Monitored Server
Check that the NMS Server can ping the monitored server.
To ping the monitored server:

From the NMS Server, ping the monitored server. Optionally, you can use the NMSUtil (where available) to ping the monitored server. See Using the NMSUtil Tool on page 446.

Checking the SNMP Service Configuration
Verify that the SNMP Service on the monitored server is running, and that it is configured correctly. Ensure that the latest DLL files are installed.
To check the SNMP Service configuration: 1. On the client machine, verify that the status of SNMP Service is Started. 2. If the SNMP Service is not started, start the service and verify that there are no service errors

in Event Viewer.
Figure 14-2 SNMP Service

3. Right-click SNMP Service and select Properties.
Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 421

Client-Side Troubleshooting

4. Click the Traps tab.
Figure 14-3 SNMP Service Properties - Traps Tab

5. Verify the Community name is public (in lower case letters). 6. Verify the Trap destinations by checking the IP addresses (or computer names) of the NMS

servers where trap messages will be sent.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

422

Client-Side Troubleshooting

7. Click the Security tab.
Figure 14-4 SNMP Service Properties - Security Tab

8. Confirm that:

• •

The Send authentication trap checkbox is not selected. In the Accepted community names area, public appears with READ ONLY privileges and admin appears with READ-WRITE privileges.

IMPORTANT If your site uses customized community names, you must change public and admin to the appropriate community name. 9. Ensure that Accept SNMP packets from any host is selected (localhost appears in the list.)

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

423

Client-Side Troubleshooting

10. In the registry, open the following key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SNMP\Parameters\E xtensionAgents
Figure 14-5 Registry Editor - Extension Agents

This lists the various agents that are loaded when the SNMP service loads, and the Data column indicates the location of the agent key registry.
11. Navigate to the agent key to view the location of the agent DLL.
Figure 14-6 Registry Editor - Agent Key

12. Verify that the DLL is in this folder, that it is the latest version and that it is not corrupt. If

necessary, consult with NICE Support.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

424

Client-Side Troubleshooting

Verifying that SNMP Agent is Sending Traps
Verify that the Monitored Server's SNMP agent has initialized correctly and is sending traps.
To verify that the SNMP agent is sending traps: 1. Copy Nice Systems\Applications\Tools\SNMP Tools\dbwin32.exe from the Applications

Server to the Monitored Server.
2. On the Monitored Server, double-click .dbwin32.exe. 3. Check whether the SNMP agent is alive. If there are no messages, or no indication that the

SNMP agents are up or no errors this may indicate a problem.
Figure 14-7 Debug Messages

Repairing the SNMP Service
The SNMP Service may need to be repaired.
To repair the SNMP Service: 1. In the registry, open the following key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SNMP\Parameters\E xtensionAgents 2. Backup the registry key. 3. Remove the SNMP Service using the Windows Add or Remove Programs. 4. Reinstall the SNMP Service from Add/Remove Windows Components option. 5. Ensure that you have the Windows installation disk or have access to the I386 folder. 6. Reload the registry key.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

425

Client-Side Troubleshooting

Reinstalling the NICE Agents
You can reinstall some of the NICE Agents if there is a problem during installation.
IMPORTANT Reinstalling the NICE Agents must be done ONLY in consultation with NICE Support. This is NOT recommended for production sites.

To reinstall the NICE Agents:

After installing the SNMP Service, install the NICE Agents. Several of the components have a separate installation for the SNMP agents while others install the SNMP agent during the software installation.
Component Screen Logger Where/How to Reinstall Nice SNMP Agents Screen Logger has a separate SNMP Agent installation which can be removed and reinstalled without harming the Screen Logger installation. Install separately from the NiceCall Focus III software. See Installing the OS/SQL Agent (Optional) Appendix, in the NMS Installation Guide, for further details. See Installing the OS/SQL Agent (Optional) Appendix, in the NMS Installation Guide, for further details. Telephony Services Server and Stream Server have separate SNMP Agents that can be removed and reinstalled without harming the Telephony Services Server and Stream Server installations. Installed as part of the VRG. The SNMP agent is part of the image prepared and delivered with the VRG. Installed as part of the Logger software. The SNMP agent is part of the image prepared and delivered with the Logger. Installed as part of the Interactions Center. Installed as part of the Integration. Installed as part of the Applications Server. Installed as part of the Storage Center. See Installing the SNMP Script Agent, in the NMS Installation Guide, for further details.
Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 426

Accusys RAID OS Agent to monitor the System Administrator service SQL Agent to monitor the SQL

Telephony Services Server and Stream Server

VRG Logger

Interactions Center Integration (Driver) Applications Server Storage Center NMS Script Agent

Server-Side Troubleshooting

Server-Side Troubleshooting
If none of the SNMP Agents or monitored servers are able to send traps, the problem may be located on the NMS server. See the following:

• • • •

Collecting the System Log Files on page 427 Checking the NMS Services on page 428 Checking the Trap Mechanism on page 435 Checking the MIB File on page 439

Collecting the System Log Files
There are log files for the various system services and processes running on the NMS Server. Note that these log files are different from the trap messages logs.
To collect the system log files: 1. Navigate to <Installation Drive>:\TND\SERVICES\VAR\LOG.
Figure 14-8 Services Log

2. Open the required log file, for example, aws_sadmin.log.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

427

Server-Side Troubleshooting

Figure 14-9

Opening the Log File

3. Send a zip file of the log files when logging a case.

Checking the NMS Services
The following services run on the NMS Server:

CA-Unicenter: One of the main services of the Enterprise Manager (Console), this service is responsible for displaying new messages on the NMS Console. This service starts up several sub-services. Unicenter TND Agent Technology: The main service for all agent technology-related

• • •

services (DSM, 2-D Map). This service starts up several sub-services.
SNMP Trap Service: This service is responsible for the polling of SNMP agents that send

traps to the server.
SNMP Nice Trap Formatter: This service receives traps after basic processing, and

"translates" the traps into the format in which they are displayed in the Console.
NOTE: You can also use the NMSUtil for some of these checks. See Using the NMSUtil Tool (where available).

Checking the CA-Unicenter Service and Sub-Services
One of the main services of the Enterprise Manager (Console), this service is responsible for displaying new messages on the NMS Console. This service starts up several sub-services.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

428

Server-Side Troubleshooting

To check the CA-Unicenter service: 1. Check that the CA-Unicenter service is started. If the service is stopped no new events are

displayed.
Figure 14-10 CA-Unicenter Service

2. If necessary, restart the CA-Unicenter service. The following message may appear:
Figure 14-11 CA-Unicenter Services Error Message

3. If this message appears, the caunint user password must be updated. The CA-Unicenter

service is run under the caunint user which is created automatically during the NMS installation. Update the caunint user password as follows:
a.

Right-click My Computer and select Manage. The Computer Management window appears.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

429

Server-Side Troubleshooting

Figure 14-12

Computer Management Window

b. Right-click caunint and select Set Password. The Set Password for caunint warning

message appears.
Figure 14-13 Set Password for caunint Warning Message

c.

Click Proceed. The Set Password for caunint window appears.
Figure 14-14 Set Password for caunint Window.

d. Enter a new password in the New password field. e.

Enter the new password again in the Confirm password field, and click OK.
Chapter 14: NMS Troubleshooting Flow 430

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Server-Side Troubleshooting

4. Update the password for the CA-Unicenter Service: a.

In the Service window, right-click CA-Unicenter service. Click the Log On tab.

b. Select Properties. c.

d. Update the password. 5. When the CA-Unicenter service starts up, it starts up a number of other services: a.

Open the command prompt, and type unifstat.
Figure 14-15 CA-Unicenter Sub-Services

b. Check the number of relevant sub-services. c.

For each sub-service, check that current status is 500 - ACTIVE. services, as previously seen or by typing:
unicntrl stop all unicntrl start all

d. If any of the sub-services are not active, restart the CA-Unicenter service using Windows

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

431

Server-Side Troubleshooting

Checking the Unicenter TND Agent Technology Service
The main service for all agent technology related services (DSM, 2-D Map). This service starts up several sub-services (AWServices).
To check the Unicenter TND Agent Technology service: 1. Check that the Unicenter TND Agent Technology service is started.
Figure 14-16 Unicenter TND Agent Technology Service

2. When the Unicenter TND Agent Technology service starts up, it starts up a number of other

AWServices. Check the status of the AWServices:
a.

Open the command prompt.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

432

Server-Side Troubleshooting

b. Type awservices status and press <Enter>.
Figure 14-17 AWServices Status

3. If any of the sub-services are not active, stop and restart the AWServices as follows: a.

In the command prompt, type awservices stop and press <Enter>.

b. Type awservices start and press <Enter>.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

433

Server-Side Troubleshooting

Checking the SNMP Trap Service
The SNMP Trap service is responsible for the polling of SNMP agents that send traps to the server. This service is an OS service (in addition to the familiar SNMP Service) and is usually only relevant on the NMS Server.
To check the SNMP Trap service:

Check that the SNMP Trap service startup type is set to Manual (any application that needs this service will start the service) and that the service is started.
Figure 14-18 SNMP Trap Service

Checking the SNMP Nice Trap Formatter Service
The SNMP Nice Trap Formatter service receives traps after basic processing, and "translates" the traps into the format in which they are displayed in the Console.
To check the SNMP Nice Trap Formatter service: 1. Check that the SNMP Nice Trap Formatter service startup type is started. 2. Check that the ConsoleMessageHandlerService has started.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

434

Server-Side Troubleshooting

Figure 14-19

SNMP Nice Trap Formatter Service

Checking the Trap Mechanism
Verify that the traps mechanism is working properly on the server side.
Figure 14-20 Trap Mechanism

Raw Trap

Basic Formatting

Formatted Based on NICE MIB File

Raw Trap:

Basic Formatting Trap:

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

435

Server-Side Troubleshooting

MIB File Translation Trap:

To check the trap mechanism: 1. To check whether traps are being sent (or not) by the agent: a.

Open the command prompt and type awm_catch POLL_EVENT TRAP: and press <Enter>.
Figure 14-21 Checking the Trap Mechanism

b. Either wait a few moments, or cause a malfunction in the system to initiate the sending of a trap. To cancel the capture operation, press <Ctrl> + C. c.

If a trap is sent from the SNMP agent it is displayed (this is the raw and unformatted trap before it is captured by the DSM). If nothing is displayed, no trap was received on the server-side. DSM is not handling the trap, or that it has not been formatted properly. This may be due to a NICE Trap Formatting service problem (See Checking the SNMP Nice Trap Formatter Service on page 434), or due to a problem with the MIB file resulting in the raw event not being processed.

d. If the raw trap is displayed, but the trap is not visible in the Console, it means that the

2. To check whether the traps are being formatted by the DSM (indicated by the NICE_TRAP

prefix), enable the NICE_TRAP filter in the EM Console.
a.

In the EM Console, select Config > Settings.
Disable all the filters.

b. Click the Filters tab. c.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

436

Server-Side Troubleshooting

Figure 14-22

EM Console Configuration (CA-Server)

d. Click OK. e.

Check if the traps arrive in pairs, once with the NICE_TRAP prefix, and once with a TrapId.
Figure 14-23 Paired Trap IDs

f.

If the NICE_TRAP trap arrives without a matching TrapId trap, check the SNMP Nice Trap Formatter service (see Checking the SNMP Nice Trap Formatter Service on page 434).

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

437

Server-Side Troubleshooting

3. To view the OID nodes (all NICE objects start with: 1.3.6.1.4.1.3167) and verify they exist on

the client, use the following commands:
awwalk -h [computer name/IP] -o OID(node) awget - h [computer name/IP] -o OID(leaf)
Figure 14-24 Viewing the OID Nodes

Interactions Center - 1.3.6.1.4.1.3167.1.2 Logger - 1.3.6.1.4.1.3167.1.3 Integration - 1.3.6.1.4.1.3167.1.4 Applications Server - 1.3.6.1.4.1.3167.1.5 Storage Center - 1.3.6.1.4.1.3167.1.6 Accusys Raid - 1.3.6.1.4.1.3167.1.11 Drivers - 1.3.6.1.4.1.3167.1.12 VRG - 1.3.6.1.4.1.3167.1.13 Accusys Raid 2 - 1.3.6.1.4.1.3167.1.14 4. Run the SNMPUtil tool. This tool listens to any incoming traps on the NMS Server and

indicates if there is a problem receiving the traps, or if there is a problem with formatting and translating the traps after they are received.
a.

Open the command prompt.

b. Change to the appropriate directory (Nice Systems\Applications\Tools\SNMP Tools\ on the Applications Server or <Installation Drive>:\TND\Tools on the

NMS Server).

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

438

Server-Side Troubleshooting

c.

Type Snmputil.exe trap and press <Enter>.
Figure 14-25 SNMPUtil Tool Command Prompt

Checking the MIB File
When new traps are added to existing components, the MIB file may also be updated. The MIB file is maintained by NICE and is updated with every new NMS SP.
To check the MIB file: 1. Check that the NICE-MIB-II.mib and NICE-MIB-II.txt files are in the following folders: <Installation Drive>:\TND\SCHEMA\INCLUDED. <Installation Drive>:\TND\SERVICES\CONFIG\MIBS 2. Check that the MEGARAID.txt file is in: <Installation Drive>:\TND\SERVICES\CONFIG\MIBS NOTE: The MIB file should NOT be updated onsite as this may cause the loss of traps onsite.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

439

Server-Side Troubleshooting

Checking Polling Timeout Errors
Critical timeout errors may cause problems for the SNMP agents.
To check polling timeout errors: 1. In the Unicenter - Node View, right-click the SNMP agent and select Event Browser.
Figure 14-26 Event Browser

2. Extend the timeout parameter: a.

Right-click the <Installation Drive>:\TND\services\CONFIG\AWS_NSM\SYS\NiceEmDef.atph file. Open the file with Notepad. Save the file and change the properties back to Read-only.
Figure 14-27 Increasing the pollTimeout Value

b. Select Properties. Ensure that Read-only is not selected. c.

d. Increase the pollTimeout value. e.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

440

Server-Side Troubleshooting

3. Reset the services: a.

In the command prompt, type resetdsm skipwv and press <Enter>. All managed nodes will be dropped, and then rediscovered with the new values. The time taken depends on the number of managed nodes and the new timeout parameter is global for all managed nodes.

b. After the command has finished, in the command prompt type awservices start and press <Enter>. c.

To verify that are the services have been started and are running, in the command prompt type awservices status and press <Enter>.
Figure 14-28 Resetting the Services

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

441

Server-Side Troubleshooting

Troubleshooting Scenarios
Scenario 1: Monitored Server Does Not Appear in the 2D Map
1. In the User Administrator, verify that you are working in Technician mode: From the Settings menu, select Technician Mode.
Figure 14-29 Technician Mode

2. In the Organization tree, select Network Management System . 3. Select the required NMS Server and click the Monitored Servers tab. 4. Verify that the monitored server appears in the Attached Servers list. 5. On the NMS Server, restart the Nice CAClient service. 6. Check the NiceApplications.CACLient.Host.exe.log file. 7. Check the server operating system. See Checking for a CAClient Serialization Issue

on page 445.
8. Check for Updates.

Scenario 2: Drilldown Fails in the 2D Map
1. Ping the monitored server. 2. Check the SNMP Service Configuration on the client (see Checking the SNMP Service

Configuration on page 421).
3. Run the AWSERVICES STATUS command. See Checking the Unicenter TND Agent

Technology Service on page 432 or Using the NMSUtil Tool on page 446.
4. Use the AWWALK command to query the SNMP Agent. See Using the NMSUtil Tool

on page 446 or Checking the Trap Mechanism on page 435.

Scenario 3: Only Some Agents are Reporting
1. Check the SNMP Service Configuration on the client. See Checking the SNMP Service

Configuration on page 421.
2. Use the AWWALK command to query the SNMP Agent. See Checking the Trap

Mechanism on page 435.
3. Use the SnmpUtil.exe Tool and simulate a trap by restarting any of the monitored services. 4. Use the NMSUtil Tool.

Scenario 4: There are No Traps in the Console
1. Check that the CA Unicenter service is up and running. See Checking the CA-Unicenter

Service and Sub-Services on page 428.
Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 442

Server-Side Troubleshooting

2. Check that the ConsoleMessageHandlerService service is up. 3. Disable the NICE_TRAP filter in the EM Console to check for SNMP Nice Trap Formatter

problems.
4. Check that the SNMP Nice Trap Formatter service is up. See Checking the SNMP Nice Trap

Formatter Service on page 434.
5. Use the SnmpUtil.exe Tool.

Scenario 5: No Email Notifications Are Received
1. Check that the CA Nice SMTP Service is up. 2. Double-click <Installation Drive>\TND\Bin\NiceEmailMgr.exe to configure the email

address. If there is more than one email address per field, separate them with semicolons.
Figure 14-30 Nice Emails Manager Window

3. Check the details in <Installation Drive>\TND\Bin\NiceUsers.txt. 4. Log in to the System Administrator on the NMS Server (not from another computer), and

check that the email is working:
a.

Verify that you are working in Technician mode: From the Settings menu, select Technician Mode.
Figure 14-31 Technician Mode

b. In System Administrator, in the Organization tree, under Network Management System,

select the relevant NMS Server.
Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 443

Server-Side Troubleshooting

c.

Click the Logging tab. the email can be sent from the NMS Server.
Figure 14-32 Log Mail

d. Verify the Smtp Server and the e-mail address, and click Check Email. This checks that

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

444

Changing the System Administrator Server Name

Changing the System Administrator Server Name
The NMS communicates with the Applications Server where the System Administrator resides. If the host name changes for the Applications Server where the System Administrator resides, this information must be updated via a configuration file.
To change the host name of the server where the System Administrator resides: 1. On the NMS Server, open Windows Explorer. 2. Navigate to <Installation Drive>\TND\BIN. 3. Open the file NiceApplications.CAClient.Host.exe.config. 4. In the following line, change the server name to the new name.

5. Save the file.

Checking for a CAClient Serialization Issue
If all the servers are defined in the System Administrator, but one or more of them do not appear in the 2D map, there may be a problem with the server operating systems. The CAClient requires operating system parity, that is, all the servers must be on the same operating system and service pack. This occurs when the servers have been updated to different SPs, for example, the Applications Server is Windows 2003 SP2 and the NMS Server is Windows 2003 SP1.
To check for a CAClient serialization issue:

Ensure that all the servers are running the same operating system and SP.

Other Troubleshooting Issues

If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. DO NOT use text wrap as this will corrupt your configuration file.

IMPORTANT Any change of configuration files should be done ONLY in consultation with NICE Support.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

445

Using the NMSUtil Tool

Using the NMSUtil Tool
NMSUtil is a tool which automates the troubleshooting of NMS. This tool is available on the Tier 4 Team Site.
To use the NMSUtil tool: 1. Copy the following 5 files to a directory on the server where the NMS is installed.

• • • • •

LongNameServiceStart.bat LongNameServiceStop.bat NMSUtil.exe run.bat Snmputil.exe
Using the NMSUtil Tool

2. To start the utility, double-click NMSUtil.exe.
Figure 14-33

3. The tool includes the following options:

Table 14-1: Tool Options # 1. Option Run Snmp Util Description After selecting this option another window opens with the SNMP Utility. If you see activity on the window, the NMS server is receiving SNMP messages. When To Use Check if the NMS server is receiving any traps from the monitored servers.

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

446

Using the NMSUtil Tool

Table 14-1: Tool Options (Continued) 2. Check awservices status Checks if the NMS services are up and running. If the services are not running, select option 3 to restart the services. Starts the NMS services. If none of the SNMP Agents or monitored servers are able to send traps, check the status of the Unicenter TND Agent Technology Service. Stop and start the services when the Unicenter TND Agent Technology Service sub-services are not active.

3.

Start NMS services

4. 5.

Stop NMS services Run awwalk

Stops the NMS services. Checks if the SNMP agents on a monitored server are running or not. Enter either the host name or IP address. Allows you to ping the monitored server and check communication between the NMS and monitored servers. Enter either the host name or IP address. Resets the NMS engine. To check the trap mechanism and view the OID nodes.

6.

Ping Monitored Server

Check when specific agents or monitored servers do not send traps.

7.

Reset DSM Engine

When you have critical timeout errors and you modify the pollTimeout and need to restart the services. This should be done ONLY in consultation with NICE Support!!!!

8.

Clean Sadmin

Cleans the admin settings for NMS. Closes the tool.

9.

Exit

Chapter 14: NMS Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

447

Blank page for double-sided printing.

15
Telephony Services Server Troubleshooting Flow
Contents
Troubleshooting Workflow ..........................................................................................450 Gathering Information ..................................................................................................451 Checking the Telephony Services Server Configuration .........................................452 Checking the Dialogic Service ....................................................................................453 Troubleshooting the Dialogic Interface ......................................................................454 Troubleshooting using the NICE Playback Telephony Layer Tester .......................474 Checking the Telephony Services Server Service ....................................................478 Checking the Applications Server Service ................................................................478 Checking the Logs .......................................................................................................479 Checking Ports .............................................................................................................482 Shutting Down and Starting the Telephony Services Server ...................................485 Troubleshooting Dial Plans .........................................................................................486 Other Troubleshooting Issues.....................................................................................486

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

449

Troubleshooting Workflow

Troubleshooting Workflow
If you encountered any problems with your Telephony Services Server, following the steps described in this workflow will enable you to isolate and properly identify the problem.
NOTE: To troubleshoot Customer Feedback issues, see the Customer Feedback Guide.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

450

Gathering Information

Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

451

Checking the Telephony Services Server Configuration

Checking the Telephony Services Server Configuration
Verify that the Telephony Services Server is correctly defined in the System Administrator.
To check the Telephony Services Server configuration: 1. Log in to the NICE Perform Application Suite. From the Accessories menu, select System Administrator. 2. From the Settings menu, select Technician Mode. 3. In the Organization tree, expand a site and select Playback > Telephony Services.
Figure 15-1

4. Select your Telephony Services Server and verify that it is correctly defined.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

452

Checking the Dialogic Service

Checking the Dialogic Service
IMPORTANT This procedure is not relevant to the Diva analog interface.

To check the Dialogic service: 1. From the Start menu, select Programs > Dialogic Systems > Configuration Manager DCM.

The Dialogic Product Configuration Manager appears.
Figure 15-2 Dialogic Configuration Manager

2. Ensure that the Dialogic service is running.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

453

Troubleshooting the Dialogic Interface

Troubleshooting the Dialogic Interface
This section includes the following topics:

• • • • • •

Troubleshooting the Dialogic Analog Interface (below). Troubleshooting the Dialogic Digital Interface. See page 457. Troubleshooting the Dialogic IP Interface. See page 460. Using the ISDIAG Utility. See page 461. Testing the Extension Using the Dialogic Voice Demo. See page 464. Using the Universal Dialogic Diagnostics Utility. See page 470.

Troubleshooting the Dialogic Analog Interface
IMPORTANT This procedure is not relevant to the Diva analog interface.

To troubleshoot the Dialogic analog interface: 1. Check that the Dialogic service is running. See Checking the Dialogic Service (above). 2. Ensure that the installed Dialogic software is the latest released. In the Dialogic Configuration Manager, from the Help menu, select About.

Check that the version is System Release 6.0 Build 171.
Figure 15-3 Dialogic Configuration Manager - About System Release 6.0

3. Check that the extensions are connected to the Dialogic board on the correct ports.
Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 454

Troubleshooting the Dialogic Interface

4. Dialogic JCT PCIExpress board only : There may be a Power Budgeting issue.

Certain PCI Express boards include a built-in power management feature. If the installed Dialogic SR 6 software does not support the power management PCI Express board feature, the board may not start correctly, or may start but will fail to recognize the star (*) key, and errors will appear. Upgrading the software will not remedy this problem.
Solution

Power Budgeting Jumper P3 is comprised of a three-pin jumper which determines the way in which the board responds to the system power budgeting feature. The Power Budgeting Jumper P3 can be configured in one of the two following ways:

• •

P3 Jumper in Pins 2-3 (default): The Dialogic SR 6 board adheres to power

budgeting values set by the system.
P3 Jumper in Pins 1-2:

The Dialogic SR 6 board ignores the power budgeting values set by the system.

Figure 15-4

Dialogic Board - Physical Layout

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

455

Troubleshooting the Dialogic Interface

Check the Event Viewer.
Figure 15-5 Event Properties Window - Error Message Example

If Power Budgeting is not implemented: The Dialogic JCT PCIExpress board must be plugged into a x4 or higher slot with the P3 jumper in position 1-2 (power budgeting ignored). If Power Budgeting is implemented: A Dialogic JCT PCIExpress board can be plugged into a x1 slot but the P3 jumper must be in position pins 2-3 (power budgeting adhered to). TIP:

When using the HP DL380 G5 Dialogic board, set the jumper on pins 1 and 2, causing power budgeting to be ignored. When this jumper is set on the default pins (2,3) the board may not have power, preventing the board from starting. Ensure that the latest software is installed.

For more information about this issue, see P3 Jumper Settings for Dialogic® PCI Express Boards.
5. For each Dialogic board, check that the SW100 switch is set to the required ID number. 6. If more than one Dialogic board is installed, check the H100 cable connections. The first

connector on the H100 cable must be connected to board ID 0, and the last connector on the H100 cable must be connected to the last Dialogic board.
7. Check that the Dialogic board is placed correctly into the slot. 8. Move the Dialogic board to a different slot. 9. Use the ISDIAG Utility to check the E1 or T1 trunks. See Using the ISDIAG Utility

on page 461.
10. Check that the Dialogic boards have been configured correctly according to the type of board.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

456

Troubleshooting the Dialogic Interface

11. Use the Dialogic Voice Demo to test the channel, check that you can record and playback

WAV and VOX files, and test that you can get digits from the extension. See Testing the Extension Using the Dialogic Voice Demo on page 464.
12. Test the lines from the patch panel with an analog phone and from the Y-Cable with a

telephone butt set.
13. Try to configure and test one port at a time. 14. Verify the pinouts. 15. Check the Dialogic board using the Universal Dialogic Diagnostics Utility. See Using the

Universal Dialogic Diagnostics Utility on page 470.

Troubleshooting the Dialogic Digital Interface
To troubleshoot the Dialogic digital interface: 1. Check that the Dialogic service is running. See Checking the Dialogic Service on page 453. 2. Ensure that the installed Dialogic software is the latest released. In the Dialogic Configuration Manager, from the Help menu, select About. Digital HMP Interface: Check that the version is HMP Release 3.0 Service Update 214. Digital SR 6.0 Interface: Check that the version is System Release 6.0 Build 171.
Figure 15-6 Dialogic Configuration Manager - About System Release 6.0

3. Check the cables that are connected to the Dialogic boards. 4. Dialogic JCT PCIExpress board only : There may be a Power Budgeting issue.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

457

Troubleshooting the Dialogic Interface

Certain PCI Express boards include a built-in power management feature. If the installed Dialogic SR 6 software does not support the power management PCI Express board feature, the board may not start correctly, or may start but will fail to recognize the star (*) key, and errors will appear. Upgrading the software will not remedy this problem.
Solution

Power Budgeting Jumper P3 is comprised of a three-pin jumper which determines the way in which the board responds to the system power budgeting feature. The Power Budgeting Jumper P3 can be configured in one of the two following ways:

• •

P3 Jumper in Pins 2-3 (default): The Dialogic SR 6 board adheres to power

budgeting values set by the system.
P3 Jumper in Pins 1-2:

The Dialogic SR 6 board ignores the power budgeting values set by the system.

Figure 15-7

Dialogic Board - Physical Layout

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

458

Troubleshooting the Dialogic Interface

Check the Event Viewer.
Figure 15-8 Event Properties Window - Error Message Example

If Power Budgeting is not implemented: The Dialogic JCT PCIExpress board must be plugged into a x4 or higher slot with the P3 jumper in position 1-2 (power budgeting ignored). If Power Budgeting is implemented: A Dialogic JCT PCIExpress board can be plugged into a x1 slot but the P3 jumper must be in position pins 2-3 (power budgeting adhered to). TIP:

When using the HP DL380 G5 Dialogic board, set the jumper on pins 1 and 2, causing power budgeting to be ignored. When this jumper is set on the default pins (2,3) the board may not have power, preventing the board from starting. Ensure that the latest software is installed.

For more information about this issue, see P3 Jumper Settings for Dialogic® PCI Express Boards.
5. For each Dialogic board, check that the SW100 switch is set to the required ID number. 6. If more than one Dialogic board is installed, check the H100 cable connections. The first

connector on the H100 cable must be connected to board ID 0, and the last connector on the H100 cable must be connected to the last Dialogic board.
7. Check that the Dialogic board is placed correctly into the slot. 8. Move the Dialogic board to a different slot. 9. Use the ISDIAG Utility to check the E1 or T1 trunks. See Using the ISDIAG Utility

on page 461.
10. Check that the Dialogic boards have been configured correctly according to the type of board,

that is, Digital T1 or Digital E1.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

459

Troubleshooting the Dialogic Interface

11. For Digital E1 or T1 ISDN PRI trunks, check the switch configuration. The configuration

varies according to the type of switch integration.
12. Test the lines from the patch panel with an analog phone and from the Y-Cable with a

telephone butt set.
13. Try to configure and test one port at a time. 14. Verify the pinouts. 15. Check the Dialogic board using the Universal Dialogic Diagnostics Utility. See Using the

Universal Dialogic Diagnostics Utility on page 470.

Troubleshooting the Dialogic IP Interface
To troubleshoot the Dialogic board for the IP interface: 1. Check that the Dialogic service is running. See Checking the Dialogic Service (above). 2. Ensure that the installed Dialogic software is the latest released. In the Dialogic Configuration Manager, from the Help menu, select About.

Check that the version is HMP Release 3.0 Service Update 214.
Figure 15-9 Dialogic Configuration Manager - About System Release 6.0

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

460

Troubleshooting the Dialogic Interface

Using the ISDIAG Utility
IMPORTANT This procedure troubleshoots Dialogic digital trunk boards only.

This procedure confirms that the Dialogic board and the trunk are properly connected. In this procedure, you enter a telephone extension from your site. The program dials the telephone extension and the telephone rings. Before you begin, you will need the following:

• • •

The location of the isdiag.exe file. A telephone extension connected to your site. The trunk connected from the switch to the Logger.

To test the Dialogic SR 6 board using the ISDIAG utility: 1. Verify that a green LED is lit on the board panel. 2. Open a command window and type isdiag.exe. 3. On the command line, type the following:

• •

Digital T1 environment: Type isdiag 1 1 t s. Digital E1 environment: Type isdiag 1 1 e s.
Cmd.exe Window
Digital E1 environment: isdiag 1 1 e s

Figure 15-10

Digital T1 environment: isdiag 1 1 t s

The Main Menu appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

461

Troubleshooting the Dialogic Interface

Figure 15-11

Cmd.exe - isdiag 1 1 e s Window

Verify that the value for LOS is 0

4. Verify that all values are set to zero (0). WARNING If any of the values are not set to zero, check that both the Dialogic board and the switch are configured correctly.

• •
a.

Ericsson only: Proceed to Step 5. All switches except for Ericsson: Proceed to Step 6.

5. Ericsson only:

At the prompt, type 1 (set call parameters). Set 03 to A-law.
Cmd.exe - isdiag 1 1 e s Window

b. Type 9 (change user info layer1 protocol). c.

Figure 15-12

6. Return to the main menu and at the prompt, type 6.

The Call prompt appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

462

Troubleshooting the Dialogic Interface

Figure 15-13

Call Prompt Example

Extension number

7. Type an extension number and press <Enter>.

The entered telephone extension rings.
8. Close the command window.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

463

Troubleshooting the Dialogic Interface

Testing the Extension Using the Dialogic Voice Demo
The Dialogic Voice Demo confirms system health between Dialogic and the telephone system installed at your site. Use the Dialogic Voice Demo to test the following:

• • • • •

Connectivity between Dialogic board and the extension Recording and playing back WAV and VOX files Getting digits from the extension. Testing an Analog Extension Using the Dialogic Voice Demo Testing a Digital Extension Using the Dialogic Voice Demo

This section describes:

Testing an Analog Extension Using the Dialogic Voice Demo
To test an analog extension using the Dialogic Voice Demo: 1. When you install the Dialogic software, ensure that Demos is selected. 2. After the installation is complete, in the Dialogic Configuration Manager, check that the

Dialogic services are running and that the board is functioning properly.
Figure 15-14 Dialogic Configuration Manager

3. Start the VoiceDemo application by selecting Start > Programs > Dialogic Service > Demos > Voice > Voice Demo.

The Dialogic Voice Demo Application appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

464

Troubleshooting the Dialogic Interface

Figure 15-15

Dialogic VoiceDemo Application

4. To select an extension to use for your test: a.

Select File > Open. The Select (Voice) Channel window appears.
Figure 15-16 Select (Voice) Channel Window

b. Select a channel (extension) and click OK.

The Device opened message appears.
Figure 15-17 Device Opened Message

5. To test the extension: a.

Select Function > Dial. The Dial window appears.
Chapter 15: Telephony Services Server Troubleshooting Flow 465

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Troubleshooting the Dialogic Interface

Figure 15-18

Dial Window

b. Enter the extension number you need to test. Click OK. c.

Ensure that the extension rings.

6. To close the call, select Function > On Hook. 7. To test recording and playing a WAV file: a.

Record to a WAV file, by selecting Function > Record WAV. Talk into your phone handset. A WAV file is automatically created. The WAV file is played back to your extension.
Figure 15-19 Playing WAV File Message

b. To play back the WAV file, select Function > Play WAV.

8. To test recording and playing a VOX file: a.

Record to a VOX file, by selecting Function > Record VOX. Talk into your phone handset. A VOX file is automatically created. The VOX file is played back to your extension.

b. To play back the VOX file, select Function > Play VOX.

9. To verify that the phone digits are received correctly (IVR): a.

Select Function > Getdigits. A message appears informing you that the digits were received.
NOTE: To verify phone digits, the extension must be off-hook.

b. Enter 5 digits.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

466

Troubleshooting the Dialogic Interface

Figure 15-20

Received Digits Message

Testing a Digital Extension Using the Dialogic Voice Demo
To test a digital extension using the Dialogic Voice Demo: 1. When you install the Dialogic software, ensure that Demos is selected. 2. After the installation is complete, in the Dialogic Configuration Manager, check that the

Dialogic services are running and that the board is functioning properly.
Figure 15-21 Dialogic Configuration Manager

3. Start the VoiceDemo application by selecting Start > Programs > Dialogic Service > Demos > Voice > Voice Demo.

The Dialogic Voice Demo Application appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

467

Troubleshooting the Dialogic Interface

Figure 15-22

Dialogic VoiceDemo Application

4. To select an extension to use for your test: a.

Select File > Open. The Select (Voice) Channel window appears.
Figure 15-23 Select (Voice) Channel Window

b. Select a channel (extension) and click OK.

The Select (Digital) Channel window appears.
Figure 15-24 Select (Digital) Channel Window

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

468

Troubleshooting the Dialogic Interface

c.

Select a channel and click OK. The Device opened message appears.
Figure 15-25 Device Opened Message

5. To test the extension: a.

Select Function > Make Call. The Dial window appears.
Figure 15-26 Dial Window

b. Enter the extension number you need to test. Click OK. c.

Ensure that the extension rings.

6. To close the call, select Function > Release Call. 7. To test recording and playing a WAV file: a.

Record to a WAV file, by selecting Function > Record WAV. Talk into your phone handset. A WAV file is automatically created. The WAV file is played back to your extension.
Figure 15-27 Playing WAV File Message

b. To play back the WAV file, select Function > Play WAV.

8. To test recording and playing a VOX file: a.

Record to a VOX file, by selecting Function > Record VOX. Talk into your phone handset. A VOX file is automatically created. The VOX file is played back to your extension.

b. To play back the VOX file, select Function > Play VOX.

9. To verify that the phone digits are received correctly (IVR): a.

Select Function > Getdigits.

b. Enter 5 digits.
Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 469

Troubleshooting the Dialogic Interface

A message appears informing you that the digits were received.
NOTE: To verify phone digits, the extension must be off-hook.
Figure 15-28 Received Digits Message

Using the Universal Dialogic Diagnostics Utility
You use the Universal Dialogic Diagnostics Utility to check the Dialogic board.
To use the Universal Dialogic Diagnostics Utility: 1. Stop all active applications on all Dialogic boards. 2. From the Start menu, select Programs > Intel Dialogic System Software > Universal Dialogic Diagnostics Utility.

The UDD Initialization window appears.
Figure 15-29 UDD Initialization

3. Click Continue.

A warning message appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

470

Troubleshooting the Dialogic Interface

Figure 15-30

Warning Message

4. Click Yes.

The Universal Intel Dialogic Diagnostics window appears.
Figure 15-31 Universal Intel Dialogic Diagnostics Window

5. Select the board you need to test and click Specify Tests.

The Specify Tests window appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

471

Troubleshooting the Dialogic Interface

Figure 15-32

Universal Intel Dialogic Diagnostics - Specify Tests

6. Click Select All Tests and then click OK.

The Test Progress window appears.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

472

Troubleshooting the Dialogic Interface

Figure 15-33

Universal Intel Dialogic Diagnostics - Test Progress Window

7. If any failure occurs, a new screen appears displaying information about the failed board. To view the details about the failed board, click Details. 8. Click Done.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

473

Troubleshooting using the NICE Playback Telephony Layer Tester

Troubleshooting using the NICE Playback Telephony Layer Tester
The NICE Playback Telephony Layer Tester tests the connection between the Telephony Services Server and your phone lines. The NICE Playback Telephony Layer Tester can be used for all line types except for SIP extension.
NOTE: The NICE Playback Telephony Layer Tester is automatically installed with the Telephony Services Server.

To troubleshoot line connections using the NICE Playback Telephony Layer Tester: 1. Navigate to ...\NICE Telephony Services\bin and double-click NiceApplications.Playback.TelephonyServices.TelLayer.GuiTester.exe.

The NICE Playback Telephony Layer Tester appears.
Figure 15-34 NICE Playback Telephony Layer Tester Main Window

2. Click Detect Resources.

The NICE Playback Telephony Layer Tester displays the detected resources.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

474

Troubleshooting using the NICE Playback Telephony Layer Tester

Figure 15-35

NICE Playback Telephony Layer Tester - Detected Resources

Detected resources

3. IP lines only: If you need to test IP lines, in the Telephony Configuration and IP

Configuration areas, enter the relevant information.
4. Click Init Telephony.

The available lines are displayed.
Figure 15-36 NICE Playback Telephony Layer Tester - Available Lines

Available lines

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

475

Troubleshooting using the NICE Playback Telephony Layer Tester

5. Select one or more lines or click Create All Lines. The selected lines become available for

testing.
NOTE: When testing IP lines, to confirm connectivity, you only need to test one line. When testing analog lines, we recommend testing all available lines. 6. Click the Lines tab and select a line.
Figure 15-37 NICE Playback Telephony Layer Tester - Lines Tab

7. Enter a phone number and click Make Call.

The phone will ring.
8. Pick up the phone receiver.

• •

If you need to test voice, proceed to Step 9. If you do not need to perform further tests on the line, close the call. A message appears indicating that the call started and ended successfully.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

476

Troubleshooting using the NICE Playback Telephony Layer Tester

Figure 15-38

NICE Playback Telephony Layer Tester - Information Messages

9. To play a recorded call, in the Play area, click Browse and select a WAV file to play. Then click Play.

The selected file plays. The NICE Playback Telephony Layer Tester displays a message indicating that the file played successfully.
Figure 15-39 NICE Playback Telephony Layer Tester - File Played Successfully

10. Click End Call.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

477

Checking the Telephony Services Server Service

Checking the Telephony Services Server Service
After the Telephony Services Server has been configured in the System Administrator and the Telephony Services Server software is completely installed, you can verify that the NICE Telephony service is operating properly.
NOTE: If the Telephony Services Server is installed but System Administrator is not yet configured, the Telephony Services Server service may fail to start.

To check the NICE Telephony Server service:

Open the Services window and verify that the service Status is Started and its Startup Type is set to Automatic.
Figure 15-40 Services Window

Checking the Applications Server Service
Check that following service is running on the Applications Server.

NICE System Administrator: This is the main Applications Server service that controls the login process to NICE applications.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

478

Checking the Logs

Checking the Logs
This section describes:

• • • • •

Setting the Logs Reporting Level (below). Viewing the Telephony Services Server Logs. See page 480. Viewing the NICE Playback Streaming Services Logs. See page 481. Viewing the NICE Playback Administration Service Logs. See page 481. Viewing the NICE Player Logs. See page 481.

For information about the Log Collector, see the Troubleshooting Guide, Collecting Information Using the Log Collector.

Setting the Logs Reporting Level
Set the log level to DEBUG.
To set the logs reporting level: 1. Log in to the NICE Perform Application Suite. From the Accessories menu, select System Administrator. 2. From the Settings menu, select Technician Mode.
Figure 15-41 Technician Mode

3. In the Organization tree, expand your site and select Playback > Telephony Services.
Figure 15-42

4. Select a Telephony Services Server, select the Log Level tab, and from the Log Level dropdown, select Debug.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

479

Checking the Logs

Figure 15-43

Log Level - Debug

5. Click Save

.

Viewing the Telephony Services Server Logs
The Telephony Services Server log files are located in the following folder:
...\Program files\NICE Systems\NICE Telephony Services\log To view the Telephony Services Server Logs:

Check the TelephonyServicesLog.txt log files for the Telephony Services Server service.
Figure 15-44 Telephony Services Server Log Files

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

480

Checking the Logs

Viewing the NICE Playback Streaming Services Logs
The NICE Playback Streaming Services run on the Telephony Services Server and receive streamed packets from the Stream Server and synchronize them for playback to the Player. The log files are located in the following location:
...\Program files\NICE Systems\NICE Telephony Services\log To view the NICE Playback Streaming Services Logs:

Check the MediaServices.txt log files.

Viewing the NICE Playback Administration Service Logs
The NICE Playback Administration service runs on the Applications Server and manages the state of each Playback to Extension resource, port and so on. If no resources are available, the NICE Playback Administration service displays an error. The log files are located in the following location:
...\NICE Systems\Applications\ServerBin To view the NICE Playback Administration Service Logs:

Check the PlaybackAdministration.txt log files.

Viewing the NICE Player Logs
The log files are located on the Client Workstation in the following location:
C:\Documents and Settings\[user]\Local Settings\Application Data\Release3\NicePlayer. To view the NICE Player Logs:

On the Client Workstation, check the log files for the NICE Player.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

481

Checking Ports

Checking Ports
Check the required ports are open.
To check connectivity: 1. Log in to the NICE Perform Application Suite. From the Accessories menu, select System Administrator. 2. From the Settings menu, select Technician Mode. 3. In the Organization tree, expand your site and select Playback > Telephony Services. 4. Select a Telephony Services Server, select the General tab, and check the HTTP and TCP

ports:

Troubleshooting Playback to Extension Problems
To troubleshoot Playback to Extension Problems:

No free resources error message:

• • • •

Restart the NICE Playback Administration service on the Applications Server. Check the port configuration in System Administrator. See Checking Ports on page 482.

The phone is not ringing (a phone is not responding and an error message appears): Search for a dial command in the log. See Viewing the Telephony Services Server Logs on page 480 or see the Troubleshooting Guide: Collecting Information Using the Log Collector. Check that the extension is connected to the Dialogic board on the correct port. See Troubleshooting the Dialogic Interface on page 454.

• •

Pressed star (*) but nothing happens or an error message appears:

• •

Check for a network problem, where the server to client event connectivity is blocked. See Checking Ports Using Port Tester on page 483. Check that the NICE Playback Streaming service is running on the Applications Server. See Checking the Telephony Services Server Service on page 478 and Checking the Applications Server Service on page 478. Check the Player log for errors. See Troubleshooting Guide: Collecting Information Using the Log Collector.

• •

Server down message but the Telephony Services Server service is running:

Check for a network problem.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

482

Checking Ports

Checking Ports Using Port Tester
Use the Port Tester to check the ports between the Player and the Telephony Services Server.
To check ports using port tester: 1. Log in to the NICE Perform Application Suite. From the Accessories menu, select System Administrator. 2. In the Organization tree, expand your site and select Applications > Configurations > General. Then click the General tab and check whether the system is configured to work

with random ports or specific ports range.
Figure 15-45 General Settings Tab

3. On the workstation: a.

Copy RemotingClientTester.exe and ClientRemotingPortTester.dll to the workstation. In the From field, enter the lower port range, and in the To field, enter the upper port range. This can be done in batches of 500 ports, in the range of 1024 to 5000. If the system is configured to use specific port ranges (as shown in step 1), use the relevant port ranges.

b. Double-click RemotingClientTester.exe. c.

d. Click Run.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

483

Checking Ports

Figure 15-46

Client Tester Form Window

4. On the Telephony Services Server: a.

Copy RemotingServerTester.exe and ServerRemotingPortTester.dll to the workstation. In the Client Name field, enter the client workstation name.

b. Double-click RemotingServerTester.exe. c.

d. In the From field, enter the lower port range, and in the To field, enter the upper port range. This can be done in batches of 500 ports, in the range of 1024 to 5000. If the

system is configured to use specific port ranges. If the system is configured to use specific port ranges (as shown in step 1), use the relevant port ranges.
e.

Click Run.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

484

Shutting Down and Starting the Telephony Services Server

Figure 15-47

Server Remoting Port Tester Window

5. If a number of ports fail, this indicates a communication problem between the server and

client.

Shutting Down and Starting the Telephony Services Server
To shut down the Telephony Services Server:

Open the Services window, select the NICE Telephony Services service and change its service Status to Stopped.
-or-

Open the Services window, select the NICE Telephony Services service and change its service Status to Started.
Figure 15-48 Services Window

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

485

Troubleshooting Dial Plans

Troubleshooting Dial Plans
IMPORTANT If a switch does not require a dial plan, select Empty Prefix when defining it. Do not leave dial plan definitions as Disabled!

Other Troubleshooting Issues
• • •
If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options. Do not use text wrap as this will corrupt your configuration file. In any event, any change of configuration files should be done ONLY in consultation with NICE Support!!!! Do not use a loopback IP address (such as 127.0.0.1) for the Telephony Services Server address, even if the Telephony Services Server co-resides on the Applications Server. This may prevent the Telephony Services Server from starting properly. When picking up the phone, the prompt that is heard is Please press star (*). This may be confused with Please press Start. Make sure you press * to begin playing the call. If you discover unidentified problems when running the Telephony Services Server, you may need to install the latest Update Pack. Check if an Update Pack is available. If the Playback to Extension option appears grayed out, this may be due to a problem with the resource managers. In this same scenario, Playback to Speaker works fine. See below.
Figure 15-49 Playback to Extension - Select Output Window

• • •

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

486

Other Troubleshooting Issues

If Playback to Extension appears grayed out, do the following:
a.

In System Administrator, expand the Master Site and select Playback > Telephony
Services > Dial Plan. The following message may appear.
Figure 15-50 Dial Plan Updates Window

b. Click OK and restart the NICE Playback Administration Service.

Customer Feedback: If you have correlation issues between your surveys and the agents’ interactions, resulting in interaction playback failure, this may be due to one of the following:

• •

A faulty UCID configuration. Check your switch configuration. See Samples of Avaya’s UCID Configuration on page 323. The port-extension pairs in System Administrator may have been defined incorrectly. See Installing Dialogic SR 6.0 Boards for the Analog/Digital Interface on page 28 and the Customer Feedback Setup Guide.

The extensions may not have been defined as monitored devices. See the integration guide relevant to your site.

Chapter 15: Telephony Services Server Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

487

Blank page for double-sided printing.

16
NiceScreen Troubleshooting Flow
Installation of NiceScreen components is performed using an installation wizard designed with easy-to-follow steps. This section lists some of the most common problems you may encounter and simple solutions. If you continue to need help, contact your local supplier or the nearest NICE System Customer Service Center. Contact information is located in the inside cover of this manual.
NOTE: Any references made in this chapter to values which must be changed, refers directly to the Agent workstation registry unless stated otherwise. Page numbers are the location of the description of each parameter.

Contents
Overview........................................................................................................................491 Isolating the Problem ...................................................................................................492 Gathering Information ..............................................................................................492 Checking the NiceScreen Logger Service Status ....................................................492 Checking the ScreenAgent Interactions Center Registration...................................493 Checking the Configuration......................................................................................494 Checking SAMS for Errors .......................................................................................494 Isolating the Problem Further...................................................................................496 Checking the NiceScreen Logs................................................................................496 Checking the ScreenAgent Logs..............................................................................496 Checking the RCM Logs ..........................................................................................497 Checking the Call Server Logs.................................................................................497 Checking the Call Status in the Database ................................................................497 Checking the Rule....................................................................................................497 Checking the Naming Resolution.............................................................................498 Checking Connectivity..............................................................................................498 Verifying the ScreenAgent Configuration .................................................................500 Testing the ScreenAgent ..........................................................................................500 Other Troubleshooting Issues ..................................................................................501
Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 489

Other Useful Tools....................................................................................................501 NICE ScreenAgent Installation/Operational Problems..............................................503 NICE ScreenAgent Optimization and Performance Issues.......................................505 NiceScreen Logger Configuration Parameters..........................................................507

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

490

Overview

Overview
If you experience a problem with Screen recording follow the steps below to isolate the problem:
Figure 16-1 Workflow Diagram

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

491

Isolating the Problem

Isolating the Problem
Gathering Information
Important issues to consider are:

• • • • •

Is this a new installation? If yes, when was the installation performed? Were there any changes to the environment that might have influenced the system? If yes, check these changes first. How often does the problem occur? Is the problem reproducible? If yes, what were the steps? Were the Servers restarted? If yes, did this resolve the problem?

Checking the NiceScreen Logger Service Status
When to check the NiceScreen Logger service status:

• •

If all screens are not being recorded, check whether the NiceScreen Logger service is active. If some agents screens are being recorded, you can skip this check.

To check the NiceScreen Logger service status: 1. On the computer where the NiceScreen Logger is installed, check that the NiceScreen Logger service is running. If the service is not running select Start > Settings > Control Panel > Administrative Tools > Services, and start the service. 2. Point your cursor to the
Figure 16-2

icon in the tray, a tooltip displays information about NiceScreen Logger. When you click on the icon, status details are displayed.
Logger Status Details

3. On the computer where the NiceScreen Logger is installed check that mmloggersvc.exe is

an active process running on the computer. If not, the installation may have failed.
Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 492

Isolating the Problem

Checking the ScreenAgent Interactions Center Registration
When to check the ScreenAgent Interactions Center registration:

If all or some screens are not being recorded.

To check the ScreenAgent Interactions Center registration: 1. On the Interactions Center, run Tester.exe from the D:\program files\Nice systems\Nice CLS\BIN folder.
Figure 16-3 Tester Application

2. Click Connect

. The Connect to CLS window appears.

3. Click OK. A SUCCESS message appears. 4. Select Maintenance > Universal Agent Table.
Figure 16-4 Universal Agent Table

5. Click Refresh. The ScreenAgents that were registered to the Interactions Center appear in the

list. If a ScreenAgent is not registered, it will not appear in this list and will not be recorded.
6. Perform the following checks based on the outcome:

If there are no agents logged in, there may be a network problem. Check the network connectivity and the ports. See Checking Connectivity on page 498.
Chapter 16: NiceScreen Troubleshooting Flow 493

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Isolating the Problem

If the agents that are not being recorded do not appear in the list:

• • •

Check the agent workstation where ScreenAgent is installed and screenagent.exe is running. Check that the particular workstation is configured correctly. See Checking the Configuration on page 494. Check the particular workstation for a port blockage See Checking Connectivity on page 498.

7. Check the SAMS logs. See Checking SAMS for Errors on page 494. 8. Search the ScreenAgent logs in order to identify problems with regards to registration to the

Interactions Center. See Checking the ScreenAgent Logs on page 496.

Checking the Configuration
Verify that:

• •

The NiceScreen Logger, ScreenAgents, Channel Pools and Sources Pool are correctly defined in the System Administrator. The Agent is configured correctly in User Administrator.

To check the configuration: 1. In System Administrator, verify that you are working in Technician mode: In the Organization tree, click Organization. Then select Technician Mode and click Save. 2. Expand the Master Site > Logger Servers > NiceScreen Logger and check the details for

the required NiceScreen Logger. Verify that these details correspond with any criteria in the
Rules Manager > Scheduler > Filter. 3. In the Organization tree, select CLS Definitions > Channel Mapping > 2. Sources Definition. Verify that the ScreenAgents are correctly defined. 4. In the Organization tree, select CLS Definitions > Channel Mapping > 4. Dynamic Mapping. Check the following tabs: a.

Channels Pool Attach/Detach

b. Sources Pool c.

5. Check that the Agent is correctly configured in User Administrator, including the Operating

System login name.

Checking SAMS for Errors
SAMS monitors ScreenAgents; debug information and logs that are sent from ScreenAgents are received by the SAMS. Every time a ScreenAgent activates, stops, or fails to function properly, status information is displayed in a console window called SAMS Report. The status is displayed

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

494

Isolating the Problem

according to severity. From least to most severe, severity is as follows: Debug, Info, Warn, Fatal, Bug, and Event. By default, the SAMS threshold is set to Fatal. This means that status with a severity of Fatal, Bug, or Event will be displayed in the SAMS Report console.
To check SAMS: 1. Check that the SAMS service is running. If the service is not running select Start > Settings > Control Panel > Administrative Tools > Services, and start the service. 2. Ensure that SAMS has been installed properly. 3. Once the SAMS service is running, the SAMS Server icon is displayed in the computer's

system tray

.

4. To start the SAMS Report, right-click the SAMS Server icon and select Start. 5. Messages are sent to the SAMS Report console window.
Figure 16-5 SAMS Report

All messages sent to the SAMS Report console window display the following parameters:

• • • • •

Date and time message was sent Severity level - From least to most severe: Debug, Info, Warn, Fatal, Bug, Event Name of the component that sent the message (for example, SA - for ScreenAgent) IP address and port number or host name of the machine from which the message was sent Report type:

• • • • •

Install - message report success/failure Uninstall - message report success/failure Startup - information report Shutdown - information report Online - message sent while ScreenAgent is running
Chapter 16: NiceScreen Troubleshooting Flow

495

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Isolating the Problem

Message content

6. Check the SAMSReport.log file to review the status information that is no longer displayed

in the SAMS Report console. The file is in the SAMS installation folder:
C:\Program Files\Nice Systems\SAMS Server\SAMSServerLog.

Isolating the Problem Further
Try to isolate the problem and identify the faulty component.
To isolate the problem further:

Try and isolate the problem by checking the logs to see:

• •

If the StartRecord command was called by the RCM. See Checking the RCM Logs on page 497. If the ScreenAgent received the StartRecord command. See Checking the ScreenAgent Logs on page 496. The following is a sample StartRecord in the ScreenAgent log: INFO: 02/01/08 10:56:31:015 IncCommandsHandler: StartRecording received from IC IpAddr(screen-env3:2010). capture handle(10000001), DestinationAddr(screen-env3:30100), DestinationResourceId(93); Context = (676)

If NiceScreen received the StartRecord command. See Checking the ScreenAgent Logs on page 496.

Checking the NiceScreen Logs
Settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration file located in the NiceScreen Logger installation directory. LogFile_Path indicates the location of the NiceScreen log files. The logs are located in the installation folder, by default in:
C:\Programs\Nice Systems\Multimedia Logger. To check the NiceScreen Logs:

Review the NiceScreen Logger logs (logfileXX.log).

Checking the ScreenAgent Logs
In NICE Perform Release 3 Service Pack 3 the logs are located in C:\Program Files\Nice Agent.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

496

Isolating the Problem

In NICE Perform Release 3 Service Pack 4 and above, the logs are located in different folders depending on the Operating System. For Windows 2000, Windows XP, Windows 2003 Server the logs are in C:\Documents and Settings\All Users\Application Data\Nice\Screen Agent. For Windows Vista the logs are in C:\ProgramData\Nice\Screen Agent.
To check the ScreenAgent Logs:

Review the ScreenAgent logs (LogfileXX.dat).

Checking the RCM Logs
The log files are located in the D:\Program files\NICE Systems\NICE CLS\Bin\Log.
To check the RCM Logs:

On the NICE Interactions Center, open the RCM process (Resource Manager console), or review the RCM.log.xxx.txt files.

Checking the Call Server Logs
The log files are located in the D:\Program files\NICE Systems\NICE CLS\Bin\Log.
To check the Call Server Logs:

Review the Call Server log files (Call Server.log.xxx.txt files).

Checking the Call Status in the Database
Check the status of the calls that failed to record the screen in the database.
To check the call status in the database:

Check tblExceptionXX in the nice_interaction table (query on iInteractionID). For more details, see the Interaction Data Reference Guide.

Checking the Rule
In the Rules Manager verify that there is a valid rule for recording screens. See the Rules Manager Guide for more information and the Interaction Data Reference Guide for details on the fields in the Filter Expressions.
To check the rule: 1. In the Audit Trail, check if the rule has been changed recently. See the Audit Trail Guide for

more information.
2. In the NICE Perform applications suite, open Rules Manager > Scheduler.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

497

Isolating the Problem

3. Select the relevant rule, and click 4. Click

Edit Rule.

next to Filter.
Rule Filter

Figure 16-6

5. Verify that Screen is selected, check all the Rule Filter tabs and verify that the filter does not

exclude the required screen interactions.

Checking the Naming Resolution
Naming resolution is the ability to resolve between the computer name and IP. NICE Perform uses the computer name.
To check the naming resolution:

Ensure that the DNS is configured on the workstation and the Interaction Server.

Checking Connectivity
Perform connectivity checks.
To check connectivity: 1. Check that port 30100 is open for incoming communication on the NiceScreen Logger server. 2. Check that port 2010 is open for incoming communication on the NICE Interactions Center

server.
3. Check that port 2001 is open for incoming communication on the NICE SAMS server. 4. Verify the NAT/PAT connectivity and configuration:

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

498

Isolating the Problem

a.

The services must be accessed by fully qualified machine names; a DNS service should be available to allow names resolution. configuration file for each service.

b. For a NAT/PAT environment, verify that useMachineName is set to True in the

IMPORTANT Any change of configuration files should be performed ONLY in consultation with NICE Support! 5. The ScreenAgent process uses arbitrary ports to communicate with the ScreenLogger,

Interactions Center and SAMS Server. Verify that outgoing communication is not blocked from the ScreenAgent workstations.
6. Check that the following ports are open. Port List

Port List (cont.)

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

499

Isolating the Problem

Verifying the ScreenAgent Configuration
The ScreenAgent must be configured for the correct Interaction Server. This is set during the installation and can be verified in the ScreenAgent registry.
To verify the ScreenAgent configuration: 1. In the workstation registry, open the registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Nice Systems\Nice Screen\ClsList 2. Check the ClsAddr1 … ClsAddr5 parameters. These are the network addresses of the

associated NICE Interactions Centers.

Testing the ScreenAgent
Start a call and verify monitoring the screens.
To test the ScreenAgent: 1. Start a call on an agent workstation. 2. In the NICE Perform applications suite, open Monitor. 3. Click the Settings

button. In the Settings window, select Screen as your Recording .

Media. Click Save

4. In the Folders pane, select the group that includes the agent whose activity you want to monitor. The agent names in the selected group are displayed in the Details pane. 5. Select an agent who is logged in and on the phone. (Indicated by this icon 6. Click Monitor

).

. The Screen icon

should appear in the Monitor column.

7. NICE Player opens and plays the agent's screen. If there is a recording problem, the relevant

error will appear. Use this reason to resolve the recording failure.
Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 500

Isolating the Problem

Other Troubleshooting Issues

If you edit the configuration files, ensure that you use a plain text editor such as Notepad without any options.

IMPORTANT

• • • •

DO NOT use text wrap as this will corrupt your configuration file. Any change of configuration files should be done ONLY in consultation with NICE Support!!!!

In the event of a change in configuration, restart the relevant service. Restart the Interaction Server during non-production hours.

• • • • • •

Open Interaction Server service manager and click Stop NICE CLS. Shut down Windows normally.

Reinstall the ScreenAgent software. This should only be done in consultation with NICE Support. All the files on the F: partition are managed by the NiceScreen Logger and are required. Do not delete these files. If you are using Terminal Server on Citrix check the connectivity. If you are doing your mapping in System Administrator (and not manually using a file):

• • • • • • •

In CLS Definitions > CLS Servers, select a CLS Server. Click the Configuration tab and click RCM. Validate that the Read Mapping from File parameter value is set to No.

Solutions for various additional troubleshooting and performance issues are detailed in: NICE ScreenAgent Installation/Operational Problems on page 503 NICE ScreenAgent Optimization and Performance Issues on page 505 NiceScreen Logger Configuration Parameters on page 507

Other Useful Tools

Run MML_ManagementTester.exe to view a list of the NiceScreen Loggers on the network.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

501

Isolating the Problem

Figure 16-7

MMI Management Tester

The Expansion Tool is an internal utility which enables you to expand the number of input channels and/or the amount of storage capacity of the NiceScreen Logger. The Expansion Tool performs the necessary modification and adjustment to the NiceScreen Logger data system and configuration file while preserving the existing metadata. For further details, see the NiceScreen Expansion Tool Guide.

IMPORTANT This tool should only be used in consultation with NICE Support!!!!

Run LogCollector.exe on the workstation to gather the Player log, Playlist, Screen Log and any other selected files. The files will be collected in the C:\Program Files\NICE Systems\NICE Player\temp folder. After a full day of collecting the logs, zip the directory and send the zip file to NICE Technical Support.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

502

NICE ScreenAgent Installation/Operational Problems

NICE ScreenAgent Installation/Operational Problems
If... You want to verify if a remote control application is installed on the workstation Do the following... You need to check that no secondary capture driver was installed by the remote-control application. To do so follow the following steps: 1. 2. Enter the Registry (use RegEdit) Examine the value of the “\Device\Video0” key located in the following hive path: HKEY_LOCAL_MACHINE\\HARDWARE\\DEVICE MAP\\VIDEO. The video driver name is identified by the string that is located before the \Device0 string. For example: If the string is: “\Registry\Machine\…\Services\atirage3\Device0”, the video driver name is: atirage3. 3. Locate the video driver name (found in the above step) in the following hive path: HKEY_LOCAL_MACHINE\SYSTEM\CurrentCont rolSet\Services. Enter the Device0 folder located under this folder. Open the InstalledDisplayDrivers key. The InstalledDisplayDrivers key contains a list of the device drivers that are loaded for the Display adapter. In a normal case the list should contain one driver which is the driver used by the Display adapter itself. If more than one driver is listed then there is a good chance that a remote-control application added the second driver.

4. 5.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

503

NICE ScreenAgent Installation/Operational Problems

If... ScreenAgent installation fails and regular uninstallation procedures fail

Do the following... Manually remove all ScreenAgent related files as follows: 1. 2. 3. Make sure ScreenAgent is down (use ScreenAgentTester) Delete the ScreenAgent folder from the workstation’s hard disk. Open the Workstation registry and delete the following keys: Before deleting this key, write down the string value of AppGUID. HKEY_LOCAL_MACHINE\Software\NICE Systems\NICE Screen.

Under the following key HKEY_LOCAL_MACHINE\Software\Microsoft\Wi ndows\CurrentVersion\Run - Delete the following String keys: AGNTREC and AGENTREC (if exist). Open the following hive: HKEY_LOCAL_MACHINE\Software\Microsoft\Wi ndows\CurrentVersion\Uninstall. There are several keys with a long string name (for example: {23e76x3…-…-…-…-…}). Search for the key that matches the string from the AppGUID key. When located, delete the key.

4. ScreenAgent fails to load

Reboot the workstation and install the ScreenAgent.

Change the order in which the ScreenAgent loads. To do this:

Increase the WaitForActivate parameter in the Agent registry file.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

504

NICE ScreenAgent Optimization and Performance Issues

NICE ScreenAgent Optimization and Performance Issues
This table provides adjustments for optimized performance.
If... There is a slow-down in general performance of the workstation Do the following... Make the following changes in ScreenAgent registry.

• • • •

Set the CaptureMethod to Scraper in the Agent registry file. Set the CompressionLevel to 1 in the Agent registry file. Lower the workstation’s screen resolution/color depth. Set EnableAnimation=0, EnableFullDrag=0, EnableScreenSaver=0, and EnableWallPaper=0 in the Agent registry file. Obtain and install the latest display drivers. Scraper CaptureMethod only: Lower Thread Priority to 0 (Below Normal). Lower the workstation’s screen resolution/color depth. Set EnableAnimation=0, EnableScreenSaver=0, and EnableWallPaper=0 in the Agent registry file. Set EnableFullDrag=1 in the Agent registry file. Set CompressionLevel=2 in the Agent registry file. Increase the value of CacheSize in the Agent registry file. Set CaptureMethod=1 (Hooking) in the Agent registry file. In most cases, the Scraper method produces more data than the Hooking method, but there are exceptions to this rule, therefore it is advisable to check this per workstation. Make adjustments for a high data rate (see the previous item in this list) Increase RecBuffSize in the Agent registry file. Increase the interval between two consecutive KeyFrames: increases MinTimeBetweenKF in the Agent registry file.

• •
The Data Rate is too high

• • • • • •

The ScreenAgent disconnects due to a high data rate

• • •

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

505

NICE ScreenAgent Optimization and Performance Issues

If... The following message appears when you start the workstation: Capture layer initialization failed. Application is aborting. If you are after installation restart the computer.

Do the following... This message indicates that the Capture component did not successfully load.

• •

Increase the timeout of WaifForActivate in the Agent registry file. Check for conflicts with the Display Driver or other application that is preventing ScreenAgent to load the Capture component. Change CaptureMethod in the Agent registry file.

You need to change the scraping method used by the Scraper capture method.

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

506

NiceScreen Logger Configuration Parameters

NiceScreen Logger Configuration Parameters
Your system settings for your NiceScreen Logger are stored in the MMLogger.cfg configuration file located in the NiceScreen Logger installation directory. This section provides information about the parameters contained in the MMLogger.cfg file.
WARNING Do not edit values in the MMLogger.cfg file. Editing values directly in the MMLogger.cfg file can cause your system to malfunction!

The following table lists the parameters contained in the MMLogger.cfg file:
NiceScreen Logger Configuration Parameters Parameter General Logger_Id Max_Input_Bandwidth_KbSec Periodical_Check_Resource_Hours Logical ID for the NiceScreen Logger Not used. Determines an idle time interval between which the Logger saves all unsaved buffers and synchronizes time with the client for a specific channel. Default = 1 Default = 6000 Default = 1 Description Value

Concurrency Num_Of_Dispatch_Threads Num_Of_JobProcessor_Threads File Logic_File_Size_In_Kb Same as sector in a conventional file system. Used once for Data System construction. A large file size will reduce the Data System size but could cause ineffective disk usage. Part of the file which the Logger retains in memory, called cache quantum or page. Usually the Logger reads/writes a whole page from/to disk. This value should not exceed the file size. A large page size improves I/O operation performance but utilizes a lot of computer memory. Default = 384 Number of threads in the Logger engine thread pool. Job processor thread pool size. Note: Do not define more than 20. Default = 25 Default = 5

File_Page_Size_In_Kb

Default = 128

Storage

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

507

NiceScreen Logger Configuration Parameters

NiceScreen Logger Configuration Parameters (Continued) Parameter Storage Description Value

This section lists all the storage portions used by NiceScreen Logger. The number on the right side of the equal sign (=) indicates the order in which this portion is used for Logger storage. Each portion (file and partition), must be listed here with its ordinal number. Example: STORAGE_PORTION_1 = 1 STORAGE_PORTION_2 = 2 Note: You can define your own storage portion name, but you must then use this same name to configure the Storage Portion parameters (path+size). See below.

Storage Portion

Each storage portion must be defined as a separate section in the file using the same portion name that appears in the Storage section (described above). Two parameters are listed for each portion:

File_Address - specifies portion location If the storage portion is a file, the complete path to the file must be defined. If the portion is a partition, the partition letter must be defined as follows: \\.\X, where X is an example of a partition letter.

Size_In_Mb - size of the portion in MB

Following is an example of a file storage portion: [STORAGE_PORTION_1] File_Address = D:\Data\Storage_File_Name1.dat Size_In_Mb = 5000 ; MB Following is an example of a partition storage portion: [STORAGE_PORTION_2] File_Address = \\.\Z: Size_In_Mb = 17000 ; MB File_Address Address of file or partition. The Logger will create a file if it does not exist yet. Default = D:\MMLData\M ML_Data_File_1 .dat or \\.\Z:

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

508

NiceScreen Logger Configuration Parameters

NiceScreen Logger Configuration Parameters (Continued) Parameter Size_In_Mb Description Size of storage. Note: Even if a file or partition is larger than this value, the Logger will use the amount of storage space defined here. Data System DB_Path Path of the Logger’s Data System files. Default = D:\MML_Root\D ataSystem Default = 200 Value Default = 5000

Num_Of_Cached_Pages_For_Table

Number of Data System Table pages that the Logger retains in memory. Note: We recommend retaining the Number of Input Channels + Number of Output Channels * 2 pages. However, if your computer does not have enough memory, you can define a lower value.

Average_Packet_Size_Kb

Average packet that the client workstation sends to the Logger. Dependent on client behavior. Note: A smaller value may increase the number of descriptors resulting in a larger Data System size. A larger value may cause ineffective storage usage.

Default = 10

Auto Deletion Auto_Deletion_Enabled Enables/Disables the auto deletion mechanism. Auto deletion starts when the percentage of stored files is greater than this value. Auto Deletion stops when the percentage of stored files is less than this value. Auto deletion starts when the percentage of descriptors exceeds this value. Auto Deletion stops when the percentage of used descriptors is less than this value. Default = 1 1 = Yes, 0 = No Default = 80 Default = 50 Default = 80 Default = 50

Files_High_Watermark_Percent Files_Low_Watermark_Percent Descriptors_High_Watermark_Percent Descriptors_Low_Watermark_Percent Channels Num_Of_Input_Channels

Number of recording input channels. Used once for Data System construction.

Default = 100

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

509

NiceScreen Logger Configuration Parameters

NiceScreen Logger Configuration Parameters (Continued) Parameter Num_Of_Output_Channels Memory Cache Cache_Confidence_Coefficient Increases memory cache size to provide ample memory above the memory allocated per recording channel. If the Logger does not have enough memory, this value can be decreased. Memory Allocation Protocol_Message_Size Protocol_Message_Allocator_Coeffici ent Log_Message_Size Maximum message size buffer for request and response (not data!!). Allocation coefficient for static memory pool of protocol messages. Log message buffer (response) in bytes. Maximum log message size that would be sent to the remote client/log server. Number of log messages in the static pool. Remaining log messages are then rejected until space in the pool is freed. Default = 1024 Default = 3 Default = 512 Default = 3 Description Number of output channels used for playback. Value Default = 3

Log_Messages_In_Pool

Default = 500

Communication Connection_Alive_Check_Interval_In_ Sec Connection_Alive_Check_Duration_In _Sec Logger_Port Max_Async_Writes_Per_Connection Time interval (seconds) for checking whether an idle connection is alive. Time duration (seconds) of the check connection time. Accept incoming connections port. Maximum number of asynchronous write operations per connection. -1 = no restriction Remaining requests are queued. A higher value improves performance for a system with a slow disk. Allocation_Retries_On_Full_Cache Number of retries to allocate cached buffer, when Memory Manager is full, before aborting. A higher value reduces performance in a busy system but will give greater safety, since the number of aborts will be reduced. Default = 10 Default = 60 Default = 3 Default = 30100 Default = 10

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

510

NiceScreen Logger Configuration Parameters

NiceScreen Logger Configuration Parameters (Continued) Parameter Items_Limit_In_Connection_Output_Q ueue Description Watermark threshold of items in output queue per connection, before error is created. A higher value increases memory usage and possibly reduces performance, but ensures greater safety since there are no aborts on output. Value Default = 50

Logs to Screen Severity_To_Screen Logs to File LogFile_Path Location of the NiceScreen log files. Default = C:\Program Files\NICE Systems\Multim edia Logger Number_Of_LogsFiles Size_Of_Each_LogFile Severity_To_File Logs To System Log Host_Name Severity_To_SysLog Publisher General Verbose_Mode Do not change this parameter. Default = 32 Name of host that appears in the message source in the System Log. Log severity of system log. Default = MML Default = 1985 Size of each log file in Kb. Log severity of log files. Default = 5 Default = 1000000 Default = 18417 Log severity to screen. Default = 2033

Chapter 16: NiceScreen Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

511

Blank page for double-sided printing.

17
Storage Center Troubleshooting Flow
This chapter describes common troubleshooting issues and how to verify that the Storage Center was properly installed. It also includes a troubleshooting workflow to solve various issues.

Contents
Common Troubleshooting Issues...............................................................................514 Installation Freezes: SNMP Dependencies..............................................................514 Archiving Failure: Multiple Network Interface Cards ................................................515 Storage Center Is Not Communicating with the NMS ..............................................515 Allowing Duplicate Archiving ....................................................................................521 Storage Center Does Not Archive Calls ...................................................................521 Other Useful Tools....................................................................................................524 Verifying the Storage Center Installation ...................................................................526 Implementing the Troubleshooting Workflow............................................................527 Verifying that the Storage Center Database is Selected ..........................................527 Verifying the Used and Free Space..........................................................................528 Gathering Storage Center Activity Statistics.............................................................528 Checking that the Archiving Rule Inserts New Calls ................................................528 Troubleshooting the Archiving Rule .........................................................................529 Checking the Storage Center Archiving ...................................................................531 Checking the Storage Center Logs ..........................................................................531 Checking the Nice Storage Streaming Service Logs................................................532 Verifying Playback from Storage Center Only ...........................................................533

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

513

Common Troubleshooting Issues

Common Troubleshooting Issues
Installation Freezes: SNMP Dependencies
During the software installation, the SNMP service is stopped. If the SNMP service has dependencies, the service cannot be stopped. This causes the installation to freeze. This problem occurs on HP servers. Workaround: Stop the dependencies manually and then run the installation.
To stop SNMP service dependencies: 1. From the Control Panel, select Administrative Tools > Services. The Services window

appears.
2. Double-click the SNMP service to open its Properties window. Click the Dependencies tab.

In the example above, the SNMP service has four dependencies.
3. If the SNMP service has dependencies, do the following: a.

Close the SNMP Service Properties window. Repeat this procedure, for each dependent service.

b. In the Services window, right-click the dependent service and select Stop. c.

4. Install the Storage Center software. 5. After the installation is completed, restart the Storage Center. (The services will then restart

automatically.)

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

514

Common Troubleshooting Issues

Archiving Failure: Multiple Network Interface Cards
If interactions are not being archived, check if the Storage Center server has multiple active network interface cards. If the Storage Center server has more than one active network interface card, the Storage Center IP address defined in the System Administrator must be the primary network connection.
To check the network connections of the Storage Center: 1. From the Settings menu, select Network Connections. The Network Connections window

appears.
2. From the Advanced menu, select Advanced Settings. The Advanced Settings window

appears.

3. Verify that the first connection is defined with the IP address of the Storage Center server. 4. If the first connection is not defined with the IP address of the Storage Center server, change

the order of the connections (using the Up/Down arrows), or change the IP address of the first connection to the IP address of the Storage Center server.

Storage Center Is Not Communicating with the NMS
After the Storage Center is installed and the NMS server is configured, the Storage Center agent appears in the NMS Unicenter 2D Map view and the Storage Center sends traps to the NMS console. If the Storage Center agent does not appear in the Map view and the Storage Center traps do not appear on the NMS Console, check if the SNMP service, the NICE Storage Center service and the DCOM agent are configured properly. These objects are configured transparently during the installation. Perform the following verification tests to ensure that all the objects are configured properly.

Verify SCLoader Service User on page 516
Chapter 17: Storage Center Troubleshooting Flow 515

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Common Troubleshooting Issues

• •

Verify the SnmpAgent Object on page 517 Verify the SNMP Service Settings on page 520

Verify SCLoader Service User
This procedure verifies that the SCLoader Service is logging on to the same account (user) as the SNMP Agent and SNMP service objects.
To verify which user will be logged on by the SCLoader: 1. From the Start menu, select Run.

2. In the Run window, enter services.msc and click OK.

The Services window appears.

3. Right-click the SCLoader Service and select Properties. The SCLoader Properties window

appears.
Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

516

Common Troubleshooting Issues

4. Click the Log On tab. Verify that This account is selected and that the correct username

appears in the field.

The next steps verify that the SnmpAgent object and SNMP Service both log on using the same account.

Verify the SnmpAgent Object
This procedure verifies that the SnmpAgent object is logging on to the same account (user) as the SCLoader and SNMP service objects.
To verify the SnmpAgent Object: 1. From the Start menu, select Run. 2. In the Run window, enter dcomcnfg and click OK.

The Component Services window appears.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

517

Common Troubleshooting Issues

3. Navigate to Components Services > Computers > My Computer > DCOM Config.

4. Right-click the SnmpAgent Object and select Properties.

The SnmpAgent Object Properties window appears.

5. Click the Identity tab. 6. Verify that the This user radio button is selected, and that the User is the same user as

defined in the other Storage Center objects running this application.

• •

If the user is properly defined, this procedure is complete. Continue with Verify the SNMP Service Settings on page 520. If the user is not properly defined, continue with step 7.

7. If the This user radio button is not selected or the username is incorrect, you must do the

following before changing the agent properties:
Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 518

Common Troubleshooting Issues

a.

Stop the Storage Center Server. Verify that there are no StorageCenter.exe processes running in the Windows Task Manager.

b. Stop the SNMP Service. c.

d. Select the This User radio button and define the user name. 8. Click the Security tab. Permissions areas are displayed.

9. In the Configuration Permissions area, click Edit. The Change Configuration Permissions

window appears.

10. Verify that the user belongs to the Administrators Group with Full Control Permission.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

519

Common Troubleshooting Issues

11. Click OK to close the SnmpAgent Object Properties window. Then exit the application.

Verify the SNMP Service Settings
This procedure verifies that the SNMP Service is logging on to the same account (user) as the SCLoader and the SnmpAgent objects.
To verify the SNMP Service settings: 1. From the Start menu, select Run. 2. In the Run window, enter services.msc and click OK.

The SNMP Services window appears.
3. Right-click SNMP Service and select Properties.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

520

Common Troubleshooting Issues

4. Click the Log On tab. Verify that the This account field contains the same username as the

Storage Center uses for logon.

Allowing Duplicate Archiving
By default, an interaction can only be archived one time. If an interaction matches the filter for several rules, the interaction will be archived according to the first rule that runs. It will not be archived again when other rules run. You can change this policy so that an interaction may be archived per rule. In this scenario, an interaction may be archived multiple times - each time the interaction matches a running rule filter. This is configured via the Rule Engine configuration file.
To allow multiple archiving: 1. Navigate to the following directory: Program Files\Nice Systems\Applications\Server Bin. 2. Open the NiceApplications.RuleEngine.Service.exe.config file. 3. Navigate to the following key: AllowDuplicateArchiving 4. Change the value to True. (The default is False.)

Storage Center Does Not Archive Calls
Perform the following check sequence:
Rule Engine Inserts New Calls

Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the relevant calls to see whether the Rule Engine inserts new calls:
Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 521

Common Troubleshooting Issues

Query on iArchiveID to see if it increases, and check the last interactions Start and Stop times. When the time of the last interaction is known, you can tell whether new interactions were inserted. If the Rule Engine inserts calls, the problem is in the Storage Center.

New Rows are Not Inserted

New rows are not inserted in the tblStorageCenterXX tables:

• •

Check whether new tasks are added to the tblSCTasks table in the nice_admin database. If tasks are added, check that they match the rule filters.

Errors in the Rule Engine File

Check the Rule Engine Log file for errors.
Active Rule in the Rule Manager

Check that there is an Active Rule in the Rule Manager, and check its schedule.
Storage Center Rule Schedule

In the nice_rule database, check the next schedule time of the Storage Center rule by running the following query:
SELECT tblREStorageCenterRule.nvcRuleName, tblRELocalRuleSchedule.iScheduleID, tblRELocalRuleSchedule.dtNextScheduleTime, tblRELocalRuleSchedule.dtLastRunTime FROM tblRELocalRuleSchedule INNER JOIN tblREStorageCenterRule ON tblRELocalRuleSchedule.iScheduleID = tblREStorageCenterRule.iScheduleID WHERE (tblREStorageCenterRule.iStatus = 1)

A section of the tblREStorageCenterRule table appears below, showing the format of the date fields.

dtNextScheduleTime is the GMT time. dtLastRunTime is the local time of the Application Server. Check that the Rule Engine database is not full.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

522

Common Troubleshooting Issues

Rule Engine Generates Actions

Check that the Rule Engine generates actions for the Storage Center Rules by running the following query:
SELECT * FROM tblREStorageCenterAction

Check the Start time, Stop time and interaction ids in the results of the query. If the table does not fill up, check that the Event Providers collect the interactions: Run the query:
SELECT * FROM tblRECallServerEvent

Check that the results contain calls that are not archived.

• •

If the table does not contain the calls, the error may be in the Event Collection step. Call the NICE Customer Services. If the interaction is in the Event table, the problem may be in the Generation step. The reason could be one of the following:
a. The Generation Query takes a long time and does not complete because of timeout.

Maybe there are missing indexes. See the Database Administrator.
b. The rule has problems with the file name mask and the post process of the Storage Center rule, stored procedure spREStorageCenterRulePostProcessing, failed.

Check that the File name mask is not empty by running the following query:
SELECT nvcRuleName, iRuleId, nvcDescription, iStatus, nvcFileNameMask FROM tblREStorageCenterRule

Check that nvcFileNameMask is not empty for the required rule.

The problem could be in the Execution step: In a Multi Data Hub environment, the secondary Rule Engine Log file may contain the following message: [StorageCenterExecuter.OnProcessWorkItem] Failed to execute Storage Center action System.Data.SqlClient.SqlException: The operation could not be performed because the OLE DB provider 'SQLOLEDB' was unable to begin a distributed transaction. [OLE/DB provider returned message: New transaction cannot enlist in the specified transaction coordinator. ] OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin::JoinTransaction returned 0x8004d00a]. It may be that the Microsoft Distribute Transaction Coordinator (MSDTC) is not configured properly.
Chapter 17: Storage Center Troubleshooting Flow 523

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Common Troubleshooting Issues

Storage Center rules require that the MSDCT works correctly. This can be checked by performing the following command:
BEGIN TRANSACTION SELECT * FROM nice_rule_link.nice_admin.dbo.tblSCTasks COMMIT

If the command fails, MSDTC must be fixed. See Enabling MSDTC to Function in a Multi Data Hub Environment.

In a Multi Data Hub environment, the problem could be in the Rules Manager behavior. Rules defined by the Rules Manager are relevant for all sites. The Rules definitions are written to the Master site and are replicated to all secondary sites.

Other Useful Tools

Run NMFExplorer.exe. The NMF Explorer utility provides NMF file analysis and enables you to:

• • •

Group packets by type in tree view View full packet details Save as AUD Format

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

524

Common Troubleshooting Issues

Play a file using NICE Media Player

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

525

Verifying the Storage Center Installation

Verifying the Storage Center Installation
After installing Storage Center, you verify that it was installed properly by generating a call from a Logger and then playing it back.
To verify Storage Center installation: 1. Configure and install Storage Center. 2. Restart the Storage Center. 3. Create a new storage group and unit in the System Administrator. 4. Create a new rule that generates a call from the Logger in .nmf format and plays the call back

in the Business Analyzer.
5. Open the log files for the Storage Center and Storage Streaming Service and verify that no

errors occurred.
6. Verify that the following services are enabled:

• • •

LogService Nice Storage Streaming Service Nice Storage Center Service

Verify that these services are enabled

7. If the installation was not successful, continue to Implementing the Troubleshooting

Workflow on page 527.
8. Continue to Verifying Playback from Storage Center Only on page 533.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

526

Implementing the Troubleshooting Workflow

Implementing the Troubleshooting Workflow
The diagram below describes the workflow for troubleshooting problems with Storage Center.

Verifying that the Storage Center Database is Selected
In the System Administrator, check that the Storage Center database is selected.

In System Administrator, in the Organization tree, under Master Site > Database Servers, General tab, check that the nice_storage_center database is selected in the Databases on this SQL Server area.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

527

Implementing the Troubleshooting Workflow

Verifying the Used and Free Space
Check the used and free space on the Storage Center servers and storage areas.
1. In System Administrator, in the Organization tree, under System Monitoring > Storage Center Servers branch, expand the hierarchy to view a list of all Storage Centers. 2. Expand a Storage Center to view its storage areas. 3. Select a Storage Center or one of its storage areas. The Status window appears.

Gathering Storage Center Activity Statistics
Use the Storage Center Investigator to verify that all the required files are archived on the file system. The Storage Center Investigator is installed together with the Storage Center in the same installation directory. The default path is: C:\All Programs\NICE Systems\Storage Center.

Checking that the Archiving Rule Inserts New Calls
Archiving to Storage Center starts with the archiving rule. The rule determines which calls needs to be archived and then adds the required information to the tblStorageCenter and tblSCTasks tables in the database.
1. Check the Storage Center tables, tblStorageCenterXX in the nice_interactions of the

relevant calls to verify that the Rule Engine inserts new calls. From Storage Center Investigator, query iArchiveID to see if the ID number increases, and check the Start and Stop times of the last interactions. When you know the time of the last interaction, you can verify whether new interactions were inserted.
2. Check whether new tasks are added to the tblSCTasks table in the nice_admin database. If

tasks are added, check that they match the rule filters.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

528

Implementing the Troubleshooting Workflow

If the Rule Engine inserts new calls into tblStorageCenterXX, the archiving rule is probably correct and the problem is in the Storage Center. In this case continue with Checking the Storage Center Archiving on page 531. If there are no new calls, continue with Troubleshooting the Archiving Rule on page 529.

Troubleshooting the Archiving Rule
Troubleshooting the archiving rule comprises the following steps:
1. Checking the Rule Engine Log file for errors. 2. Checking that there is an Active Rule in the Rule Manager. 3. Checking the Storage Center rule schedule. See the procedure below. 4. Checking that the Rule Engine generates actions. See page 529. To check the Storage Center rule schedule: 1. In the nice_rule database, check the next scheduled time of the Storage Center rule by

running the following query:
SELECT tblREStorageCenterRule.nvcRuleName, tblRELocalRuleSchedule.iScheduleID, tblRELocalRuleSchedule.dtNextScheduleTime, tblRELocalRuleSchedule.dtLastRunTime FROM tblRELocalRuleSchedule INNER JOIN tblREStorageCenterRule ON tblRELocalRuleSchedule.iScheduleID = tblREStorageCenterRule.iScheduleID WHERE (tblREStorageCenterRule.iStatus = 1)

A section of the tblREStorageCenterRule table appears below, showing the format of the date fields.

dtNextScheduleTime is the GMT time. dtLastRunTime is the local time of the Applications Server. 2. Check that the Rule Engine database is not full. To check that the Rule Engine generates actions: 1. Check that the Rule Engine generates actions for the Storage Center Rules by running the

following query:
SELECT *
Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 529

Implementing the Troubleshooting Workflow

FROM tblREStorageCenterAction 2. In the query results, check the start time, stop time and interaction IDs. 3. If the table does not fill up, check that the Event Providers collect the interactions by running

the query:
SELECT * FROM tblRECallServerEvent 4. Check that the results contain calls that are not archived.

• • • •

If the table does not contain the calls, the error can be in the Event Collection step. Call the NICE Customer Services. If the interaction is in the Event table, the problem can be in the Generation step. The reason could be one of the following: The Generation Query takes a lot of time and does not complete because of timeout. Maybe there are missing indexes. See the Database Administrator. The rule has problems with the file name mask and the post process of the Storage Center rule. The database function spREStorageCenterRulePostProcessing failed. Check that the File name mask is not empty by running the following query:
SELECT nvcRuleName, iRuleId, nvcDescription, iStatus, nvcFileNameMask FROM tblREStorageCenterRule

Check that nvcFileNameMask is not empty for the required rule.

The problem could be in the Execution step. In a Multi Data Hub environment, the secondary Rule Engine Log file can contain the following message: [StorageCenterExecuter.OnProcessWorkItem] Failed to execute Storage Center action System.Data.SqlClient.SqlException: The operation could not be performed because the OLE DB provider 'SQLOLEDB' was unable to begin a distributed transaction. [OLE/DB provider returned message: New transaction cannot enlist in the specified transaction coordinator.] OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin:JoinTransaction returned 0x8004d00a].

It is possible that the Microsoft Distribute Transaction Coordinator (MSDTC) is not configured properly. Storage Center rules require that the MSDTC works correctly. This can be checked by performing the following command:
BEGIN TRANSACTION SELECT * FROM nice_rule_link.nice_admin.dbo.tblSCTasks COMMIT

If the command fails, MSDTC must be fixed.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

530

Implementing the Troubleshooting Workflow

In a Multi Data Hub environment, the problem could be in the Rules Manager behavior. Rules defined by the Rules Manager are relevant for all sites. The Rules definitions are written to the Master site and are replicated to all Secondary Data Hubs.

Checking the Storage Center Archiving

Check the iFSArchiveClass column in the tblStorageCenterXX table. The values represent the following:

• • • • •

0 - Not archived yet. Check if the Storage Center service is up and check the Storage

Center log files.
1 - Archiving in progress. Check the Storage Center log files. 2 - Archived. Everything seems to be OK. 5 - Archiving failed. Check the Storage Center log files. 7 - Call was not found in the Logger. Check why the call is not found in the Logger.

Checking the Storage Center Logs
1. Set the log level to DEBUG (the level is usually set to LOW), and change the entry in the

registry:
a.

From the Start menu, select Run. The Run window appears. From the Edit menu, select Find, and enter DebugCategoryMask. Restart the Storage Center service.

b. Enter Regedit. The Registry Editor window appears. c.

d. Set the value to 255. e.

2. Review the Storage Center logs. By default, the log files are stored in the same directory

where the Storage Center is running. The default log file names are:

• •

StorageCenterLog.log StorageCenterLog1.log through StorageCenterLog100.log
Time Event Category Description

Owner

Thread ID

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

531

Implementing the Troubleshooting Workflow

3. Track the task snapshots in the log files. Each task snapshot details the current tasks and the

number of elements distributed among Loggers.
Task Name Task ID Rule ID

Logger ID

Number of Elements

Checking the Nice Storage Streaming Service Logs
You check the Nice Storage Streaming Service logs to verify that playback performed from Storage Center is successful. By default, the log files are stored in the following directory: ...\Program Files\Nice
Systems\Storage Center\Nice Playback Service SC\StorageStreaming<Process ID>Logs. The Process ID is a value that changes each time the system is restarted.

The following procedure lists the text strings you need to find in the Nice Storage Streaming Service logs in order to verify that playback from the Storage Center was successful.
To check the Nice Storage Streaming Service logs: 1. Search for the text ERR:, and verify that it does not appear in the log file. 2. Search for the text StartOutgoingSessionEvent, and verify that it appears. 3. Search for the text OpenSession, and verify that it appears. 4. In the same row as the OpenSession text, verify that Result=OK appears. 5. Search for the text StartStreamingPackets, and verify that it appears. 6. Search for the text StopOutgoingSessionEvent, and verify that it appears. 7. Search for the text StopStreamingPackets, and verify that it appears.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

532

Verifying Playback from Storage Center Only

Verifying Playback from Storage Center Only
Verifying playback from the Storage Center entails the following main steps:

• • •

Verify that the Storage Center is archiving properly. Configure NICE Perform to locate interactions in the Storage Center only. From Business Analyzer, playback an archived interaction from the Storage Center.

The procedure below comprises these steps.
To verify playback from Storage Center only: 1. Complete Verifying the Storage Center Installation on page 526. 2. In the System Administrator, change the Locate Settings to locate interactions in the Storage

Center only.
IMPORTANT When you change the Locate Settings, playback is temporarily discontinued while the Playback Service restarts. Only perform this procedure at times when playback can be interrupted. a.

Verify that you are working in Technician mode: From the Settings menu, select Technician Mode.

b. Navigate to Applications > Configurations > Locate Settings.

c.

Note the original settings in order to change them back in Step 4.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

533

Verifying Playback from Storage Center Only

d. In the Locate Settings area, select Search Storage Center only.

The Storage Center list appears.
e.

Click Add. The Available Storage Centers window appears listing all defined Storage Center servers that were not yet added the Locate Order list.

f.

Select one or several Storage Center servers and click OK. The servers appear in the list. .

g. Click Save

The Playback Service restarts and registers the new Locate Settings.
NOTE: When the Playback Service is restarting, playback is temporarily stopped. 3. From the Business Analyzer application, query for archived calls. See the Business Analyzer

Guide for more detailed information.
a.

Edit or create a query.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

534

Verifying Playback from Storage Center Only

The Advanced Query window appears.

b. Select Call Profile, and in the Archiving area, select Archived for Voice and Screen. c.

Click Save & Run. The archived interactions are listed in Business Analyzer. This list includes interactions stored in the Storage Center as well as other locations. Since we defined the Locate Settings to occur only in the Storage Center, not all these interactions will play back.

d. Play back an interaction. If playback is not successful, select another interaction until you

find one from the Storage Center.
e.

Verify in the Playback Service logs that the playback was performed from Storage Center. See Checking the Nice Storage Streaming Service Logs on page 532.

4. To change the Locate Settings back to their original settings, repeat Step 2 but use the original

Locate Settings in Step d.

Chapter 17: Storage Center Troubleshooting Flow NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

535

Blank page for double-sided printing.

A
Collecting Information Using the Log Collector
Each NICE Perform server and application stores vast amounts of information that proves useful for analyzing and troubleshooting a system. Introducing the Log Collector! The Log Collector is an easy-to-use utility that gathers the information from all your NICE Perform servers and local workstation, in the form of reports, for easy transfer for analysis. The Log Collector is compatible with all versions of NICE Perform Release 3, Service Pack 4 and higher. The Log Collector does not require an installation process. Simply copy the Log Collector files to your hard drive and you are ready to go!

Contents
What is the Log Collector? ..........................................................................................538 What is a Component?.............................................................................................538 What is a Report? ....................................................................................................539 FAQs ..............................................................................................................................540 Limitations.....................................................................................................................541 Getting Started ..............................................................................................................542 Collecting Information from NICE Perform Servers ..................................................543 Collecting Information from Your Local Machine......................................................550 Saving and Exporting Reports ....................................................................................555 Importing Saved Reports for Review..........................................................................556 Scheduling Collection ..................................................................................................558 Viewing Collected Reports ..........................................................................................560 Viewing and Saving Reports in other Applications...................................................562 Reports Created with the Log Collector .....................................................................563
Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 537

What is the Log Collector?

What is the Log Collector?
Your NICE Perform system is composed of many components such as Loggers, servers, and applications. They can be installed on one or several different machines. The Log Collector has the ability to collect all pertinent information generated by each component, in the form of reports, to assist in evaluating and troubleshooting your system. The Log Collector has two different collection groups:

Server Reports - scans your NICE Perform system configuration using two different

techniques:

Read from Database - you define the location of the SQL server where your NICE

Perform databases reside. Based on this information, the Log Collector locates all NICE Perform components and relevant information stored on all machines.

• •

Add Manually - you define a server, and optionally an SQL server. You then select only

those components whose data you want to collect.
Client Reports - collects data from your local workstation. The data collected includes configuration files, registry files, etc.

Server and Client reports can be combined in one export file. They can also be viewed side-by-side in Log Collector window. The Log Collector is compatible with all versions of NICE Perform Release 3, Service Pack 4 and higher. The Log Collector does not require an installation process. Simply copy the Log Collector files to your hard-drive and your are ready to go!

What is a Component?
A component is a functional unit of your NICE Perform system from which information can be collected for troubleshooting procedures. Each component can produce several reports. Examples of components are:

• • •

ScreenSense Playback Server SQL Server

Figure A-1 Components

Components

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

538

What is the Log Collector?

What is a Report?
A report is a piece of information retrieved by the Log Collector for a specific component. A report is the lowest level in the Log Collector Reports tree. Examples of reports are:

• •

System Administrator logs ScreenSense configuration

Figure A-2 Reports

Component Reports

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

539

FAQs

FAQs
Can I select what information to collect? Yes

The Log Collector has a discovery phase where all information that can potentially be collected is listed. You can select to exclude items from your collection.
Can I specify a time frame for information collected in each report? No

Currently, the Log Collector has a predefined time frame for each report. This is internal and can differ for each report. This time frame cannot be modified. Example: A report has a time frame for one month. You run the report on February 1. Information will be collected from January 1 until February 1. If you run the report again on February 15, information will be collected from January 15 until February 15.
Can I preschedule more than one collection? Yes

To schedule repeated collections, or separate collections for Clients and for Servers (or for selected components), start another instance of the Log Collector. Each instance of the Log Collector runs as a separate application and you can specify a separate schedule in each. Note that the first set of reports will be overwritten by the second set of reports. To avoid losing the first set of reports, save the first set before the next run of the Log Collector.
Can I export the collected reports to be opened on a different machine? Yes

To transport collected reports from one machine to another, create an Export file. Then import the file on the new machine.
Where are the reports saved?

Reports are automatically saved to a temporary location on your local machine. You cannot change this location. The reports will automatically be overwritten the next time that Log Collector is run. To save reports, create an Export file. You can also save selected files by opening them in another application and then saving.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

540

Limitations

Limitations
The following limitations are currently in effect:

• • • • •

The Packages report (in the Applications Suite section) fails when run on NICE Perform Release 3, Service Pack 4. If the SQL is down, the Automatic Discovery (Read from Database) option does not work. Reports must be selected manually. ClearSight reports are produced only when the system is configured for NT authentication. The Log Collector can only connect to Loggers which are installed as services. Loggers which are installed as processes cannot be polled for reports. The Log Collector saves its reports in your local temp folder. You must verify that there is at least 1 GB of free space on the local temp folder to store reports. Reports will not be saved if enough free space is not available. The temporary location for all Log Collector reports cannot be changed. Reports are saved in your local temp folder until they are exported to another location or overwritten by another run of the Log Collector. Reports which are not exported to another location, will be automatically overwritten at the time of the next run (scheduled or manual). When you use the Open With Viewer option to open a report, Log Collector opens the report from its source (from the server), not from the copy it saved locally. Thus the contents of the report seen might differ from the contents of the saved report.

• •

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

541

Getting Started

Getting Started
Verify that there is at least 1 GB of free space on the local temp folder to store reports. Reports will not be saved if enough free space is not available. The temporary location for reports cannot be changed. The Log Collector does not require an installation process. Simply copy the Log Collector files to your hard drive and you are ready to go!
Figure A-3 Log Collector Application Files

To start the Log Collector, double-click NiceApplications.LogCollector.exe. Continue with:

• •

Collecting Information from NICE Perform Servers on page 543
-or-

Collecting Information from Your Local Machine on page 550

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

542

Collecting Information from NICE Perform Servers

Collecting Information from NICE Perform Servers
Server Reports scans your NICE Perform system configuration using two different techniques:

Read from Database - you define the location of the SQL server where your NICE Perform

databases reside. Based on this information, the Log Collector locates all NICE Perform components and relevant information stored on all machines.

Add Manually - you define a server and select only those components whose data you want to

collect. When you create an export file, all reports that are currently in the Log Collector (Server reports and Client reports) will be joined together in the export file. Server and Client reports can be viewed side-by-side in the Log Collector window.
To collect information from all NICE Perform Servers: 1. If you have already collected Client reports, do not open the Log Collector again. The Server

reports can be added to the Client reports so that they can be included in the same export file. Continue with Step 2.
-or-

To open the Log Collector, do the following:
a.

Copy the Log Collector folder to your local hard drive.
Figure A-4 Log Collector Application Files

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

543

Collecting Information from NICE Perform Servers

b. Double-click NiceApplications.LogCollector.exe to start the Log Collector. This can

take a few seconds. The NICE Perform Log Collector appears.
Figure A-5 NICE Perform Log Collector

Selection Area

Reports

Collection Log Progress Bar

2. If Server Reports is not the current view, select Server Reports.

3. You can connect to a server to allow automatic discovery of all components. Continue with

Step 4.
-or-

You can manually add only selected components. Continue with Step 5.
-or-

You can do both.
4. To connect to a server for automatic discovery, do the following: a.

Click Read From Database

. The SQL Server Credentials window appears.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

544

Collecting Information from NICE Perform Servers

Figure A-6 SQL Server Credentials Window

b. Enter the Hostname or IP address for the SQL Server where the NICE Perform

databases reside.
c.

Select SQL Server connection credentials. Log Collector runs a discovery task and fills the Server Reports list with all the information components that can be collected from your NICE Perform servers and connected Data Hubs.
Figure A-7 Server Reports

d. Click OK.

e.

Expand a branch to see all potential reports. Clear the checkbox of any report not currently needed.
Appendix A: Collecting Information Using the Log Collector 545

NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Collecting Information from NICE Perform Servers

If you leave an item selected that is not installed (i.e., you leave ScreenSense selected and ScreenSense is not installed), it will appear as an error after the collection is run.
Figure A-8 Server Reports - Selection

Component Reports

Component Reports

5. To manually add only selected components, do the following: a.

Click Add Manually

.

The Choose Component window appears.
Figure A-9 Choose Component window

b. Enter the Host Name or IP Address of the Server where the components reside. c.

If the SQL server is installed on a different server, select Use dedicated SQL Server. Then enter its host name or IP address.

d. Select components from the list. Then click OK.
Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 546

Collecting Information from NICE Perform Servers

The selected components appear under the new branch, Data Hub:Undefined.
Figure A-10 Server Reports - Undefined

Data Hub:Undefined

6. Click Start

to start collection now or click Schedule Collection a later time (see Scheduling Collection on page 558). The Server Credentials window appears.
Figure A-11 Server Credentials Window

to start collection at

Click here to fill the entire table with the same values

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

547

Collecting Information from NICE Perform Servers

7. In the Machine Credentials area, select one of the following types of Authentication:

• •

Current Windows User Different Windows User (enter a Password and Domain (optional) for the user) TIP: You can enter the information (Authentication or Password and Domain) in one row then click Copy Credentials to fill the entire table with the same values.

8. In the SQL Server Credentials area, select one of the following types of Authentication:

• • •

Current Windows User Different Windows User (enter a Password and Domain (optional) for the user) SQL User (enter a Password and Domain (optional) for the user) TIP: You can enter the information (Authentication or Password and Domain) in one row then click Copy Credentials to fill the entire table with the same values.

9. Click Collect. The Collection begins.
Figure A-12 Server Reports Results

10. When the collection is complete, you will see the following results: a.

Successful collection is indicated by a green checkmark .

b. Unsuccessful collection is indicated by a red X c.

The log area shows the status of collected reports. To clear the log area, select File > Clear Log.

d. The Progress bar indicates Collection Finished.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

548

Collecting Information from NICE Perform Servers

11. Double-click a report that completed successfully in the Server Reports list to see its

contents in the right pane. Each report appears in a separate tabbed page. For tips on viewing more than one report at a time, see Viewing Collected Reports on page 560.
NOTE: If you double-click a report that was unsuccessful or was not selected for collection, the Log Collector will run again, for that report only, and try to locate its information and create a report.
Figure A-13 Server Tabbed Page Output

12. To save reports or to prepare them for shipping, export them as follows: a.

Click File > Export. The Save As window appears. All Client and Server reports are saved in one zipped file. The new zipped file can be imported into the Log Collector on a different machine for viewing.

b. Enter a name for the zipped file and browse to a location. Then click Save.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

549

Collecting Information from Your Local Machine

Collecting Information from Your Local Machine
Client Reports collect information from your local workstation.

When you create an export file, all reports that are currently in the Log Collector (Server reports and Client reports) will be joined together in the export file. Server and Client reports can be viewed side-by-side in the Log Collector window.
To collect information from your local machine: 1. If you have already collected Server reports, do not open the Log Collector again. The Client

reports can be added to the Server reports so that they can be included in the same export file. Continue with Step 4.
-or-

To open the Log Collector, proceed with the following steps.
2. Copy the Log Collector folder to your local hard-drive.
Figure A-14 Log Collector Application Files

3. Double-click NiceApplications.LogCollector.exe to start the Log Collector. This can take a

few seconds. The NICE Perform Log Collector appears.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

550

Collecting Information from Your Local Machine

Figure A-15

NICE Perform Log Collector

Selection Area

Reports

Collection Log Progress Bar

4. Select Client Reports.

All information that the Log Collector can potentially collect from your local machine appears under the Machine LOCALHOST branch. This includes machine settings and NICE applications.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

551

Collecting Information from Your Local Machine

Figure A-16

Client Reports

5. Expand the Machine LOCALHOST branch to see all potential reports. Clear the checkbox of

any report not currently needed. If you leave an item selected that is not on your machine (i.e., you leave Player selected and Player is not installed), it will appear as an error after the collection is run. The information found on your machine will be collected and saved in the form of separate reports.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

552

Collecting Information from Your Local Machine

Figure A-17

Client Reports - Selection

Component Reports Component Reports

6. Click Start

to start collection now or click Schedule Collection a later time (see Scheduling Collection on page 558).
Figure A-18 Client Reports Results

to start collection at

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

553

Collecting Information from Your Local Machine

7. When the collection is complete, you will see the following results: a.

Successful collection is indicated by a green checkmark .

b. Unsuccessful collection is indicated by a red X c.

The log area shows the status of collected reports. To clear the log area, select File > Clear Log.

d. The Progress bar indicates Collection Finished. 8. Double-click a report that completed successfully in the Client Reports list to see its contents

in the right pane. Each report appears in a separate tabbed page. For tips on viewing more than one report at a time, see Viewing Collected Reports on page 560.
NOTE: If you double-click a report that was unsuccessful or was not selected for collection, the Log Collector will run again, for that report only, and try to locate its information and create a report.
Figure A-19 Client Tabbed Page Output

9. To save reports or to prepare them for shipping, export them as follows: a.

Click File > Export. The Save As window appears. All Client and Server reports are saved in one zipped file. The new zipped file can be imported into the Log Collector on a different machine for viewing.

b. Enter a name for the zipped file and browse to a location. Then click Save.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

554

Saving and Exporting Reports

Saving and Exporting Reports
When reports are created, they are stored in a temporary location on your workstation. The Log Collector provides the ability to save all reports (Server reports and Client reports) in one zipped file. This zipped file can be opened using the Log Collector on another machine or unzipped and the reports opened one-by-one.
NOTE: Some types of reports, such as log files and configuration files, can also be opened using other applications and then saved. See Viewing and Saving Reports in other Applications on page 562.

All Successful reports will be exported. Failed reports will be copied to maintain the Reports tree structure. Reports not marked to run will not be copied to the export file.
Figure A-20 Reports Exported

Create Export file from:

Export file after reimporting:

To create a export file: 1. In the Log Collector window, select File > Export. The Save As window appears. All reports

are saved in one zipped file.
2. Enter a name for the zipped file and browse to a location. Then click Save.

All reports (Client reports and Server reports) are zipped together and saved. The new zipped file can be imported into the Log Collector on another machine for viewing or unzipped and viewed file-by-file.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

555

Importing Saved Reports for Review

Importing Saved Reports for Review
Reports that were zipped using the Export feature of the Log Collector can be imported on another machine and viewed in the Log Collector window. The machine to which you import the reports, does not have to be connected to the network. Reports that are imported cannot be updated or rerun. When you import a Log Collector file, the file structure of all reports is maintained making the information you need quickly accessible.
To import exported files: 1. In the Log Collector window, select File > Import. The Open window appears. 2. Browse to the location of the saved zip file. Then click Open.

All reports (Client reports and Server reports) appear together in the Imported Reports list.
Figure A-21 Imported Reports window

Server Reports Client Reports

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

556

Importing Saved Reports for Review

3. Expand the branches in the Imported Reports list. Then double-click a report to view it in the

right-hand pane. Each report appears in a separate tabbed page. For tips on viewing more than one report at a time, see Viewing Collected Reports on page 560.
NOTE: Reports that were unsuccessful cannot be rerun from the Import Reports list.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

557

Scheduling Collection

Scheduling Collection
The Log Collector can be prescheduled to create reports at a predetermined time. Each instance of the Log Collector can be scheduled for one run. To schedule repeated collections, or separate collections for Clients and for Servers (or for selected components), start the Log Collector application again.
To schedule regular collection: 1. Open the Log Collector and select reports (Server and/or Client). 2. Click Schedule Collection
Figure A-22

. The Schedule Collection window opens.

Schedule Collection window

Click here to fill the entire table with the same values

3. Select a Start Time. 4. In the Machine Credentials area, select one of the following types of Authentication:

• •

Current Windows User Different Windows User (enter a Password and Domain (optional) for the user) TIP: You can enter the information (Authentication or Password and Domain) in one row then click Copy Credentials to fill the entire table with the same values.

5. In the SQL Server Credentials area, select one of the following types of Authentication:

• • •

Current Windows User Different Windows User (enter a Password and Domain (optional) for the user) SQL User (enter a Password and Domain (optional) for the user) TIP: You can enter the information (Authentication or Password and Domain) in one row, then click Copy Credentials to fill the entire table with the same values.

6. Click Schedule.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

558

Scheduling Collection

7. The Log Collector window is minimized and a tray icon appears in the Notification area of

your desktop until the time when the Log Collector runs.
Figure A-23 Notification Area with Log Collector Icon

When the scheduled time arrives, Log Collector runs and the Log Collector window opens with the results.
TIP: To cancel the collection before it runs, right-click the Log Collector icon and select Cancel Scheduling.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

559

Viewing Collected Reports

Viewing Collected Reports
Collected reports can be viewed from within the Log Collector window. You can open and view several reports side-by-side. Server reports and Client reports can be opened simultaneously. Some types of reports, such as log files and configuration files, can be opened using other applicable applications such as Notepad or Internet Explorer, etc. These files can then be searched or saved using these applications. See Viewing and Saving Reports in other Applications on page 562.
To view collected reports: 1. In the Log Collector window, fill the left pane of the window with either Server Reports,

Client Reports, or Imported Reports. For instructions, see the relevant sections.

• • •

Collecting Information from Your Local Machine on page 550 Collecting Information from NICE Perform Servers on page 543 Importing Saved Reports for Review on page 556

2. Expand a branch and double-click a report. Each report appears in a separate tabbed page in

the right-hand pane. You can select reports from the Server, Client, and Imported reports list simultaneously.
Figure A-24 Tabbed Pages

If you click a report tab, the report will be highlighted in the Reports tree.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

560

Viewing Collected Reports

3. To split the viewing screen:
Figure A-25 Tabbed Pages - Splitting Screen

a.

Click a tab and hold down the left mouse button until the directional indicator appears. The split screen appears.
Figure A-26 Tabbed Pages Split Screen

b. Drag the mouse to the directional indicator and select a location for the tabbed page.

4. If you have many tabbed pages opened, use the dropdown list to switch between pages.
Figure A-27 Tabbed Pages Dropdown List

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

561

Viewing Collected Reports

Viewing and Saving Reports in other Applications
Some types of reports, such as log files and configuration files, can be opened using other applications such as Notepad or Internet Explorer, etc. These files can then be searched or saved using these applications.
To open a report using another application:

In the left pane of the Log Collector window, right-click a report and select Open With Viewer. This option only appears for reports which can be opened in another application.
Figure A-28 Open With Viewer

The file opens in an appropriate application.
Figure A-29 Sample Report opened in Notepad

The file can then be saved, or searched as a standard file.

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

562

Reports Created with the Log Collector

Reports Created with the Log Collector
The Log Collector produces many reports. A current list of available reports is accessible from the Log Collector application.
To view a list of available reports created with the Log Collector:

In the Log Collector window, select Help > Reports List. The list of Log Collector reports appears.

Server-Side Reports
The following sets of server-side reports are available from each of your Data Hubs. You can select a whole group of reports or individual reports. General Server-Side Reports Applications Suite Reports Playback Telephony Services Server Reports Logger Reports SQL Server Reports Interactions Center Reports Storage Center Reports Reporter Reports NiceScreen Logger Reports ScreenSense Reports Stream Server Reports Business Services Framework (BSF) Server Reports CTI Integrations Reports Media Library Reports Audio Analysis Reports VRG Reports MPCM (Media Provider Controller Manager) Reports RTS Reports

General Server-Side Reports
General Server-Side Reports Windows Services Collects Windows Services details on the machine

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

563

Reports Created with the Log Collector

General Server-Side Reports (Continued) NICE Registry Time Settings Computer Configuration Application Events System Events Security Events Collects the registry under HKLM\SOFTWARE\NICE SYSTEMS General Collects system information data report Collects Application events Collects System events Collects Security events

Applications Suite Reports
Applications Suite Reports Retention Configuration Retention Logs Locate Logs Audio Analysis Logs Business Analyzer Configuration ClearSight Logs Packages Evaluation Charts Locate Database Settings Summary Page Logs Audio Analysis Configuration My Universe logs Administrative Database Data System Administrator Configuration System Administrator Logs Rules Manager Logs Rules Manager Configuration Business Analyzer Logs Collects Retention Service configuration file Collects Retention Service log files Collects Locate log files Collects Audio Analysis log files Collects Business Analyzer configuration file Collects ClearSight log files Collects the packages charts Collects the charts from Evaluation reports Collects Locate setting from the nice_admin.dbo.tblApplicationsSystemSettings table Collects Summary Page log files Collects Audio Analysis configuration file Collects My Universe log files Collects administrative data nice_admin database Collects System Administrator configuration file Collects System Administrator log files Collects Rules Manager log files Collects Rules Manager configuration file Collects Business Analyzer log files

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

564

Reports Created with the Log Collector

Applications Suite Reports (Continued) Investigator Logs Locate Configuration Server Bin Files Client Bin Files Lexicon Manager Logs Insight Manager Categorization Insight Manager Tasks Coaching Logs Monitor Logs Lexicon Manager Words ClearSight Categories ClearSight Topics ClearSight Categories Correlation Users Administrator Logs Rule Engine Logs Rule Engine Configuration Query Configuration Monitor Configuration Evaluation Logs Importer Logs IIS Logs IIS Configuration IP Phone Logs IP Phone Configuration Branches Logs Applications Registry Collects Investigator log files Collects Locate configuration file Collects versions of files in Applications\ServerBin directory Collects versions of files in Applications\ClientBin directory Collects Lexicon Manager log files Collects Insight Manager categorization status Collects Insight Manager Reports Collects Coaching log files Collects Monitor log files Collects Lexicon words groups Collects number of transcribed and not transcribed calls for each category Collects information about Top 20 topics for each category Collects information about ClearSight categories correlation Collects Users Administrator log files Collects Rule Engine log files Collects Rule Engine configuration file Collects Query configuration Collects Monitor configuration file Collects Evaluation log files Collects Importer log files Collects Internet Information Services (IIS) log files Collects IIS configuration Collects IP Phone Applications log files Collects IP Phone configuration file Collects Branches log files Collects registry under HKLM\SOFTWARE\NICE Applications

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

565

Reports Created with the Log Collector

Playback Telephony Services Server Reports
Playback Telephony Services Server Reports Files Versions PBSLink Configuration Playback GUI configuration Playback Administrator Configuration Resource Manager Configuration General Logs Playback Telephony Services Report Charts Configuration Collects file versions of Playback Telephony Services Server Collects PBSLink configuration file Collects Playback GUI configuration file Collects Playback Administrator configuration file Collects Resource Manager configuration file Collects Playback Telephony Services Server log files Collects the chart from Playback Telephony Services Server reports Collects Playback Telephony Services Server configuration

Logger Reports
Logger Reports File Versions Voice Logger Configuration Voice Logger SNMP Registry Voice Logger SCSI Registry Voice Logger Registry VoIP Firmware Files Voice Logger Logs VoIP CIM Service Registry Voice SCSI Registry VoIP Registry VoIP %APPDATA% Logs VoIP Logs Collects Voice Logger files versions Collects Resource Manager configuration files Collects Voice SNMP Agent registry under HKLM\SOFTWARE\NICE Systems\VoiceLoggerSnmpAgent Collects Voice SCSI registry under HKLM\HARDWARE\DEVICEMAP\Scsi Collects Voice Logger registry under HKLM\SOFTWARE\NICE Systems\Setup\NICE NiceLog Collects VoIP firmware files Collects Voice Logger log files Collects VoIP Logger CIMService registry under HKLM\SYSTEM\CurrentControlSet\SERVICES\CIMService Collects VoIP SCSI registry under HKLM\HARDWARE\DEVICEMAP\Scsi Collects VoIP Logger registry from HKLM\SYSTEM\CurrentControlSet\SERVICES\Nicelog Collects VoIP Logger %APPDATA% log files Collects VoIP Logger log files
Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 566

Reports Created with the Log Collector

SQL Server Reports
SQL Server Reports Logs Calls Volume Database Information Collects SQL error and log files Collects volume of inserted calls Collects Database disk size information data report

Interactions Center Reports
Interactions Center Reports Database Server Performance Counters Call Server Performance Counters .NET Memory Performance Counters Process Performance Counters Configuration XML data SOAP files Files Versions NiceCTI Registry Interactions Center Logs Collects Database Server performance counters values Collects Call Server performance counters values Collects .NET Memory performance counters values Collects Process performance counters values Collects Interactions Center configuration Collects Interactions Center .XML files Collects Interactions Center .SOAP files Collects Interactions Center file versions Collects Interactions Center registry under HKLM\SOFTWARE\NiceCTI Collects Interactions Center log files

Storage Center Reports
Storage Center Reports Logs Configuration Process Performance Counters Collects Storage Center log files Collects Storage Center configuration files Collects Storage Center process performance counters values

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

567

Reports Created with the Log Collector

Reporter Reports
Reporter Reports Logs Reports List Collects Reporter log files Collects a list of existing reports from the database

NiceScreen Logger Reports
NiceScreen Logger Reports Registry Collects NiceScreen Logger registry under HKLM\SOFTWARE\Nice Systems Ltd.\Nice Multimedia Logger Collects NiceScreen Logger file versions Collects NiceScreen Logger log files Collects NiceScreen Logger configuration file

File Versions Logs Configuration

ScreenSense Reports
ScreenSense Reports Process Performance Counters Server Performance Counters Logs Configuration Database Configuration Collects ScreenSense process performance counters values Collects ScreenSense server performance counters values Collects ScreenSense log files Collects ScreenSense configuration files Collects ScreenSense database configuration from nice_screen_sense.dbo.tblConfiguration table

Stream Server Reports
Stream Server Reports Logs Playback Streaming Logs Encoder Configuration Encoder Logs Streaming Startup Configuration Collects Stream Server log files Collects Playback Streaming log files Collects Streaming Encoder configuration file Collects Playback Stream Encoder log files Collects Streaming Startup configuration file

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

568

Reports Created with the Log Collector

Stream Server Reports (Continued) Files Versions Collects Stream Server files versions

Business Services Framework (BSF) Server Reports
Business Services Framework (BSF) Server Reports Configuration Logs Collects Business Services Framework configuration file Collects Business Services Framework log files

CTI Integrations Reports
CTI Integrations Reports NiceCTI Registry Logs Collects NICECTI registry under HKLM\SOFTWARE\NICECTI Collects CTI Server log files

Media Library Reports
Media Library Reports File Versions Database Logs Configuration Registry Collects Media Library file versions Collects logs from the nice_interactions.tblMLLog table Collects Media Library configuration files Collects Media Library Framework registry under HKLM\SOFTWARE\Nice Systems\NICE Media Library Framework Collects Media Library Server DataBase registry under HKLM\SOFTWARE\Nice systems\Setup\NICE Applications\Media Library Server\DataBase Collects Media Library installation registry under HKLM\SSOFTWARE\Microsoft\Windows\CurrentVersion\Unin stall

Database Registry

Installation Registry

Audio Analysis Reports
Audio Analysis Reports Compure Registry File Versions Playback Administrator Logs Collects Compure registry under HKLM\SOFTWARE\NICE Applications\Audio Analysis\Compure Collects file versions Collects Playback Administrator logs files
569

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

Reports Created with the Log Collector

Audio Analysis Reports (Continued) Configuration Analysis Data Charts Process Performance Counters Logs Machine Information Analysis Data Collects Audio Analysis configuration file Collects analysis graphic data Collects Audio Analysis process performance counters values Collects Audio Analysis log and text files Collects machine information Collects the analysis data

VRG Reports
VRG Reports Database Configuration Collects VRG database configuration from nice_admin VRG tables

MPCM (Media Provider Controller Manager) Reports
MPCM (Media Provider Controller Manager) Reports Logs Collects MPCM log files

RTS Reports
RTS Reports RTS Logs RTS File Versions Collects RTS log files Collects RTS file versions

Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

570

Reports Created with the Log Collector

Client-Side Reports
The following sets of client-side reports are available from your local machine. You can select a whole group of reports or individual reports. General Client-Side Reports Player Reports ScreenAgent Reports ScreenSense Agent Reports

General Client-Side Reports
General Client-Side Reports IE Local Machine Settings IE Trusted Sites Settings NICE Registry Computer Configuration Gets Internet Explorer settings for Local Machine zone Gets Internet Explorer settings for Trusted Sites zone Collects registry under HKLM\SOFTWARE\NICE SYSTEMS Collects system information data report

Player Reports
Player Reports Configuration Logs Collects Player configuration files from the %APPS_DATA%\NicePlayer\Release3 path Collects Player log files from the %APPS_DATA%\NicePlayer\Release3 path

ScreenAgent Reports
ScreenAgent Reports ScreenAgent Registry Logs Collects Voice ScreenAgent registry under HKLM\SOFTWARE\Nice Systems\Nice Screen Collects ScreenAgent log files from the \nice\Screen Agent path

ScreenSense Agent Reports
ScreenSense Agent Reports Logs Configuration Collects ScreenSense Agent log files from the %APPS_DATA%\Nice systems\ScreenSense log path Collects ScreenSense Agent configuration file
Appendix A: Collecting Information Using the Log Collector NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 571

Blank page for double-sided printing.

B
Enabling Display of NICE System Information in System Administrator
Contents
Overview........................................................................................................................574 Common Terms.............................................................................................................575 Verifying the System Administrator User...................................................................576 Adding the System Administrator User to Computers not in an Active Directory Environment..................................................................................................................578 Editing DCOM Permissions .........................................................................................582 Setting Security Permissions through the WMI Control...........................................588

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

573

Overview

Overview
The System Administrator includes a summary page that displays the NICE product version and the installed Updates. See the System Administrator Guide for more information.
Figure B-1 Summary Page in the System Administrator
Version

Update

This summary page is generated by accessing the product information from the computers in the NICE system. In order to access this product information, the System Administrator user needs to be included in the DCOM and WMI security settings on each computer in the NICE system. The System Administrator user is the user associated with the Nice SystemAdministrator service. This association is defined when NICE Perform applications are installed on the Applications Server. To verify the identity of this user, see Verifying the System Administrator User on page 576. In this appendix, we describe how to change the security settings to enable the System Administrator user to access information on these computers.

Workflow
The workflow differs slightly depending on whether or not you are working in an Active Directory environment.
1. Verifying the System Administrator User. To verify the identity of the System Administrator user, you check which user is associated with the Nice SystemAdministrator

service.
2. (Not Active Directory Environment Only) Adding the System Administrator User to

Computers not in an Active Directory Environment. The System Administrator user needs to be included on all computers in the NICE system. If you are not working in an Active Directory environment, you need to add this user through the Control Panel, if necessary.
3. Editing DCOM Permissions. You edit the DCOM permissions to enable the remote launch

and remote activation of WMI queries.
4. Setting Security Permissions through the WMI Control. By means of the WMI Control,

you set security permissions that enable the System Administrator user to remotely access NICE system information.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

574

Common Terms

Common Terms
Some common Microsoft terms used in this appendix are listed in the table below.
Table B-1: Common Microsoft Terms Term COM Meaning Component Object Model. COM is a platform-independent, distributed, object-oriented system for creating binary software components that can interact. Distributed Component Object Model. DCOM is a protocol that enables software components to communicate directly over a network. Windows Management Instrumentation. WMI is the Microsoft implementation of Web-based Enterprise Management (WBEM), which is an industry initiative to develop a standard technology for accessing management information in an enterprise environment. Remote WMI connections are made through DCOM. Each WMI namespace has a security descriptor, which allows each namespace to have unique security settings that determine who has access to the namespace data and methods.

DCOM WMI

WMI Namespace

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

575

Verifying the System Administrator User

Verifying the System Administrator User
The System Administrator user is the user associated with the Nice SystemAdministrator service. This association is defined when NICE Perform applications are installed on the Applications Server. If you need to verify the identity of the System Administrator user, you check which user is associated with the Nice SystemAdministrator service.
NOTE: In an Active Directory environment, verify that the domain name of the System Administrator user is the same domain as that of the computer, or is a trusted domain. To verify the System Administrator user: 1. On the Applications Server, click Start, and then select Run.

The Run window appears.

2. In the Open field, enter services.msc, and click OK.

The Services window appears.

3. In the Name column, find Nice SystemAdministrator.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

576

Verifying the System Administrator User

4. Verify the name in the Log On As column.

User Name

This name is the user name of the System Administrator.
IMPORTANT The System Administrator user cannot be Local System. The Local System user cannot be granted the correct permissions for all the machines in the system. If the System Administrator user is Local System, you need to change it to another user.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

577

Adding the System Administrator User to Computers not in an Active Directory Environment

Adding the System Administrator User to Computers not in an Active Directory Environment
The System Administrator user needs to be included on all computers in the NICE system. If you are not working in an Active Directory environment, you need to add this user through the Control Panel, if necessary.
IMPORTANT Before starting this procedure, obtain the user name and password of the System Administrator user. See Verifying the System Administrator User on page 576 to obtain the user name.

To add the System Administrator user to a computer: 1. From the Start menu, select Control Panel.

The Control Panel appears.

2. In the Name column, double-click Administrative Tools.

The Administrative Tools window appears.
3. Double-click Computer Management.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

578

Adding the System Administrator User to Computers not in an Active Directory Environment

The Computer Management window appears.

4. Expand System Tools > Local Users and Groups > Users.

A list of existing local users appears in the right pane.

5. Verify that the System Administrator user appears in the list. If the user does not appear,

proceed to Step 6.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

579

Adding the System Administrator User to Computers not in an Active Directory Environment

6. Right-click the Users directory, and then select New User.

The New User window appears.

7. Complete the fields as follows:

• • • •

User name: Enter the user name exactly as it appears in the Services window on the

Applications Server. See Verifying the System Administrator User on page 576.
Full name: Enter the first name and last name of the user. Description: (Optional) Enter text describing the System Administrator user. Password and Confirm Password: Enter the password for the System Administrator

user.
8. Clear the User must change password at next logon check box. 9. Click Create.
Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 580

Adding the System Administrator User to Computers not in an Active Directory Environment

The System Administrator user appears in the list in the left pane.
10. Proceed to Editing DCOM Permissions on page 582.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

581

Editing DCOM Permissions

Editing DCOM Permissions
You edit the DCOM permissions to enable the remote launch and remote activation of WMI queries.
IMPORTANT Before you begin this procedure, you need the System Administrator user name.

To edit DCOM permissions: 1. From the Start menu, select Run.

The Run window appears.

2. In the Open field, enter DCOMCNFG, and then click OK.

The Component Services window appears.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

582

Editing DCOM Permissions

3. In the Console Root tree, expand Component Services > Computers.

Right-click My Computer

4. Right-click My Computer and select Properties.

The My Computer Properties window appears.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

583

Editing DCOM Permissions

5. Click the COM Security tab.

6. In the Launch and Activation Permissions area, click Edit Limits.

The Launch Permission window appears.

7. In the Group or user names area, verify that the name of the System Administrator user

appears, and proceed to Step 9.
8. If the name of the System Administrator user does not appear in the Groups or user names

area, complete the following steps:
a.

Click Add.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

584

Editing DCOM Permissions

The Select Users, Computer, or Groups window appears.

b. In the Enter the object names to select area, enter the name of the System

Administrator user.
c.

Click Check Names. The user name and domain name appear.

d. Click OK.

The Launch Permission window reappears.
9. In the Group or user names area, select the System Administrator user. 10. In the Permissions for <User> area, in the Allow column, select Remote Launch and Remote Activation, and then click OK.

The COM Security tab reappears.
11. In the Access Permissions area, click Edit Limits.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

585

Editing DCOM Permissions

The Access Permission window appears.

12. In the Group or user names area, verify that the name of the System Administrator user

appears, and proceed to Step 14.
13. If the name of the System Administrator user does not appear in the Groups or user names

area, complete the following steps:
a.

Click Add. The Select Users, Computer, or Groups window appears.

b. In the Enter the object names to select area, enter the name of the System

Administrator user.
c.

Click Check Names. The user name and domain name appear.

d. Click OK.

The Access Permission window reappears.
14. In the Group or user names area, select the System Administrator user. 15. In the Permissions for <User> area, do the following:
Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2) 586

Editing DCOM Permissions

• •

Verify that Local Access is selected. Select Remote Access.

Verify Local Access Selected Select Remote Access

16. Click OK.

The COM Security tab reappears.
17. Click OK to close the My Computer Properties window. 18. Proceed to Setting Security Permissions through the WMI Control on page 588.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

587

Setting Security Permissions through the WMI Control

Setting Security Permissions through the WMI Control
By means of the WMI Control, you set security permissions that enable the System Administrator user to remotely access NICE system information.
To set security permissions through the WMI Control: 1. From the Start menu, select Run.

The Run window appears.

2. In the Open field, enter wmimgmt.msc, and then click OK.

The Windows Management Infrastructure (WMI) window appears.

Right-click WMI Control

3. Right-click the WMI Control (Local) icon, and then click Properties.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

588

Setting Security Permissions through the WMI Control

The WMI Control (Local) Properties window appears.

4. Expand Root, and select the DEFAULT namespace.

5. Click Security.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

589

Setting Security Permissions through the WMI Control

The Security for ROOT\DEFAULT window appears.

6. In the Group or user name area, verify that the name of the System Administrator user

appears, and proceed to Step 8.
7. If the name of the System Administrator user does not appear in the Groups or user names

area, complete the following steps:
a.

Click Add. The Select Users, Computer, or Groups window appears.

b. In the Enter the object names to select area, enter the name of the System

Administrator user.
c.

Click Check Names. The user name and domain name appear.

d. Click OK.

The Security for ROOT\DEFAULT window reappears.
8. In the Group or user names area, select the System Administrator user.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

590

Setting Security Permissions through the WMI Control

9. In the Permissions for <User> area, in the Allow column, select Execute Methods and Remote Enable.

10. Click OK, and then close the Windows Management Infrastructure (WMI) window.

Appendix B: Enabling Display of NICE System Information in System Administrator NICE Perform ® 3.1: Troubleshooting Guide (Rev. A2)

591

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->