Expert Workshop E143 Chapter 9

Troubleshooting 800xA
Time Synch, Graphics,
Consistency, Licensing

Extended Automation System 800xA

© ABB Consult IT - 1
Revision C
Troubleshooting
800xA System Topics

„ Time Synchronization
„ Graphics
„ Control Builder Synchronization
„ Engineering Services
„ Licensing
„ Collecting Log Information
„ Hardware Issues
© ABB Consult IT - 2
Revision C
Troubleshooting
800xA System A Typical Time Synch Scheme

Do main 800xA
Controller 800xA 800xA 800xA

AfwTime Clients

Primary Connectivity Server Af wTi me Backup Connectivity Server

AfwTime Server (Active) Se rvice AfwTime Server (Standby)
Time Adaptor Time Adaptor
CNCP Clock Slave CNCP Clock Slave
SNTP

AC800M AC800M
AC800M CNCP

CNCP Clock CNCP Clock

Master Master Backup
For a
© ABB Consult IT - 3
Revision C
Troubleshooting
800xA System Clock Protocols

„ NTP, SNTP, and W32Time in Windows

„ 800xA AfwTime
„ CNCP
„ MB300
© ABB Consult IT - 4
Revision C
Troubleshooting
800xA System NTP, SNTP, and W32Time in Windows
„ UDP Port 123 (routable over TCP/IP)
„ Supported by a wide variety of nodes (AC 800M, Windows, GPS clocks, etc)
„ Server and client roles can be activated/deactivated individually in Windows
registry
„ The W32Time Service client behavior:
„ Workgroup The PC synchronizes with time.microsoft.com once a week
„ Domain The PC synchronizes with the domain controller(s) at boot and each user login
„ The W32Time Service server modes:

The AnnounceFlags setting in Windows registry settings controls availability and quality. Default
is 10 (=not always reliable). To force “Always Available & Reliable” set AnnounceFlags to 5.

Bit value Meaning (* = default)

1 Always available
2 Automatic available *
4 Always reliable
8 Automatic reliable *
© ABB Consult IT - 5

„ More information is available on the Internet

Revision C

http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/intmgmt/26_xpwts.mspx
Troubleshooting
800xA System 800xA AfwTime
„ ABB (proprietary)
„ Synchronizes 800xA server and client nodes
„ The AfwTime Service Provider in ”Service” state is controlling the time
„ Interfaces with CNCP and MB300 Clock Sync Adaptors
„ Adjustment method:
„ SV4.1 Æ Abrupt “Clock Set” is triggered when Deviation Limit is reached (+/- 1 second)
„ SV 5.0 Æ A soft “Clock Adjust” is continuously adjusting the PC clock (+/- 20 milli seconds)
© ABB Consult IT - 6
Revision C
Troubleshooting
800xA System How the AfwTime Time is Distributed
Control network (time master)
Synchronization direction

Connectivity server

Timeserver receives the

accurate time
1.TimeServer
TimeServer

Timeserver broadcasts the

time to all nodes

1.CSLib
CSLib

CSLibbroadcasts the
time messages
TimeServerhandler

Plant network
(TCP/IP)

Node Node

CSLib
1.CSLib 1.CSLib
CSLib
© ABB Consult IT - 7

The handler receives the The handler receives the

accurate time accurate time
Revision C

TimeServerHandler TimeServerHandler
Troubleshooting
800xA System Clock Synch Status Tool in CBM
Called from CBM
Tools menu

TQ Relative error to the time source

CNCP TQ0 Time Quality undefined
„ ABB (proprietary) TQ1 Time Undefined
„ Requires RNRP
TQ2 Not externally synchronized
„ Synchronizes AC 800M CPUs and Connectivity
Servers TQ3 error > 100 ms
„ Synchronization telegrams are multicasted on TQ4 error < 100 ms
UDP port 3341
TQ5 error < 10 ms
„ Use redundant Control Networks for maximum
clock synchronization accuracy TQ6 error < 1 ms
© ABB Consult IT - 8

TQ7 error < 100 us

TQ8 error < 25 us
Revision C
Troubleshooting
800xA System Troubleshooting Time Synch Problems
Example when Controller is Clock Master
Time from Interaction window is taken over
in Connectivity servers and time clients
© ABB Consult IT - 9

Domain:
C:\Net Time /querysntp Æ Shows SNTP Time server
C:\w32tm /resync Æ command completed succesfully
Revision C
Troubleshooting
800xA System Troubleshooting with AfwApplogViewer
TimeServer
© ABB Consult IT - 10
Revision C
Troubleshooting
800xA System Problem Scenarios

„ No Sync
„ “Multiple Sync”
Computer X

Time = ???
X! W32Time
AfwTime Y!
Z! GPSTime
RTA Board W!

„ Unknown clock synchronization order

„ Drifting time in Domain Controllers
„ Unreliable time sources (or a less desired configuration) is causing frequent
“Clock Set” operations rather than soft “Clock Adjust”
© ABB Consult IT - 11
Revision C
Troubleshooting
800xA System Process Graphics Architecture

Graphics Graphics
View Handling Viewer Builder
& Navigation

Client
Machines

FSD
OPC Handler Object Client
Manager
Cache

Connectivity
Servers
Aspect
Servers
Connector Aspect FSD
Directory Server

Adapter
© ABB Consult IT - 12
Revision C
Troubleshooting
800xA System Process Graphics Files

Graphics Graphics
Builder Viewer

Source code Executable Executable

fileset FileSet FileSet

File Set Distribution

• Executable filesets on OperateItData\Temp\FSDCache

• Contains dll’s (ocx’s) and .sub files

• Push of execut able filesets depending on priorities

© ABB Consult IT - 13

• Source code filesets on OperateItData\Temp\ActiveGraphics

Revision C
Troubleshooting
800xA System Diagnostic Graphic

Bad and Failed items

must be corrected, File Set time is dependant
influences performance, if Graphic is cashed or not
correct also VB errors

Indicates how good the

subscription is
© ABB Consult IT - 14
Revision C
Troubleshooting
800xA System Diagnostic OPC data in Graphics
Shows OPC quality codes
C0 = Good
0 = Bad

Error code can be translated:

© ABB Consult IT - 15
Revision C
Troubleshooting
800xA System Resolving Unresolved References

Press … and navigate to

desired object
© ABB Consult IT - 16
Revision C
Troubleshooting
800xA System Property Window of Graphic Display

Object Types and

Aspects used

Objects in Graphic display

© ABB Consult IT - 17

User Permissions
Revision C
Troubleshooting
800xA System
Graphic Considerations and Reasons for Gray Display

Considerations

„ 256 Controls
„ 112 Different types of Controls
„ GDI objects 10 000 by per process
„ User Objects 10 000 by per process

Gray display

„ Incomplete installation (missing activeX component)

„ Resource shortage exceeded GDI and or User objects
„ FSD Problem
© ABB Consult IT - 18
Revision C
Troubleshooting
800xA System Bad Performance in Graphics
„ Unresolved references
„ Blink is enabled without using expressions in Graphical
elements
„ FSD Cache is too small or not used for Graphic display
„ Extensive use of Tabs in Faceplates
„ Extensive use of AspectView control
„ Wrong combination of Backstyle and Windowless
Allowed combinations:
1. Transparent and Windowless
2. Opaque and not Windowless
„ Wrong design of OPC reference e.g. of correct design:
© ABB Consult IT - 19

Control Module:Value
„ Redraw of Graphical Elements
Revision C

„ Network problem and slow OPC subscription

Troubleshooting
800xA System Control Builder Synchronization
Creation and deletion of objects
Plant Explorer can be made in both places, Control Builder
information is transparent
© ABB Consult IT - 20
Revision C
Troubleshooting
800xA System Plant Explorer CBM Inconsistency
Causes
1. Concurrent Engineering clients on the same project

2. Engineering Clients with none U.S English locale

3. Import when object appeared in Lost and Found

4. Object is not mirrored to Plant Explorer

Corrective Measurements
1. Close down all Control Builders, Open up one
Control Builder and correct the configuration

2. Very difficult, Usually a backup must be restored after a

language clean-up in cooperation with experts

3. Remove objects from Lost and Found. Import again

© ABB Consult IT - 21

4. Start trace log for further analysis

Revision C
Troubleshooting
800xA System Consistency Problem
„ This problem may occur during engineering – primarily
during import actions
„ Use consistency tool. The structures to test
„ Functional Structure
„ Control Structure
„ Location Structure
„ Object type structure

„ Consistency check can also be made on xxx Type

Reference aspect
„ Make repair on the lowest object in the hierarchy, check
result.
Check structures with AfwSCT –s
© ABB Consult IT - 22

„
Revision C
Troubleshooting
800xA System Consistency Problem

Start
Start consistency
consistency tool
tool from
from the
the
plant
plant explorer
explorer

Select the items to be checked

© ABB Consult IT - 23
Revision C
Troubleshooting
800xA System Consistency Problem result
Several repairable errors can be
selected and repaired

Gives more information

© ABB Consult IT - 24
Revision C
Troubleshooting
800xA System FSD Data used by CBM and OPC Server
Application Code
(XML code)

HWD definitions
Aspect Directory Control
Cold Retain Builder
Values

Cold Retain Domain Data for

Values OPC subscription

OPC Server
© ABB Consult IT - 25
Revision C
Troubleshooting
800xA System FSD Tool

You only need to enter the

unique path

Domain files
Application Code
Retain files
Cold Retain files
© ABB Consult IT - 26
Revision C
Troubleshooting
800xA System Engineering

Handles Parameter and Document

Manager data

Mandatory service must

be running in order to
make backups
© ABB Consult IT - 27
Revision C
Troubleshooting
800xA System Tools for Checking the User of MSDE

Description how to verify EBInstance

Check that the service account is defined

for the database and logins
© ABB Consult IT - 28
Revision C
Troubleshooting
800xA System Licensing System Basics
„ Based on .NET and Microsoft Internet Information Services (IIS)
License communication protocol is HTTP (WWW) on TCP port 80

„ Central Licensing Server (CLS) runs on one node with IIS

The license is tied to USB dongle or MAC-address
„ ABB Client License Provider Service runs on all 800xA nodes
Allocates / deallocates license features

„ Main License Tools:

1. License Entry Tool
2. License Status Viewer
3. License Usage Aspect

„ Support Tools:
1. License Assignment Editor
2. TestLCC.exe
© ABB Consult IT - 29
Revision C
Troubleshooting
800xA System Licensing Overview
HTTP ”WWW” requests on TCP port 80

Central Licensing Server node1 800xA Clients and Servers

Internet Information Server (IIS) ABB Client Licence Provider

SFA2
Central Licensing Service (CLS)

800xA Plant Explorer

License
.SLA License Usage aspect
Entry Tool

Internet Explorer

License Assignment Editor Internet Explorer

License Assignment
Editor Web Page
© ABB Consult IT - 30
Revision C

1
Recommended to be installed on DC or AS (but any node on the domain/workgroup with IIS is possible.
2
SFA, System Feature Agent. Active on one node per system. Responsible for counting & reporting TAGS.
Troubleshooting
800xA System License Entry Tool
StartÆProgramsÆABB Industrial IT 800xAÆSystemÆLicensingÆLE

License file .SLA

© ABB Consult IT - 31

A successful .SLA load does not constitute a working license server!

Revision C
Troubleshooting
800xA System License Troubleshooting

Enter http://localhost on the license server in order to check if IIS is installed correctly.

Reason for License violation is

shown in window

Location of Log file here you

find all license violations
© ABB Consult IT - 32

TimeStamp,ErrorCode,LicenseGroup,Feature,Node/Application,MinVersion,MaxVersion,OptionString,QuantityRequested,QuantityGranted
2006/03/12 21:18:18,0x8004ab81,800XA ENGINEERING,ENG_BDM,GIT3052,1,0,(null),20,10
Revision C
Troubleshooting
800xA System License Assignment Editor
StartÆProgramsÆABB Industrial IT 800xAÆSystemÆLicensingÆLicense Assignment Editor

WEB page!
© ABB Consult IT - 33

Create shortcut to License Assignment Editor from arbitrary node to check connection to CLS server:
http://<cls server>/ABBLicense/Default/CentralLicenseService/LicenseEditor/LicenseEditor.asp
Revision C
Troubleshooting
800xA System
© ABB Consult IT - 34
Revision C
Use Aspect to show License Count
Troubleshooting
800xA System Collecting Windows Log Information

Result can be saved

Information can be
exported as text
© ABB Consult IT - 35
Revision C
Troubleshooting
800xA System Using System Checker Tool

• PC Operating system info A summary of all 800xA software installed in the

• PC Hardware info system. You can easily compare the software with the
• Installed 800xA software report converter tool
• Installed Hot Fixes

Hyperthreading
© ABB Consult IT - 36
Revision C
Troubleshooting
800xA System Controller Logs

Upload the controller logs Zip folder

© ABB Consult IT - 37
Revision C
Troubleshooting
800xA System Collecting Logs Using DCT Tool
The DCT tool can be used to collect logs
from the installed products without having
knowledge where the logs are located.
Note that controller logs must be manually
retrieved from the controller first.

If installed on other drive than C:\ you

must edit file IITConfig.xml

Retrieve the information which is

appropriate for you problem

User dump can be taken if you

have a hanging workplace
© ABB Consult IT - 38
Revision C
Troubleshooting
800xA System Searching for Crash Dump Files
„ Use a search to find Dr. Watson files , Dump files and exception
files, make the search as follow:
„ DrWtsn*
„ *.dmp
„ *exception*.txt

Search hidden files and folders

© ABB Consult IT - 39
Revision C
Troubleshooting
800xA System Hardware Issues

Topic Server Client

Hyperthreading Disabled on AC 400 On some cases it needs to
connect and PNSM servers be disabled even on clients
timing issues

Data Execution Prevention Allow for all programs make Allow only for essential
(DEP) exclusion for windows program and
AfwOPCShadow services
© ABB Consult IT - 40
Revision C
Troubleshooting
800xA System
© ABB Consult IT - 41
Revision C
ABB Logo
Maintenenance Expert workshop

Exercise 9.1 System 800xA Troubleshooting Time Synchronization

9.1.1 Description
This exercise deals with trouble shooting time synchronization. The exercise deals
with a time synchronization scheme where the time source is a controller and the time
in the Client/Service net is distributed by means of AfwTime.

9.1.2 Objectives
Upon completion of this exercise you will be able to:
• Be able to check the time synchronization by using tools in the Control
Builder.
• Be able to follow time synch events by using the AppLog Viewer.

9.1.3 Reference documentation

• Documentation in workshop binder.
• SV 4.1 Administration and Security (3BSE037410R4101)

1/3
 E143-09 Exercise 9.1 - RevA.doc

9.1.4 Exercise Steps

1. Start the Control Builder, select Maintenance. What type of nodes do you see what
is the time quality?

2. Start the AppLog Viewer select the AfwTime server node.

3. Select the AfwTimeSrv processes.
4. Select the log AfwTimeSrv Basic. Set the level to 3.
5. Do you get time received messages? To which node is the time distributed?

6. Let the log run for AfwTimeSrv process.

7. Select the AfwTimeSrv process for the AfwTimeSrv in standby. Make the same
setting as above.
8. Stop the AfwTimeSrv process in service.

2/3
Maintenenance Expert workshop

9. Does the AfwTimeSrv in standby go to Service state?

Do you get time received messages? Is the time distributed?

10. Go on-line with the control builder.

11. Open up the Set Time Interaction window. Set the time in the controller.
12. Is the new time distributed to all controllers?

13. Why can’t you use the command C:\Net Time /querysntp for trouble shooting in
the classroom set-up?

3/3
Maintenenance Expert workshop

Exercise 9.2 System 800xA Troubleshooting Graphics

9.2.1 Description
This exercise deals with trouble shooting Graphics. The students are supposed to use
the display tool as analyze aid.

9.2.2 Objectives
Upon completion of this exercise you will be able to:
• Be able to use the display tool for checking Graphic problems.
• Be able to correct Graphics by using the Graphics Builder.

9.2.3 Reference documentation

• Documentation in workshop binder.
• Graphics (3BSE030335)

1/2
 E143-09 Exercise 9.2 - RevA.doc

9.2.4 Exercise Steps

1. Select the Process Graphic display “Bad Display” in the object Support Site.
2. Use the Diagnostic tool. What type of errors do you get?

3. Start the Graphics Builder. How can you repair the Process Graphic Display?

2/2
Maintenenance Expert workshop

Exercise 9.3 System 800xA Troubleshooting Cold Retain after an

Import of an Application

9.3.1 Description
This exercise deals with Cold Retain issues in connection with import of an
application.

9.3.2 Objectives
Upon completion of this exercise you will be able to:
• Be able to understand why you don’t get the last Cold Retain change after you
have made an import.
• Be able to avoid that the latest data in the controller is overwritten after a
download of the imported application.

9.3.3 Reference documentation

• Documentation in workshop binder.
• Basic Control Software (3BSE035980)

1/2
 E143-09 Exercise 9.3 - RevA.doc

9.3.4 Exercise Steps

1. Export the Application AlarmandAdmin application. Make the export with no
dependencies.
2. Save the export.
3. Make a dummy change in the application by adding a global variable.
4. Change the alarm limit of the objects Sig2, Sig 3 and Sig4.
5. Import the exported file answer yes on questions for replacing objects and aspects.
6. Make a cold download. What happens to the cold retain values of the objects
above?

7. Change the alarm limits again and make a dummy change.

8. Import the application again.
9. Before you make a download save the Cold retain files from the OPC server.
10. What is the different this time after you have made the cold download?

2/2