SilkTestUserGuide

users guide
the e-business management company version
5.5
Copyright Notice
All Segue documentation and training materials are copyrighted, and all rights are reserved. Neither the software nor any documentation that accompanies it may be reproduced, translated, or reduced to any electronic or printed form without the prior consent of Segue Software, Inc., except as authorized in the terms of a valid license agreement.
(c) Copyright 1992 2001 Segue Software, Inc.

All Rights Reserved. Printed in the United States, September 1999.
Trademarks
eConfidence, E-Quality, E-Quality Partner, E-Quality Partner Program, 4Test, QA DBTester, QA Organizer, QA Partner, QA Performer, QA Radar, QualityWorks, Segue, SilkObserver, SilkPilot, SilkMeter, SilkMonitor, SilkPerformer, SilkRadar, SilkRealizer, SilkTest, Universal Testing Architecture (UTA), LiveQuality, SilkBeans, scenario testing, E-Business Management System, EBMS and the e-business management company are trademarks or registered trademarks of Segue Software, Inc. All other product and company names are either trademarks or registered trademarks of their respective companies.
Warranties and Disclaimers

This publication is provided "as is" without warranty of any kind, either expressed or implied, including, but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. Segue assumes no responsibility for errors or omissions in this publication or other documents which are referenced in or linked to this publication. This publication could include technical or other inaccuracies or typographical errors. Changes are periodically added and will be incorporated in new editions of the publication. Segue may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time. Should you or any viewer of this publication respond with information, feedback, data, questions, comments, suggestions or the like regarding the content of any Segue publication, any such response shall be deemed not to be confidential and Segue shall be free to reproduce, use, disclose and distribute the response to others without limitation. You agree that Segue shall be free to use any ideas, concepts or techniques contained in your response for any purpose whatsoever including, but not limited to, developing, manufacturing and marketing products incorporating such ideas, concepts or techniques. This publication is distributed internationally and may contain references to Segue products, services, and programs that have not been announced in your country. These references do not imply that Segue intends to announce such products, services or programs in your country.
Questions or comments
If you have any questions for our technical support staff, please contact us at support@segue.com. You can also phone 781.402.5900 (from North America) or +44 (0) 2890260001 (International). When you get in touch, please have your customer ID number available. If you have any questions about the documentation, please contact us at documentation@segue.com. Segue software is covered by U.S. Patent Numbers 5,600,789 5,781,720 6,046,740
Contents
Preface xi
Where this manual fits in xi Typographical conventions xiii
Part I The Basics

Chapter 1 Overview 17
The automated testing process 17 Getting started 19 The anatomy of a basic testcase 26 The built-in recovery system 27 SilkTest architecture 27 The 4Test language 29 The basic tools 34 Online information 38 Where to go from here 42
Chapter 2
Creating Testplans
47
Starting a new testplan 47 Understanding the outline model 48 Structuring an outline 50 Using a template of the GUI hierarchy as an outline Linking the testplan to scripts and testcases 59 Documenting manual tests in the testplan 65
58
Chapter 3
Enabling Extensions for Applications Under Test

How extensions work with applications 67 When to enable extensions 68 General process for enabling extensions 71 Enabling browser extensions 72 Disabling browser extensions 75
67
Chapter 4
Recording a Test Frame
77
Why window declarations make your tests robust 78

Users Guide iii
Preparing to record window declarations 78 How to record window declarations 78 Window declaration syntax and structure 83 Modifying identifiers and tags 93 Mapping custom classes to standard classes 94 Specifying how a dialog is invoked 96 Defining a method for a GUI object 96 Defining a data structure for an object 98 Modifying the message box declaration 99
Chapter 5
Designing and Recording Testcases
101
Preparing to design and record testcases 102 Test running the built-in recovery system 105 Testcase design principles 106 Defining test requirements 108 Overview of recording the stages of a testcase 109 Recording the setup stage 112 Recording the verification stage 112 Recording the cleanup stage and pasting the recording 132 Saving a script file 133 Using object files 134 Recording from within a testplan 136 How recorded commands uniquely identify GUI objects 138 Recording without window declarations 138 Recording 4Test components 139
Chapter 6
Running Tests and Interpreting Results

Preparing to run tests 141 Running tests 144 Results files 148 Using the results file to find errors 152 Fixing errors in a script 155 Managing results file information 156
141
Chapter 7
Using the Debugger
171
Starting the debugger 172 Setting breakpoints 173 Executing a script in the debugger 175 Viewing the debugging transcript 177 Viewing the call stack 178 Viewing other script elements 178 Evaluating expressions 179 Running, resetting, and terminating execution
iv Users Guide
180
Tips on debugging
180
Part II Enhancing Your Tests

Chapter 8 Generalizing a Testcase 185
Using application states 185 Using data-driven testcases 190 Testing applications that have been made Year 2000 compliant
195
Chapter 9
Handling Exceptions
209
Default error handling 209 Using do...except 210 Performing more than one verification in a testcase Adding to the default error handling 212 Trapping the exception number 214 Writing an error-handling function 214 Programmatically logging an error 216 Defining your own exceptions 216 Enabling fault trapping 217
211
Chapter 10
Adding Data to a Testplan
221
223
Linking a testplan to a data-driven testcase 222 Specifying data that is unique to a single test description Specifying data that is shared by multiple tests 226 Converting the data in a results file into a plan 232
Chapter 11
Categorizing and Marking Testplans

Overview 235 Defining and modifying attributes and values 237 Assigning attributes and values to a testplan 240 Marking a testplan 243 Creating and modifying testplan queries 246
235
Chapter 12
Working With Large Testplans
255
Navigating through a large testplan 256 Measuring testplan completion 258 Dividing a testplan into a master plan and subplans 259 Editing a master plan in a multi-user environment 262
Users Guide
Part III Cross-Platform Testing

Chapter 13 Porting Tests to Other GUIs 267
Marking 4Test code as GUI-specific 267 Conditional compilation 271 Supporting GUI-specific executables 273 Supporting GUI-specific captions 274 Supporting GUI-specific menu hierarchies 275 Supporting extra controls 275 Supporting different implementations of the same control 276 Supporting differences in application behavior 279
Chapter 14
Supporting Internationalized Applications
283
284
Built-in support for international keyboards 283 Testing applications with single-byte international characters Internationalizing tags 284
Part IV Customizing SilkTest

Chapter 15 Understanding the Recovery System 289
Introduction to the recovery system 289 Setting up the recovery system 290 The recovery systems flow of control 291 How the recovery system starts the application 292 How the recovery system closes windows 292 Specifying new window closing procedures 293 Specifying windows to be left open 296 Handling login windows 297 Adding to the default base state 298 Overriding the default recovery system 299 Modifying the default recovery system 301
Chapter 16
Extending the Class Hierarchy

The class hierarchy 304 Adding methods to a class 306 Defining new class properties 308 Defining new verification properties Defining new attributes 311 Defining new classes 314
303
308
vi
Users Guide
Chapter 17
Supporting Custom Objects
317
Why SilkTest sees objects as custom windows 317 Mapping custom classes to standard classes 318 Supporting graphical controls 320 Supporting nongraphical controls 323
Chapter 18
Modifying the Library Browser
327
The standard Library Browser contents 327 Modifying the Library Browser contents 329
Part V Testing Client/Server Applications

Chapter 19 Introduction to Client/Server Testing
Client/server testing challenges 337 Client/server testing with SilkTest 338 The kinds of network testing you can perform Client/server testing configurations 343 Features of the database tester 346 Concurrent programming issues 347
337
339
Chapter 20
Configuring SilkTest for Client/Server Testing

Understanding host and target machines 351 Networking protocols used by SilkTest 352 The configuration process 353 Selecting the SilkTest protocol 354 Configuration tasks 354
351
Chapter 21
Implementing Client/Server Testing

Evolving a testing strategy 362 Connecting to a target machine 362 Using 4Tests parallel processing features 363 Reporting distributed results 370 Incremental functional test design 371 Running tests on one remote target 372 Running tests serially on multiple targets 372 Testing clients plus server serially 373 Testing clients concurrently 374 Testing in parallel, but not synchronously 376 Application states and the recovery system 378
361
Users Guide
vii
Chapter 22
Multi-Application Testing
379
Multi-application startup and recovery 380 Testcase structure in a single-application environment 380 Testcase structure in a multi-application environment 381 Recording window declarations for remote machines 384 Code for template.t 384 template.t explained 385 Code for concurrency test example 387 Concurrency test explained 389 Code for notification example 1 393 Notification example 1 explained 394 Code for notification example 2 397 Notification example 2 explained 398
Part VI GUI-Specific Tips and Tools

Chapter 23 Using the Windows Bitmap Tool 401
Introduction to the Windows Bitmap Tool 401 Windows Bitmap Tool window 403 Starting the Windows Bitmap Tool 406 Capturing bitmaps 408 Comparing bitmaps 412 Creating and applying masks 417
Chapter 24
Calling Windows DLLs From 4Test Scripts

Declaring a DLL function 423 Aliasing a DLL name 424 Using DLL support files installed with SilkTest 425 Passing arguments to DLL functions 426
423
Chapter 25
Using PVCS with SilkTest

Overview of using PVCS 429 Setting up PVCS with SilkTest 432 Using PVCS with SilkTest 432 Possible problems 438
429
viii
Users Guide
Part VII Command Reference

Chapter 26 Command Line Syntax
Syntax of the partner command Examples 445
443
443
Chapter 27
Menu Commands
Breakpoint menu 449 Debug menu 450 Edit menu 452 File menu 456 Help menu 461 Include menu 462 Options menu 464 Outline menu 486 Record menu 488 Results menu 502 Run Menu 507 Testplan menu 510 View menu 515 Window menu 518
447
Part VIII Appendices

Appendix A Appendix B Appendix C Appendix D Testplan Editor Keywords 523 Using Drag-and-Drop 527 Supporting Owner-Draw List Boxes 529 Useful Multitestcase Code Templates 531
Parallel template 531 Client/Server template 532
Appendix E
Glossary 535
Index 543
Users Guide
ix
Users Guide
Preface
The goal of this manual is to provide you with the task-oriented information you need to accomplish your day-to-day activities using SilkTest. This manual describes how to use SilkTest on Windows platforms.
Where this manual fits in

Core printed documentation
This manual is one of the manuals in the core SilkTest documentation set. The set contains the following volumes:
Manual Description
Installation Guide
Provides instructions on how to install SilkTest and our licensing software, SilkMeter.
SilkTests Getting Started: A Introduces you to using SilkTest to test your webTutorial based or client/server applications. It contains an introduction to SilkTest and related concepts, step-by-step procedures that explain how to build and run tests and testplans, and how to interpret the results. We recommend strongly that you go through the tutorial before using the rest of the documentation.
Users Guide
xi
PREFACE Where this manual fits in
Manual
Description
Users Guide
Describes how to use SilkTest, including details on creating and running tests, debugging tests, interpreting results, generalizing tests, handling errors, cross-platform testing, customizing SilkTest, GUI-specific tips and tools, and an alphabetical reference of menu commands. Note This manual is also available online: Select Help/Online Users Guide (assuming you installed the online manual when installing SilkTest).
4Test Language Reference
Contains descriptions of SilkTests 4Test language.
Online Help
SilkTest also provides extensive online Help containing the following information and more: Complete documentation for all 4Test classes, functions, statements, and declarations What is new in this release Detailed description of the Segue QA methodology Complete documentation for all installed extensions, which provide support for third-party development environments Complete documentation for all installed versions of GO!
For more information about the Help contents, select Help Topics from the Help menu and look at the topic About this Help. For information about what is new in this release, select Help Topics from the Help menu and look at the topic Whats new.
xii
Users Guide
PREFACE Typographical conventions
Typographical conventions
Convention Example Description
Italic Bold
A script file is script: myscript.t Enter myscript.t
First use of a new term. Keywords, required punctuation, actual user input, screen output, or examples. Leave exactly as shown. Variable user input. Indicates a menu pick, as in select the Testcase menu item from the Record menu. Optional arguments. Follows an element that can optionally be repeated. Indicates continuation of the statement on multiple lines indented another level. Indicates the data type of a variable, argument, or return value; iCode is an INTEGER, sString is a STRING, bResult is a BOOLEAN, wDialogBox is a WINDOW.
Bold italic /
Type install-dir\tut.t Record/Testcase
Brackets [ ] Ellipses ( ... ) Indentation Variable prefixes (i, b, n, w, ...)
SYS_Kill ( iPid [, iSignal ] ) access share-var [, share-var]... statements access share-var [, share-var]... statements iCode = Asc (sString)
Variable prefixes
Variables are specified in Hungarian notation, that is, prefixed with an abbreviation for the data type. The data types and abbreviations are as follows: anytype (a), agentoption (ao), boolean (b), browsertype (bt), dataclass (dc), datatype (dt), guitype (gt), handle (h), integer (i), number (n), point (p), real (r), rect (re), semaphore (se), string (s), string or integer (si), sbrange (sr), textpos (tp), textrange (tr), window (w), winclass (wc), winstate (ws) and wndtag (wt). Any of these types can be a list, such as list of string (ls) or list of window (lw).
Users Guide
xiii
PREFACE Typographical conventions
xiv
Users Guide
The Basics
I tP r a
In this part
This part contains the following chapters:

Chapter Page
Chapter 1, Overview Chapter 2, Creating Testplans Chapter 3, Enabling Extensions for Applications Under Test Chapter 4, Recording a Test Frame Chapter 5, Designing and Recording Testcases Chapter 6, Running Tests and Interpreting Results Chapter 7, Using the Debugger
17 47 67 77 101 141 171
Users Guide
15
16
Users Guide
1e rh C t p a
Overview
This chapter gives you the big picture, so that you can quickly begin creating powerful reusable tests in the Segue 4Test languagetests that dramatically decrease your risk and increase the return on your QA investment. Youll not only be able to do more testing with fewer staff in the same amount of time, but by reusing tests, youll maintain a single standard of quality for your application over timeacross releases, platforms, and networks. This chapter covers the following topics:
Topic Page
The automated testing process Getting started The anatomy of a basic testcase The built-in recovery system SilkTest architecture The 4Test language The basic tools Online information Where to go from here
17 19 26 27 27 29 34 38 42
The automated testing process

The testing process has these four steps: 1 Creating a testplan (if you are using the testplan editor)
Users Guide
17
1 OVERVIEW The automated testing process
2 3 4
Creating a testplan
Recording a test frame Creating testcases Running testcases and interpreting their results
If you are using the testplan editor, you begin the automated testing process by creating a testplan. A basic testplan is structured as a hierarchical outline and contains: Descriptions of individual tests and groups of tests. You can use as many levels of description as you want. Statements that link the test descriptions in the plan to the 4Test routines, called testcases, that accomplish the actual work of testing.
The following figure shows a sample testplan for a search feature. The testplan uses two levels of description: the top level indicates that all the tests are for the Find dialog, and the next level describes eight individual tests.
Group description
Test descriptions
QA Organizer statements
Recording a test frame
Next, you record a test frame, which contains descriptions, called window declarations, of each of the GUI objects in your application. A window declaration specifies a logical, cross-platform name for a GUI object, called the identifier, and maps the identifier to the objects actual name, called the tag. In addition, the declaration indicates the type of the object, called its class. The 4Test commands in a testcase collectively perform three distinct actions: 1 2 3 Drive the application to the state to be tested. Verify the state (this is the heart of the testcase). Return the application to its original state.
Creating testcases
18
Users Guide
1 OVERVIEW Getting started
You can either use the powerful object-oriented recorder to automatically capture these 4Test commands as you interact with your application, or you can write the 4Test code yourself if you are comfortable with programming languages. For maximum ease and power, you can combine these two approaches, recording the basic testcase and then extending it using 4Tests flow of control features.
Running testcases and interpreting results
Next, you run one or more testcases, either by running a collection of scripts, called a suite, or, if you are using the testplan editor, by running specific portions of the testplan. As each testcase runs, statistics are written to a results file. The results file and its associated comparison tools allow you to quickly pinpoint the problems in your application.
Getting started
If you are a first-time user and want to get up and running quickly, you may want to begin with the following: SilkTests Getting Started: A Tutorial QuickStart Wizard
Using the QuickStart Wizard

If you are using SilkTest with the testplan editor, you can use the QuickStart Wizard, which greatly simplifies the four steps of automated testing described in the preceding section. When you start SilkTest the first time (or whenever you start and have no open windows), the QuickStart Wizard is displayed automatically. You can also invoke the wizard at any time by selecting File/New and clicking the QuickStart Wizard icon. To use the QuickStart Wizard, you simply follow its prompts. You can use it to: 1 2 Create a testplan. You simply name the file (giving it the .pln extension) and its directory. Create a test frame, which contains descriptions of the GUI objects in your application that you want to test.
Users Guide
19
As prompted, you simply open your application and open the various windows and dialogs that you want to test in the application. The wizard automatically records all the declarations in a file called frame.inc. You dont have to do any coding. 3 Record testcases. You name the testcase and provide a description for the testplan, then simply record the testcase. Again, you dont have to do any coding. The wizard automatically saves the testcase in a script (.t) file with the same name as the testplan. 4
Example
Run testcases.
The following scenario illustrates how you might get started testing the Text Editor application that ships with SilkTest. Since the Text Editor is a standard C application, you dont need to have any extensions (which provide support for testing applications created with thirdparty development environments) enabled before invoking the wizard. To learn how to disable support for a particular extension, look in the SilkTest online help for that extension. Now you are ready to use the QuickStart Wizard with the Text Editor. Procedure To use the wizard: 1 Invoke the wizard by selecting File/New and clicking the QuickStart Wizard icon.
The wizard opens.
20
Users Guide
Now you will name a new testplan, which will organize and manage your tests. 2 Click Next. The following panel displays:
Name the file edit.pln and click Next. Tip You can click Browse to look through the contents of your disks to locate or name the testplan. The next step is to record the test frame, which defines all the windows, dialogs, menus, and so on that you want to test.
Users Guide
21
To create a new test frame, leave New Test Frame selected and click Next. At this point, the wizard lists all the open (running and not minimized) applications. If Text Editor is not open, you can open it now (it is in the directory where you installed SilkTest). After you open the Text Editor, click on the QuickStart Wizard title bar to see Text Editor added to the list of applications.
5 6 7
Select Text Editor and click Next. The Capture Windows panel displays, describing the procedure. Click Next.
22
Users Guide
Now you simply open a document window and open all the dialogs that you want to test in the Text Editor. When you place the mouse pointer on a window or dialog, the wizard records all the declarations that SilkTest needs in a file called frame.inc in the same directory as your testplan. For example, when you capture the Find dialog, you see this:
When you have finished capturing the windows and dialogs in Text Editor, click Return to Wizard in the Capturing New Windows dialog. Now that you have created your test frame, you are ready to create a testcase.
10 Click Next twice. You see the following panel.
11 Name the test FindBox and enter the description Verify controls in Find dialog. Click Next. Your test is now being recorded, as indicated by the Record Status window on your screen.
Users Guide
23
12 Now go to Text Editor, select Search/Find to open the Find dialog, place your mouse pointer over the dialogs title bar, and press Ctrl+Alt to verify its state. The Verify Window dialog displays. Click OK to verify all properties for the dialog. Close the Find dialog (to return to your base state), then click Done in the Record Status window. You return to the Wizard and are asked to confirm that the test is what you want. 13 Click Next. 14 Run the test by clicking the Run Test Button. 15 The wizard reports the results. You can move the wizard to the side and look at the results file that is created whenever you run a test. 16 In the wizard, click Next to save your testcase. The testcase is saved in a script (.t) file with the same name as the testplan (in this case, edit.t). 17 Click Close to close the wizard. You see a window containing the results file from the test you just ran. In another window is the testplan:
What has been recorded in the testplan
SilkTest has recorded all the needed information for you in the testplan: The first line records the script file containing the testcase. The second section (under Header:) contains information that the wizard uses to find files. Click the plus sign to the left of header to see the information. Warning The wizard maintains the information in the header section. You should not make changes in this section. The last section lists the testcase you just recorded.
Using the wizard from a testplan
Notice the five large buttons at the top of the testplan. Those are wizard buttons. You can click them anytime to have the wizard step you through one of the testing processes.
24
Users Guide
Disable Wizard Run one testcase Run all testcases in plan Record a testcase Identify new windows
Whenever you invoke a wizard function by clicking one of its buttons, the wizard tells SilkTest which files to use for this particular testplan (it does this by automatically populating the Runtime Options dialog, which is described in Options/Runtime... on page 482). Clicking Disable Wizard removes the buttons from the testplan. You can reactivate the wizard from the Options menu.
How the wizard sets application states
When testing Web applications, the wizard sets the application state to DefaultBaseState, thus activating the recovery system. When testing nonWeb applications, the wizard sets the application state to none, thus deactivating the recovery system. For more information about application states and the recovery system, see The built-in recovery system on page 27. Once you have recorded one or more testcases using the wizard, you will probably want to go to your testplan and use the outline editor to manage it. Also, you might eventually want to use the outline editor to edit your testcases to make them more powerful and general. Before exploring the outline editor and the other tools, you should read The anatomy of a basic testcase on page 26 to get a good conceptual understanding of testcases.
Where to go next
Getting started testing Web applications

If you are using SilkTest to test Web applications, the instructions in SilkTests Getting Started: A Tutorial can get you started using a simulated business application on the Web.
Users Guide
25
1 OVERVIEW The anatomy of a basic testcase
The anatomy of a basic testcase

The following figure shows the organization of a typical automated testcase. By convention, you place a group of testcases for a particular application feature into a file called a script.
Name of testcase Testcase keyword Object-oriented commands
Testcase keyword Object-oriented commands
The testcase keyword Each automated test begins with the testcase keyword. If you have worked with computer languages before, you can consider a testcase a special kind of function, one with built-in error recovery and results reporting capabilities. The testcase name Immediately after the testcase keyword is the name of the testcase. The testcase is a name of your choice that indicates the testing task being performed. Object-oriented commands The core of a testcase are the object-oriented 4Test commands that drive, verify, and clean up your application. For example, consider this command from the preceding figure:
TextEditor.File.New.Pick ()
The first part of the command, TextEditor.File.New, is the name of a GUI object. The last part of the command, Pick, is the operation to perform on the GUI object. The dot operator (.) delimits each piece of the command. When this command is executed at runtime, it picks the New menu item from the File Menu of the Text Editor application.
26
Users Guide
1 OVERVIEW SilkTest architecture
The built-in recovery system

SilkTest maximizes your productivity by providing a completely integrated recovery system, which makes it possible for you to run your tests unattended. When the application you are testing fails, or even crashes, the recovery system restores the application to its base state so that the rest of your tests can continue to run. The recovery system can restore your application to its base state at any point during testcase execution: Before the first line of your testcase begins running, the recovery system restores the application to the base state even if an unexpected event corrupted the application between testcases. During a testcase, if an application error occurs, the recovery system terminates the execution of the testcase, writes a message in the error log, and restores the application to the base state before running the next testcase. After the testcase completes, if the testcase was not able to clean up after itself (for example, it could not close a dialog it opened), the recovery system restores the application to the base state. Note For more information, see Chapter 15, Understanding the Recovery System.
SilkTest architecture
Normal use of an application consists of a person manipulating a keyboard and mouse to initiate application operations. The person is said to be interacting with the GUI (Graphical User Interface). During SilkTest testing, SilkTest interacts with the GUI to submit operations to the application automatically. Thus SilkTest can simulate the actions of a person who is exercising all the capabilities of an application and verifying the results of each operation. The simulated user (SilkTest) is said to be driving the application. The application under test reacts to the simulated user exactly as it would react to a human user. SilkTest consists of two distinct software components that execute in separate processes: The SilkTest host software The 4Test Agent software
Users Guide
27
1 OVERVIEW SilkTest architecture SilkTest host software
The SilkTest host software is the program you use to develop, edit, compile, run, and debug your 4Test scripts and testplans. This manual refers to the system that runs this program as the host machine or the SilkTest machine. The 4Test Agent is the software process that translates the commands in your 4Test scripts into GUI-specific commands. In other words, it is the Agent that actually drives and monitors the application you are testing. One Agent can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. This manual refers to the systems that run remote Agents as target machines. In a client/server environment, SilkTest drives the client application by means of an Agent process running on each applications machine. The application then drives the server just as it always does. SilkTest is also capable of driving the GUI belonging to a server or of directly driving a server database by running scripts that submit SQL statements to the database. These methods of directly manipulating the server application are intended to support testing in which the client application drives the server. Note For more information on client/server testing, see Part V, Testing Client/Server Applications.
The Agent
The Agent menu
If the Agent has been started, an Agent button appears in the taskbar. Press the right mouse button over the Agent button to display the Agent menu. The menu contains the standard menu items for a Window, plus the following:
Menu item Description
Extensions
Provides information about the enabled extensions. Has two submenus: DetailsSelect to display a dialog listing all loaded extensions and their status. Status is listed as: Loaded if the extension is enabled, running and loaded in the application under test Enabled if the extension is enabled and running, but not loaded in the application under test An error message if an error has occurred Click the Refresh button in the dialog to update the information. UnloadSelect to unload an extension that is currently loaded in the application under test.
28
Users Guide
1 OVERVIEW The 4Test language
Menu item
Description
Network
Displays the Network dialog, which provides information about the client/server (distributed) testing you are doing: NetworkNetwork protocol being used Agent nameThe name of the remote Agent (if you are communicating with one remote Agent only) For more information about client/server testing, see Part V, Testing Client/Server Applications.
About
The version number of the Agent.
The 4Test language

4Test is an object-oriented fourth-generation language (4GL) designed specifically with the needs of the QA professional in mind. 4Tests powerful features are organized into three basic kinds of functionality: A robust library of object-oriented classes and methods that specify how a testcase can interact with an applications GUI objects. A set of statements, operators, and data types that you use to add structure and logic to a recorded testcase. A library of built-in functions for performing common support tasks. Note This section provides a high-level look at 4Test. For complete information, see the online Help.
Object-oriented features
Classes and methods
Classes are the core of object-oriented languages. For each kind of GUI object, there is an associated class that defines the actions, called methods, that can be performed on all objects of that type. For example, the PushButton class defines the methods that can be performed on all the pushbuttons in your application. Classes are organized in a hierarchy. The reason for this is that classes are related to each other, sharing characteristics common to their parent classes. For example, the parent class Control defines all the characteristics common to all kinds of controls: check boxes, text fields, pushbuttons, and so on. In
Inheritance
Users Guide
29
this way, each class does not need to define all the methods it needs; the class can just inherit the existing definition from its ancestor classes. For example, one action you can perform on objects of class DialogBox is the GetDefaultButton method. This method returns the identifier of the default pushbutton, which is the pushbutton that is pressed when the Return or Enter key is pressed. You could not use this method with objects that were not dialog boxes, such as menus or text fields. When you record testcases, the proper 4Test methods for each of your manual actions are recorded automatically for you, so it really is transparent to you which class the recorded methods belong to. However, if you decide to write or augment a testcase by hand, you can look up the class and the methods it supports in the online Help or Library Browser.
The class hierarchy
The following illustration shows the hierarchy of the core classes of GUI objects recognized by SilkTest.
30
Users Guide

AgentClass AnyWin ClipboardClass
Menu MoveableWin AppleMenu HelpMenu MenuItem PopupMenu PopupStart SysMenu ChildWin DialogMainWin MessageBox-
CursorClass
TaskbarWin
BrowserChild CustomWin GuptaTable HtmlTable Control DesktopWin HtmlColumn CheckBox ComboBox ControlMultiWin DynamicTex Header HtmlHeading HtmlImage HtmlLink HtmlList HtmlText ListBox ListView PageList PopupList PushButton RadioButton RadioList Scale ScrollBar StaticText StatusBar Table TextField ToolBar TreeView UpDown HtmlTextField VerticalScrollBar HorizontalScrollBar HtmlPopupList HtmlPushButton HtmlRadioButton HtmlRadioList HtmlListBox HtmlCheckBox HtmlComboBox
For information about all these classes, see the online Help. Note Extensions to SilkTest provide additional classes that are specific to particular development environments. For example, the Java extension includes classes that are specific to GUI objects in Java applications. For information about these classes, see the online Help for the extension (assuming you have installed the extension).
Users Guide
31
1 OVERVIEW The 4Test language Properties
A property is a characteristic of an object that you can access directly. You typically verify a GUI objects properties in a testcase. You will learn more about that in Chapter 5, Designing and Recording Testcases.
Statements
By using 4Test flow-of-control statements, you can add logic and robustness to a recorded testcase. The following table summarizes the statements.
To Use one of these 4Test statements
Execute statement for each executes a statement block once for each blocks more than once element in a list. for executes the loop once for each increment of a counter. while executes a loop until a test condition (boolean expression) is false. Conditionally execute if...else executes a statement block based on the value a statement block of a boolean expression. select executes one case from a group of cases. switch executes one of the statements that follow, depending on the value of an expression. Handle exceptions do...except handles an exception (error) rather than having it halt the script. raise raises a user-defined exception. reraise reraises an exception the testcase is handling itself within a do...except statement. break transfers control of the script out of the innermost nested for, for each, while, switch, or select statement. continue begins the next iteration of a for, for each, or while statement without completing the current iteration. exit ends the execution the current script. goto transfers control to the statement prefixed with the specified label. return returns control back to the calling function, optionally passing back a return value.
Transfer flow of control
For complete information about all 4Test statements, see the online Help.
32
Users Guide
Example The following 4Test code shows how to use the if statement to conditionally execute a method. The code picks the Close menu item from the File menu of the Text Editor application, which causes a message box to appear if there is unsaved work. The if statement tests for the existence of the message box.
TextEditor.File.Close.Pick () if MessageBox.Exists () // if the message box exists MessageBox.No.Click () // then dismiss it
Data types and variables

Built-in data types
4Test provides the following built-in data types: ANYTYPELIST ARRAY BOOLEAN BROWSERTYPE DATACLASS DATATYPE DATE DATETIME FONTSTYLE GUITYPE HANDLE INTEGER LIST LONG NUMBER REAL SEMAPHORE SET STRING TABLECOL TABLEROW TIME WINDOW
For complete information about all 4Test data types, see the online Help.
C data types for DLL functions
In addition to the 4Test data types, the following C data types are supported for use in calling functions in DLLs. char int short long unsigned char unsigned int unsigned short unsigned long float double
The first two columns above show data types that correspond to the 4Test INTEGER data type. The third column corresponds to the REAL data type.
User-defined types
You can also create new data types, including enumerated types and records. For example:
type FILE is LIST OF STRING type COLOR is enum red green
Adding variables to a testcase
In the following testcase a record data type is defined that contains each of the data values the testcase needs. The methods in the testcase process fields from the record instead of literal data values.
Users Guide
33
1 OVERVIEW The basic tools type SEARCHINFO is record STRING sText // Text to enter in document window STRING sPos // The starting position of search STRING sPattern // The string to search for BOOLEAN bCase // Case-sensitive or not STRING sDirection // The direction of the search STRING sExpected // The string you expect to find testcase Find (SEARCHINFO Data) TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern) Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () Find.Cancel.Click () DocumentWindow.Document.VerifySelText ({Data.sExpected}) TextEditor.File.Close.Pick () MessageBox.No.Click ()
Built-in functions
4Test contains a function library to handle the most common programming tasks. The following table summarizes the functions by category: Application state Array manipulation Char/string conversion Data type manipulation Distributed processing Exception handling File manipulation List manipulation Numeric operations Random values Results file operations Script information Semaphore operations Set manipulation Startup String manipulation System calls Timers, time/date info Window information
For complete information about all 4Test functions, see the online Help.
The basic tools

You can use the following powerful SilkTest tools to manage, execute, and interpret your tests, including: The outline editor The results processor The debugger
34
Users Guide
1 OVERVIEW The basic tools
The outline editor

You interact with testplans, testcases, and results files in the outline editor, an easy-to-use interactive environment that is consistent throughout all phases of testing. Because these files contain large amounts of information, a structured, hierarchical outline provides an ideal organizational model. For example, the following figure shows a testcase in the outline editor:
Expanded testcase
Notice how the statements that make up the body of the expanded testcase are indented. The outline editor uses indentation to differentiate levels. A minus icon ( ) indicates that a level is fully expanded. Similarly, a plus icon ( ) indicates that a level is collapsed and has hidden detail. Click on the icons to expand and collapse the outline, to show the desired level of detail.
Outlining commands
As you work in the outline editor, you can use menu, keyboard, or tool bar commands to change indentation levels. The following table summarizes the commands:
Action Menu Item Keystroke Icon
Indent one level Outdent one level Swap with line above Swap with line below
Line continuation character Using an ASCII text editor
Outline/Move Right Outline/Move Left Outline/Transpose Up Outline/Transpose Down
Alt + Alt + Alt + Alt + None None
Use Shift+Enter to force a soft line break within a long line of code. The outline editor considers such lines as one statement. Typically, you write scripts and testplans using the built-in outline editor, which is optimized for managing scripts and testplans.
Users Guide
35
However, you can also use your favorite ASCII text editor to write and edit scripts and testplans, then open them in the outline editor for execution. You can even make changes in the outline editor, save the changes, then reopen the modified file in your ASCII editor for continued editing. Procedure To write and edit scripts and testplans in an ASCII editor: 1 Follow these rules when creating files in your ASCII editor: Begin each line with [ ] (left bracket, followed by a space, followed by a right bracket). Use tabbing to indent levels. If you want to use spaces, instead of tabs, to denote levels, include the following line at the top of the file:
[tn]
where n is the number of spaces that correspond to a tab. For example, if [t4] is the first line of a file, every 4 spaces will be converted into a tab. 2 Save the file using the appropriate file extension.
Type of file File extension
Testplan 4Test script 4Test include
.pln .t .inc, or whatever extensions are defined in the General Options dialog
You can use these scripts and testplans the same way you use files created in the outline editor: Select File/Open to open them. You can make and save changes in the outline editor, then reopen the file in your ASCII editor for further revision.
Using Microsoft Word
You can also use Microsoft Word to develop testplans. For more information, see Developing your outline in Microsoft Word on page 55.
The results processor

After a testcase runs, a results file is displayed, as shown in the following figure. Use the commands on the Results menu to process results.
36
Users Guide
Summary statistics
Collapse or expand results
If a testcase encountered an error, click on the plus icon ( ) preceding the test description to reveal a brief description of the error. To invoke comparison tools that let you see just how the results differ from the baselines, click on the box icon, as shown in this figure:
Click on box icon
For more information, see Results files on page 148.
The debugger
SilkTest comes with a powerful set of debugging tools to help you find problems with your scripts. Using these debugging tools, you can step through a script a line at a time and stop at specified breakpoints, as well as examine local and global variables and enter expressions to evaluate.
Users Guide
37
1 OVERVIEW Online information
The following figure shows a testcase in the debugger with a breakpoint set.
Icon in margin indicates line currently executing Icon in margin indicates breakpoint
For more information, see Chapter 7, Using the Debugger.
Online information
You also have the following sources of online information: The Library Browser Online Help Online Users Guide
The Library Browser

Select Help/Library Browser to invoke the Library Browser, which you can use to quickly look up information about the classes of objects in your test application. For example, if you forget which method you use to determine a dialog boxs default pushbutton, open the Library Browser and look it up. The Library Browser provides instant, online documentation for: Built-in 4Test methods and properties Methods and properties which are defined at your site Built-in functions
38
Users Guide
Looking at methods and properties
4Test classes have methods and properties. When you select the Methods or Properties tab in the Library Browser, you see a list of all the built-in and user-defined classes in hierarchical form. Procedure To see the methods or properties for a class: 1 2 Select the Methods or Properties tab. Select the class in the Classes list box. Double-click on a + box to expand the hierarchy. Double-click on a box to collapse the hierarchy. The methods or properties for the selected class are displayed. 3 By default, only those methods or properties that are defined by the class are displayed. To see all methods or properties that are available to the class (that is, methods or properties also defined by an ancestor of the class), select the Include Inherited check box. To see all methods or properties (even those not available to the selected class), select the Include All check box. Select a method or property. Information about the selected method or property is displayed.
Users Guide
39
Note If the Library Browser isnt displaying your user-defined objects, close the Library Browser, recompile the include files that contain your user-defined objects (Run/Compile), then reopen the Library Browser.
Looking at built-in functions
Procedure To see the built-in functions: 1 2 Select the Functions tab. Select the category of functions you want in the Groups list box. To see all built-in 4Test functions, select the Include All check box. Functions are listed in the Functions list box. 3 Select the function you are interested in. You see a description of the function as well as syntax and parameters.
Enhancing the contents of the Library Browser
You can add to the standard contents of the Library Browser to provide morecomplete online documentation of your user-defined methods, properties, and functions. For more information, see Chapter 18, Modifying the Library Browser.
Online Help
Select Help/Help Topics to invoke online Help, which provides a wealth of information, including the comprehensive reference for all 4Test language constructs. For information about what is in online Help, read the About This Help topic.
40
Users Guide
Online Users Guide

Requires Adobe Acrobat Reader with Search
To view the Online Users Guide, you first need to install Adobe Acrobat Reader. When you install SilkTest, you receive instructions for installing the correct Reader version. With Acrobat Reader, you can perform a number of functions using the menu bar, toolbar, bookmarks, and buttons on the bottom of the window. The toolbar also provides tooltips that identify the tool icons. For more information about how to use Acrobat Reader, see Adobes online documentation.
How to use the Online Users Guide
To view the SilkTest Users Guide online, from the SilkTest Program Group, select Documentation, and then SilkTest Users Guide. SilkTest starts the Acrobat Reader and the Reader displays an online version of the printed Users Guide on your desktop. Through Acrobat Reader you can view information, move from topic to topic, and print pages from the Online Users Guide. Browsing using Bookmarks Bookmarks are displayed automatically when you open the Online Users Guide. Bookmarks are hotspots that link to key topics. When you click on a bookmark, Acrobat Reader displays the page referenced by the link. Click on the symbol to the left of a bookmark to display its subtopics.Your cursor changes shape from to when it passes over a bookmark or any other hotspot. You can hide bookmarks by clicking the by pressing the button. button and restore them again
Browsing from Contents and Index You can click on the Contents and Index bookmarks to view the Table of Contents and Index as they appear in the printed book. Click on any contents or index topic to view the page that describes that topic. Print pages of interest Select File/Print from the Reader menu bar to print the current page, a range of pages, or the entire Online Users Guide.
Troubleshooting
If you select SilkTest/Documentation/Online Users Guide, but cannot access the book, go back to the SilkTest Setup to make sure you install both the online Users Guide and Acrobat Reader. If you are missing either component, you will not be able to bring up the online book.
Users Guide
41
1 OVERVIEW Where to go from here
Where to go from here

Part I, The Basics
This rest of this part contains the following chapters:

Chapter Page
Chapter 2, Creating Testplans How to work in the the testplan editor environment, use the testplan editor keywords, and structure a testplan. Chapter 3, Enabling Extensions for Applications Under Test When and why to enable extensions for applications on target and host machines. Chapter 4, Recording a Test Frame How to record, understand the structure of, and modify window declarations. Chapter 5, Designing and Recording Testcases How to design and record testcases that verify application correctness. Chapter 6, Running Tests and Interpreting Results How to run testcases and interpret test results. Chapter 7, Using the Debugger How to use the built-in debugger to uncover problems in your scripts and in your application.
47
67
77
101
141 171
42
Users Guide
1 OVERVIEW Where to go from here Part II, Enhancing Your Tests

Chapter Page
Chapter 8, Generalizing a Testcase How to add 4Test flow-of-control statements to testcases, create data-driven testcases, and record application states. Chapter 9, Handling Exceptions How to handle exceptions (errors) that are generated when you run your scripts. Chapter 10, Adding Data to a Testplan How to add test data to a testplan and pass it to data-driven testcases. Chapter 11, Categorizing and Marking Testplans How to define test characteristics, called attributes, and mark tests to differentiate them. Chapter 12, Working With Large Testplans How to navigate within a testplan and know whats in scope, generate completion reports, and structure a testplan as a master plan and subplans.
Part III, CrossPlatform Testing
185
209
221
235
255

Chapter Page
Chapter 13, Porting Tests to Other GUIs How to make your window declarations valid for any GUI, thereby minimizing changes to scripts when porting. Chapter 14, Supporting Internationalized Applications How to internationalize the tags, with a description of international keyboard support.
267
283
Users Guide
43
1 OVERVIEW Where to go from here Part IV, Customizing SilkTest

Chapter Page
Chapter 15, Understanding the Recovery System How to use and modify the built-in recovery system so that testcases can run unattended. Chapter 16, Extending the Class Hierarchy How to add new classes and methods. Chapter 17, Supporting Custom Objects How to map custom classes to 4Test classes, test graphical controls, and write methods for custom objects. Chapter 18, Modifying the Library Browser How to add information about your own custom classes and functions to the Library Browser
Part V, Testing Client/ Server Applications
289
303 317
327

Chapter Page
Chapter 19, Introduction to Client/Server Testing Introduces you to testing client/server applications and/or databases in a networked environment. Chapter 20, Configuring SilkTest for Client/Server Testing How to set up to test client/server applications. Chapter 21, Implementing Client/Server Testing How to test client/server applications. Chapter 22, Multi-Application Testing How to drive multiple different applications simultaneously.
337
351 361 379
44
Users Guide
1 OVERVIEW Where to go from here Part VI, GUI-Specific Tips and Tools

Chapter Page
Chapter 23, Using the Windows Bitmap Tool How to use the Windows bitmap tool to capture and compare bitmaps and to create and apply masks. Chapter 24, Calling Windows DLLs From 4Test Scripts How to declare and pass arguments to a DLL function, alias a DLL name, and use SilkTests DLL support files. Chapter 25, Using PVCS with SilkTest How to use PVCS to manage your SilkTest test files.
Part VII, Command Reference
401
423
429

Chapter Page
Chapter 26, Command Line Syntax The syntax of the partner command and its options. Chapter 27, Menu Commands An alphabetical description of each of the menu items in the SilkTest user interface, including those menus unique to the testplan editor.
Part VIII, Appendices
443 447

Chapter Page
Appendix A, Testplan Editor Keywords The syntax of the testplan editor statements. Appendix B, Using Drag-and-Drop How to record drag-and-drop operations. Appendix C, Supporting Owner-Draw List Boxes How to give SilkTest access to text in owner-draw list boxes. Appendix D, Useful Multitestcase Code Templates Two templates that you might find useful as a basis for your client/server testing. Appendix E, Glossary Glossary of terms used in SilkTest.
523 527 529
531
535
Users Guide
45
1 OVERVIEW Where to go from here
46
Users Guide
2e rh C t p a
Creating Testplans
If you are using the testplan editor, the first step in creating automated tests is to create a testplan. (If you are not using the testplan editor, the first step is creating a test frame, which is described in the next chapter.) A testplan consists of two distinct parts: an outline that describes the test requirements, and statements that connect the outline to the 4Test scripts and testcases that implement the test requirements.
What you will learn
This chapter contains the following sections:

Topic Page
Starting a new testplan Understanding the outline model Structuring an outline Using a template of the GUI hierarchy as an outline Linking the testplan to scripts and testcases Documenting manual tests in the testplan
47 48 50 58 59 65
Starting a new testplan

Procedure To start a new testplan: 1 Select File/New.
Users Guide
47
2 CREATING TESTPLANS Understanding the outline model
The New dialog opens.
Select Testplan and click OK. An empty testplan window opens. Note If you are using the testplan editor under Windows, you can also use the QuickStart Wizard to start a new testplan and begin the testing process. For more information, see Getting started on page 19.
Understanding the outline model

Because a testplan is made up of a large amount of information, a structured, hierarchical outline provides an ideal model for organizing and developing the details of the plan.
Sample outline for a word search feature
For example, consider the Find dialog from the Text Editor application, which allows a user to search in a document:
A user enters the character(s) to search for in the Find What text field, checks the Case Sensitive check box to consider case, and selects either the Up or Down radio button to indicate the direction of the search. Here is what a partial outline of the testing requirements might look like:
I. Find dialog A. Case-sensitive 1. Forward search a. Search for a character b. Search for a word 2. Backward search a. Search for a character
48
Users Guide
2 CREATING TESTPLANS Understanding the outline model b. Search for a B. Case-insensitive 1. Forward search a. Search for a b. Search for a 2. Backward search a. Search for a b. Search for a word
character word character word
This outline describes eight distinct tests: 1 2 3 4 5 6 7 8

Concepts and terminology
Find dialog, case sensitive, forward, character search Find dialog, case sensitive, forward, word search Find dialog, case sensitive, backward, character search Find dialog, case sensitive, backward, word search Find dialog, case insensitive, forward, character search Find dialog, case insensitive, forward, word search Find dialog, case insensitive, backward, character search Find dialog, case insensitive, backward, word search
Notice that the full description for each of these eight tests is formed by joining the descriptions along the branch that leads to the test. This can be seen more clearly when the outline is rewritten as a tree:
Find dialog
Case Sensitive
Case Insensitive
Forward
Backward
Forward
Backward
Word
Char
Word
Char
Word
Char
Word
Char
Test Descriptions Because the last, or terminal, point along a path completes a test description, each terminal point in an outline is referred to as a test description. When the testplan editor is interpreting your outlines, it knows that when it reaches a terminal point, there should be a test that can be executed. Test descriptions by default are displayed in blue in the outline editor.
Users Guide
49
2 CREATING TESTPLANS Structuring an outline
Group Descriptions The other descriptive lines in an outline are called group descriptions, because they describe a group of tests, not a single test. For example, in the sample plan shown above, the single descriptive line Find dialog describes a group of eight tests and the descriptive line Case Sensitive describes a group of four tests. Group descriptions by default are displayed in black. Scope In addition to test descriptions and group descriptions, a testplan also contains the statements that implement the test requirements. The scope of a statement refers to the portion of the testplan where the statement applies. A statement placed at the group description level applies to all the test descriptions contained by the group. Conversely, a statement placed at the test description level applies only to that test description. Levels in the testplan editor are represented by indentation, and you change the level of a line by using the Outline/Move Left and the Outline/Move Right commands, both of which have keyboard, tool bar, and popup menu shortcuts.
Benefits
There are four significant benefits to structuring a testplan as a hierarchical outline. An outline: Assists the testplan author in developing thoughts about the test problem, by promoting and supporting a top-down approach to test planning Yields a comprehensive inventory of test requirements, from the most general, through finer and finer levels of detail, to the most specific Allows the statements that actually implement the tests to be shared by group descriptions or used by just a single test description Provides reviewers with a framework for evaluating the thoroughness of the plan and for following the logic of the testplan author
Structuring an outline
Because there are many ways to organize information, you can structure an outline using as few or as many levels of detail as you feel are necessary. To demonstrate the variety of organizational options, this section presents a series of sample plans, each of which adds successive levels of description to the outline. Note that for completeness, each of the plans also shows the script and testcase statements that link the descriptions to the 4Test scripts and testcases that implement the test requirements, though how to establish these links is not described until Linking the testplan to scripts and testcases on page 59.
50
Users Guide
2 CREATING TESTPLANS Structuring an outline Using a list structure
At its simplest, an outline is a hierarchy with just a single level of detail. In other words, it is a list of test descriptions, with no group descriptions. For example:
Test descriptions
the testplan editor
As the figure shows, each test is fully described by a single line, which is followed by the script and testcase that implement the test. You may find this style of plan useful in the beginning stages of testplan design, when you are brainstorming the list of test requirements, without regard for the way in which the test requirements are related. It is also useful if you are creating an ad hoc testplan that runs a set of unrelated 4Test scripts and testcases.
Using a hierarchical structure
The following testplan has a single level of group description, preceding the level that contains each of the test descriptions. The group description indicates that all the tests are for the Find dialog.
Users Guide
51
Group description
Test descriptions
the testplan editor
As the figure shows, the testplan editor indicates levels in the outline with indentation. Each successive level is indented one level to the right. The minus icons indicate that each of the levels is fully expanded. By clicking on the minus icon at any level, you collapse the branch below that level. For example, here is how the preceding plan looks after clicking on the minus icon at the Find dialog group description level:
When working with large testplans, collapsing and expanding testplan detail makes it easy to see as much or as little of the testplan as you need. The next testplan adds a second level of group description which indicates whether or not the tests in the group are case sensitive.
52
Users Guide
Group descriptions for case sensitivity
The final testplan in the series adds a third level of group descriptions which indicate whether the tests in the group search in the forward or backward direction.
As each of these sample testplans shows, the goal when writing testplans is to create a top-down outline that describes all of the test requirements, from the most general (as in Find dialog) to the most specific (as in Find dialog, casesensitive, forward, character search).
Users Guide
53
2 CREATING TESTPLANS Structuring an outline How to indent and change levels
To enter the group and test descriptions that constitute the outline, you use menu, keyboard, or toolbar commands to change the level as you are typing the descriptions. The following table summarizes the commands:
Action Menu Item Key Tool
Indent one level Outdent one level Swap with line above Swap with line below
Outline/Move Right Outline/Move Left Outline/Transpose Up Outline/Transpose Down
ALT + ALT + ALT + ALT + None None
Each command acts on the current line or currently selected lines.

Using color to indicate structure
You can also customize the outline so that different testplan components display in a unique color. Here are the default colors:
Component Default color
Test description Testplan statement Open subplan file marker Comment Other line (group description)
Blue Dark red Magenta Green Black
To change any of these, select Options/Editor Colors to invoke the Editor Colors dialog.
In the Editor Item list box at the left of the dialog, select the outline editor item to change. Then apply a color to the item by selecting a pushbutton from the list of predefined colors.
54
Users Guide
You can create a new color to apply by selecting the red, green, and blue values that compose the color, as shown in the preceding illustration.
Adding comments that display in the results
You can add comments to your testplans that will display in the results when you run your tests. Comments are a handy way to annotate your tests to make it easier to interpret your results. To add a comment, include the following statement in your plan:
comment: Your comment text
For example, running the following piece of a testplan:

Find dialog Get the default button comment: This test should return Find.FindNext script: find.t testcase: GetButton
produces the following in the results file:

Find dialog Get the default button Find.FindNext comment: This test should return Find.FindNext
For more information about running tests, see Chapter 6, Running Tests and Interpreting Results. Note You can also preface lines in all 4Test files with // to indicate a single-line comment; such comments do not display in testplan results.
Developing your outline in Microsoft Word

You can develop your testplan as an outline in Microsoft Word, then convert the Word document into a testplan.
Writing the outline
Procedure To develop your testplan in Word: 1 2 Use Words outlining feature to develop your testplan. Use Outline View and use only the styles Heading 1 through Heading 9. Use indentation to reflect the levels in the testplan.
For example, your testplan might look like this:
Users Guide
55
Setting up for the conversions
Once you have a testplan as a Word outline, you can convert it into a testplan. To do this, you use a Word macro that is provided in a template that is shipped with SilkTest. First, you use Words Macro Organizer to make the macro available to all your Word documents. Procedure To make the macro available to your documents: 1 In Word, open qapmacro.dot, which is in the directory where you installed SilkTest. The file qapmacro.dot is a template that contains the macro that converts a Word outline into a testplan. 2 3 Select Tools/Macro. Then Macros.... Word displays the Macros dialog. Click Organizer... . Word displays the Organizer dialog.
56
Users Guide
4 5 6
Converting an outline into a testplan
Click the Copy button to copy the WordToQAP macro to your global template (normal.dot). Close the dialog. The WordToQAP macro is now available to all your documents. Close qapmacro.dot. You dont need this file anymore since the macro has been copied into your normal.dot file.
Procedure To convert a Word outline into a testplan: 1 2 3 4 Open your outline in Word. Select Tools/Macro. Select WordToQAP and click Run. The macros Testplan Filename dialog appears.
Specify the full pathname of the testplan file you want to create. You must include the.pln extension. (Dont press Enter to dismiss the dialog. If you do, you will see an error dialog saying that it could not create a file.) Click OK. The macro converts the Word outline into a testplan. You see a confirmation dialog.
6 7
You can now open the testplan and continue working on it.
Users Guide 57
2 CREATING TESTPLANS Using a template of the GUI hierarchy as an outline
(Note that the conversion is one direction only. You cannot use the macro to take a testplan and convert it into a Word document.)
Characters that do not convert
A small number of characters may not convert properly. If this occurs you will have to make the corrections yourself by editing the testplan after the conversion. For example, the quote (") character may not convert properly in certain fonts.
Using a template of the GUI hierarchy as an outline

Because a testplan is initially empty, you may want to insert a template, which is a hierarchical outline you can use as a guide when you create a new testplan. The template contains placeholders for each GUI object in your application. Although you may not want to structure the testplan in a way which mirrors the hierarchy of your applications GUI, this can be a good starting point if you are new to creating testplans. Note In order to be able to insert a template, you must first record a test frame, which contains declarations for each of the GUI objects in your application. See Chapter 4, Recording a Test Frame. Procedure To insert a template: 1 Select Testplan/Insert Template. The Insert Testplan Template dialog appears, which contains a multiselect list box that lists all the GUI objects declared in your test frame.
Select each of the GUI objects that are related to the application features you want to test (because this is a multi-select list box, the objects do not have to be contiguous). For each selected object, two lines of descriptive text are inserted into the testplan.
58
Users Guide
2 CREATING TESTPLANS Linking the testplan to scripts and testcases
For example, here is the template the testplan editor inserts for the Find dialog of the Text Editor application:
Tests for DialogBox Find Tests for StaticText FindWhatText (Insert tests here) Tests for TextField FindWhat (Insert tests here) Tests for CheckBox CaseSensitive (Insert tests here) Tests for StaticText DirectionText (Insert tests here) Tests for PushButton FindNext (Insert tests here) Tests for PushButton Cancel (Insert tests here) Tests for RadioList Direction (Insert tests here)
Linking the testplan to scripts and testcases

Once you have a descriptive outline of the test requirements, you can associate the outline with the 4Test scripts and testcases that implement the requirements. You create this association by inserting script and testcase statements in the appropriate locations in the outline.
Where to insert script and testcase statements
You can insert a script and testcase statement for each test description, although placing a statement at the group level when possible eliminates redundancy in the testplan. For example, since it is usually good practice to place all the testcases for a given application feature into a single script file, you can reduce the redundancy in the testplan by specifying the script statement at the group level that describes that feature. The following testplan illustrates this, specifying the script find.t once for the entire group of tests of the Find dialog (the script statement is highlighted for emphasis):
Users Guide
59
You can also insert a testcase statement at the group level, although doing so is only appropriate when the testcase is data-driven, meaning that it receives test data from the plan (otherwise the same testcase would be called several times with no difference in outcome). See Using data-driven testcases on page 190 and Chapter 10, Adding Data to a Testplan for more details.
Three ways to link a testplan with a script or testcase
There are three ways to associate a testplan with a 4Test script and testcase: Use the Testplan Detail dialog to automate the process Enter the script and testcase statements manually Record the testcase from within the testplan
The benefits and details of the first two methods are described in the following sections. The third method is described in Recording from within a testplan on page 136.
How to link using the Testplan Detail dialog
The Testplan Detail dialog automates the process of linking to scripts and testcases. It lets you browse directories and select script and testcase names, and it enters the correct syntax into the plan for you. Procedure To use the Testplan Detail dialog to link a test description or a group description to a script or testcase: 1 Place the insertion cursor on either a test description or a group description. For example, the following figure shows how to position the cursor to link the first test description:
60
Users Guide
Select Testplan/Detail. The testplan editor invokes the Testplan Detail dialog, with the Test Execution tab showing:
The multi-line list box at the top of the dialog displays the line in the testplan that the cursor was on when the dialog was invoked, as well as its ancestor lines. The black arrow icon indicates the current line. The current line appears in black and white, and the preceding lines appear in blue. 3 4 If you know the names of the script and testcase, enter them in the Script and Testcase fields, respectively, and skip to step 8. If you are unsure of the script name, click the Scripts pushbutton to the right of the Script field to browse for the script file. the testplan editor invokes the Testplan Detail - Script dialog:
Users Guide
61
Navigate to the appropriate directory and select a script name by doubleclicking on it or by selecting it and clicking the OK pushbutton. The Testplan Detail Script dialog closes and the script name is entered in the Script field. For example:
Click the Testcase pushbutton to the right of the Testcase field, to browse for the testcase name: the testplan editor invokes the Testplan Detail - Testcase dialog:
This dialog shows the names of the testcases that are contained in the selected script. Testcases are listed alphabetically, not in the order in which they occur in the script.
62
Users Guide
Select a testcase from the list, by double-clicking on it or by highlighting it and clicking the OK pushbutton. The Testplan Detail Testcase dialog closes and the testcase name is entered in the Testcase field. For example:
Click OK. The script and testcase statements are entered in the plan. For example:
You are done.

How to link manually
If you feel comfortable with the syntax of the testplan editor statements and know the locations of the appropriate script and testcase, you can enter the script and testcase statements manually. Procedure To manually link a description to a script or testcase: 1 Place the insertion cursor at the end of a test or group description, press the Enter key to open up a new line, and indent the new line one level.
Users Guide
63
For example, to insert the cursor correctly to link the group description Find dialog:
Enter the script and/or testcase statements using the following syntax, substituting actual names for myscript and MyTestcase:
script: myscript.t testcase: MyTestcase
For example, to link to the find.t script:
If you enter a statement correctly, it appears in red, the default color used for statements. If not, it will either appear in blue, indicating the line is being interpreted as a test description, or black, indicating it is being interpreted as a group description.
64
Users Guide
2 CREATING TESTPLANS Documenting manual tests in the testplan
Documenting manual tests in the testplan

Your QA department might do some of its testing manually. You can document the manual testing in the testplan. In this way, the planning, organization, and reporting of all your testing can be centralized in one place. To indicate that a test description in the testplan is implemented with a manual test, use the value manual in the testcase statement, as in:
testcase: manual
For example, the following testplan indicates that the case-insensitive tests are manual:
Tracking manual tests
You can describe the state of each of your manual tests in a dialog in SilkTest. This information is used in reports. Procedure To describe the state of a manual test: 1 2 Open a testplan containing manual tests. Select Testplan/Manual Tests. The Update Manual Tests dialog is displayed. It lists all manual tests in the current testplan.
Users Guide
65
2 CREATING TESTPLANS Documenting manual tests in the testplan
Select a manual test and document it.

To Do this
Mark the test complete
Select the Complete radio button. Complete means that a test has been defined. A manual test marked here as Complete will be tabulated as complete in Completion reports. Select the Has Been Run radio button, select Passed or Failed, specify when the test was run, and, optionally, specify the machine. To specify when the test was run, use this syntax: YYYY-MM-DD HH:MM:SS (Hours, minutes, and seconds are optional.) For example, enter 1996-10-01 to indicate the test was run Oct 1, 1996. Manual tests marked as Passed or Failed will be tabulated as such in Pass/Fail reports, as long as you have also specified the time they were run. (A test marked Has Been Run is also considered complete in Completion reports.)
Indicate whether the test passed or failed
Add any comments you want about the test
Fill in the Comments text field.
For more information about Pass/Fail reports, see Generating a testplan Pass/Fail report on page 162. For more information about Completion reports, see Measuring testplan completion on page 258.
66
Users Guide
3e rh C t p a
Enabling Extensions for Applications Under Test

SilkTest provides extensions for testing applications that use non-standard controls in specific development and browser environments. If you plan to test applications in environments that require extensions, you might need to enable the appropriate extensions for each application before you begin testing. This chapter presents an overview of why and when to enable extensions. To learn the procedures for enabling or disabling a specific extension, please refer to the online Help for that extension.
Introduction
What you will learn

Topic Page
How extensions work with applications When to enable extensions General process for enabling extensions
67 68 71
How extensions work with applications

SilkTest consists of two components: Host software that runs on your local host machine One or more 4Test Agents that can run locally on the host machine or on remote target machines in a networked environment
Users Guide
67
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST When to enable extensions
If you plan to use extensions for testing in specific environments, you must tell SilkTest which extensions must be loaded for each application under testregardless of whether the application will run locally or on remote machines. You do this by making sure extensions are enabled on your host machine and on each target machine before you record or run tests. Once extensions are enabled for applications under test, SilkTest can send each Agent the list of extensions to be loaded. Then, the Agent dynamically loads the appropriate extensions for testing each application.
When to enable extensions

Before you can begin testing, you must enable extensions for applications you plan to test on both the target machine and the host machine.
On the target machine

On each target machine, we recommend that you enable the extensions that meet the requirements of all applications you ever intend to test on that machine, in current and future test sessions.
Extensions enabled by default
By default, the installation program enables the following extensions on the target machine: Extensions that come with SilkTest: All supported Web browser extensions Java extension
Extensions for the development environments you explicitly install. These extensions are enabled for applications that you invoke in the extensions standard runtime environments. For example, if you install all development environments with SilkTest, the installation program automatically enables the Blue Express extension, for testing AS/400 and mainframe applications that are invoked using the Blue Express runtime environment
Extensions you must enable manually
You must enable extensions manually for applications that do not run in the extensions standard runtime environments, as in these situations. You developed applications inJava or Blue Express as stand-alone applications, which you invoke outside their standard runtime environments. You explicitly change the executable name of a , Java application.
68
Users Guide

The Extension Enabler
Your application uses controls that require additional extensions. You are testing Visual Basic applications, which are not invoked in standard runtime environments.
The Extension Enabler is the utility that allows you to enable or disable extensions on your target machines. All information that you enter in the Extension Enabler is stored in the file extend.ini and allows the Agent to recognize the non-standard controls you want to test on target machines. If the default settings satisfy your testing requirements, you do not need to run the Extension Enabler. Otherwise, you need to invoke this utility to modify your settingseven if you are testing locally (that is, your target and host are the same machine). If you will run your applications on a local or remote Windows target machine, you should invoke the Extension Enabler from the Windows Start menu by selecting Start/Programs/<Silk program folder>/Extension Enabler. To invoke the Extension Enabler on a remote non-Windows target machine, run extinst.exe, located in the directory on the target machine where you installed the SilkTest Agent. For more information about how to use the Extension Enabler, see the online Help for the extensions you wish to enable for testing non-standard controls in your application.
On the host machine

On the host machine, we recommend that you enable only the extensions required for testing the current application, also referred to as the application under test (AUT). Extensions for all other applications should be disabled on the host to conserve memory and other system resources.
Users Guide
69
By default, the installation program:

Extensions dialog
Enables the extension for your default Web browser environment on the host machine (if you selected a default browser). Disables extensions on the host machine for all other browser environments.
On the host machine, you turn extensions on and off using the Extensions dialog. All information you enter in the Extensions dialog is stored in the file partner.ini and allows SilkTest to recognize the non-standard controls you want to test. To invoke the Extensions dialog, select Options/Extensions from the SilkTest menu bar on the host machine. For more information about how to use the Extensions dialog, see the online Help for the extensions you wish to enable for testing non-Windows controls in your application.
70
Users Guide
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST General process for enabling extensions
General process for enabling extensions

The following diagram presents the general process to follow for enabling and disabling extensions in network and local testing environments. Note The procedures for enabling particular extensions vary according to the extension. For the steps required to enable or disable each extension, consult the online Help for that extension.
Example of Enabling Extensions
The following section provides an example of the steps you go through in the enabling extensions; the section shows you how to enable browser extensions.
Users Guide
71
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST Enabling browser extensions
Enabling browser extensions

This section shows you how to enable support for Web browser testing by enabling browser extensions on your target and host machine.
Enabling browser support on a target machine
By default, SilkTest enables all the browser extensions on your target machine during the installation procedure. To change the default settings or verify your current settings, you use a utility called the Extension Enabler. Note If you are running local teststhat is, your target and host are the same machineyou still must make sure that both dialogs are configured properly. Procedure To enable support for browsers on a target machine: 1 From the SilkTest program group, choose Extension Enabler. The Extension Enabler dialog appears, as in this example:
2 3
Enable support for the browsers you want to use, if they arent already enabled. In the Primary Extension field for a browser, choose Enabled. Select any other extensions that the browser needs for testing, such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Note If you are testing Internet Explorer 4.0 or 5.0, check the ActiveX check box.
72
Users Guide
Tip To get information about the files used by an extension, select an extension and click Details. 4 Click OK to close the Extension Enabler dialog. Note If you enable support for ActiveX in this dialog, make sure that it is enabled in the Extensions dialog as well.
Enabling browser support on a host machine
You enable support for browsers on the host machine using the Extensions dialog. Note There is overhead to having more than one browser extension enabled, so you should enable only one browser extension unless you are actually testing more than one browser in an automated session. Procedure To enable support for browsers on a host machine: 1 2 Start SilkTest. Select Options/Extensions. SilkTest displays the Extensions dialog, which lists all the installed extensions, including the browser extensions.
Users Guide
73
Tip To get information about the files used by an extension, select an extension and click Details. 3 Click the browser you want to enable. Then in the Primary Extension field, choose Enabled. Note If you are testing Internet Explorer 4.0 or 5.0, check the ActiveX check box. If you enable support for ActiveX in this dialog, make sure that it is enabled in the Extension Enabler dialog as well. 4 Select any other extensions that the browser needs for testing, such as Active X or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Disable other browser extensions that you will not be using. In the Primary Extension field for a browser, choose Disabled. Disable all non-Web extensions. In the Primary Extension field for each non-Web extension, choose Disabled. Note To disable a Visual Basic extension, uncheck the ActiveX check box for the Visual Basic application. 7 Click OK to close the Extensions dialog.
5 6
74
Users Guide
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST Disabling browser extensions
Disabling browser extensions

If you are testing non-Web applications, you must disable browser extensions on your host machine. This is because the recovery system works differently when testing Web applications than when testing non-Web applications. For more information about the recovery system for testing Web applications, see Testing Web Applications with SilkTest. Procedure To disable support for browsers on a host machine: 1 In SilkTest, select Options/Extensions. SilkTest displays the Extensions dialog, which lists all the installed extensions, including browser extensions. 2 For each browser: 3 In the Primary Extension field , choose Disabled. In the Other Extensions field, uncheck any marked checkboxes.
Click OK.
Users Guide
75
3 ENABLING EXTENSIONS FOR APPLICATIONS UNDER TEST Disabling browser extensions
76
Users Guide
4e rh C t p a
Recording a Test Frame

Before you begin recording testcases, you first record a test frame for your application. The test frame is the backbone that supports your testcases and scripts. It is a file that contains all the information about your applications GUI objects that SilkTest needs when you record testcases. This information minimally consists of a declaration for each GUI object, but can also include any data that you want to associate with each GUI object, as well as any new classes and methods that you want to define. This chapter contains the following sections:
Topic Page
Introduction
What you will learn
Why window declarations make your tests robust Preparing to record window declarations How to record window declarations Window declaration syntax and structure Modifying identifiers and tags Mapping custom classes to standard classes Specifying how a dialog is invoked Defining a method for a GUI object Defining a data structure for an object Modifying the message box declaration
Related topics
78 78 78 83 93 94 96 96 98 99
See Chapter 13, Porting Tests to Other GUIs, to learn about the ways you modify the test frame when porting your testcases to other GUIs. See Chapter 14, Supporting Internationalized Applications, to learn about internationalizing the test frame.
Users Guide
77
4 RECORDING A TEST FRAME Why window declarations make your tests robust
Why window declarations make your tests robust

Window declarations are extremely powerful features, for two reasons.
Declarations specify logical names
A window declaration specifies a cross-platform, logical name for a GUI object, called the identifier, and maps the identifier to the objects actual name, called the tag. Because your testcases use logical names, if the objects actual name changes on the current GUI, on another GUI, or in a localized version of the application, you need only change the tag in the window declarations; you dont need to change any of your scripts. You can add variables, functions, methods, and properties to the basic window declarations recorded by SilkTest. For example, you can add variables to a dialog box declaration that specify what the tab sequence is, what the initial values are, and so on. You access the values of variables at runtime as you would a field in a record.
Declarations can encapsulate data and functions
Preparing to record window declarations

If you plan to test Web applications, make sure you enable the correct browser extension(s) on your target and host machine. Note If you do not plan to test Web applications, you must disable all browser extensions on the host machine. For information on how to enable and disable browser extensions, see Enabling browser extensions on page 72 and Disabling browser extensions on page 75.
How to record window declarations

Two stages
Record window declarations for your application in two stages: Record the window declarations for the main window (including its menus and controls). Bring up each dialog one at a time and record a declaration for each.
Recording the main window and menu declarations
The following discussion pertains to non-Web applications. If you are testing Web applications, see Recording a test frame for a Web application in Testing Web Applications with SilkTest.
78
Users Guide
4 RECORDING A TEST FRAME How to record window declarations
Procedure To record declarations for the main window and menu hierarchy of your application: 1 2 Start up your application. Select File/New. The New dialog appears.
Select the Test Frame radio button and click OK. Note If you are using the testplan editor, you can also use the QuickStart Wizard to easily record a test frame with window declarations (see Getting started on page 19). If you do not have the testplan editor, the wizard icon (the hat) shown in the preceding dialog does not appear. The New Test Frame dialog is displayed, allowing you to create a test frame file for an application displayed in the Application list box. The Application list box displays all applications that are open and not minimized; if your test application is not listed, click Cancel, open your application, and select File/New again.
The default file name for the new test frame file is frame.inc. (The .inc extension denotes an include file, a file that stores declarations.) By default, subsequent frame files in the same directory are named
Users Guide
79
frame1.inc, frame2.inc, and so on. You can edit the path, file name, or both. Click the Browse pushbutton if you need help editing the name or path. 4 Select your application from the Application list box. The Command Line, Working Directory, and 4Test Identifier fields are updated for your application: The Command Line field displays the path of the applications executable. The Working Directory field names the directory in which the applications program files and documents are located. The 4Test Identifier field displays the default identifier of the applications main window, which is based on the actual caption of the main window. Note If you are testing a Web application, you see different fields. For more information, see Recording a test frame for a Web application in Testing Web Applications with SilkTest. You can edit these fields if you want. 5 Click OK. The new test frame file is created. The file contains the 4Test declarations for the main window and all its menus, as well as a generic declaration that is valid for each of the standard message boxes in the application. For example:
Window declarations appear in the outline editor, which means that the declarations for individual GUI objects can be expanded to show detail, collapsed to hide detail, and edited if necessary.
File name is stored in runtime options
When you record a test frame, the full path of the test frame file is added to the Use Files field of the Runtime Options dialog. This means that SilkTest can use the information contained in the declarations and recognize the GUI objects in your application when you record testcases.
80
Users Guide
Note If you dont want to use the new test frame you just recorded and instead want to use the test frame you had been using, be sure to respecify the pathname of the old test frame in the Runtime Options dialog (select Options/Runtime).
Recording declarations for a dialog
After you record your test applications main window and menus, you record all the dialogs you want to test. Procedure To record your applications dialogs, use this procedure once for each dialog in your application: 1 Make sure that the test frame file that contains the declarations for the applications main window is open. The dialog declarations will be appended to this file. Select Record/Window Declarations. The Record Window Declarations dialog is displayed. 3 4 5 Make your application active and invoke one of its dialogs, referred to in this procedure as the target dialog. If necessary, arrange windows so that you can see the target dialog. Position the cursor on the title bar of the target dialog. Note that as you move the cursor toward the title bar, the contents of the Window Declaration list box change dynamically to reflect the object at which youre pointing, as well as any contained objects. When the cursor is positioned correctly, the Window Detail group box (upper left) shows the caption of the dialog in the Identifier field. 6 Press Ctrl+Alt. The declaration is frozen in the lower half of the dialog.
Users Guide
81
7 8
Close the target dialog. Click on the Paste to Editor pushbutton. The information in the Record Window Declarations dialog is cleared, and the newly recorded declarations are appended to the test frame after the last recorded declaration.
If finished recording declarations, click the Close pushbutton on the Record Window Declarations dialog. Otherwise, press the Resume Tracking pushbutton to begin recording the declarations for another dialog.
Recording a login window
Many commercial applications begin with a login window, which is not accounted for when you record the test frame. Therefore, make sure that you invoke this window and record a declaration for it when you are recording the declarations for your applications dialogs. The wStartup constant When you record the test frame, a constant called wStartup is created. By assigning the identifier of the login window to wStartup and by recording a new invoke method, your tests can start the application, enter any required information into the login window, then dismiss the login window. For more information, see Handling login windows on page 297.
Saving the test frame
After you finish recording declarations for each one of your applications dialogs, save the test frame file. Procedure To save a test frame, select File/Save when the test frame is the active window. If it is a new file, it is automatically named frame.inc (if you already have a frame.inc file, a number is appended to the file name). You can use File/Save As to select another name. When saving a file, SilkTest does the following: Saves a source file, giving it the .inc extension; the source file is an ASCII text file, which you can edit Saves an object file, giving it the .ino extension; the object file is a binary file that is executable, but not readable by you
Example If you name a frame file myframe and save it, you will end up with two files: the source file myframe.inc, in the location you specify, and the object file myframe.ino. About object files Object files are used for test frames and script files. Object files are described in Using object files on page 134.
82
Users Guide
4 RECORDING A TEST FRAME Window declaration syntax and structure
Window declaration syntax and structure

This section explains the structure and syntax of window declarations, using some of the declarations for the sample Text Editor application as an example. Every window declaration consists of a class, identifier, and one or more tags, as shown in the following illustration.
Tag
Identifier Class
The window declaration maps the objects logical, platform-independent name, called the identifier, to the objects actual name, called the tag.
About classes
The class indicates the type, or kind, of GUI object being declared.
Classes are cross platform
Note that this is the 4Test class, not the class that the GUI itself uses internally. For example, although the class might be Label on one GUI and Text on another, 4Test uses the class name StaticText to refer to text strings which cannot be edited. The class also defines methods (actions) and properties (data) that are inherited by the GUI object. For example, if you record a declaration for a pushbutton named OK, a testcase can legally use a method like Click on the pushbutton because the Click method is defined at the class level. In other words, the definition of what it means to click on a pushbutton is included within the definition of the 4Test class itself, and this definition is inherited by each pushbutton in the GUI. If this were not true, you would have to define within each GUI objects window declaration all the methods you wanted to use on that object.
Users Guide 83
A class defines data and behavior
4 RECORDING A TEST FRAME Window declaration syntax and structure The class cannot be changed
The class as recorded cannot be changed. The one exception is that if the recorded class is CustomWinmeaning that SilkTest does not recognize the objectyou can, when appropriate, map the class to one that is recognized, as described in Mapping custom classes to standard classes on page 94. For more information on using object-oriented classes, see Chapter 16, Extending the Class Hierarchy. Refer to the online Help for descriptions of the variables, methods, and properties defined by each class.
Related topics
About identifiers
The identifier is the GUI objects logical name.
Default identifier is based on the label/ caption
By default, SilkTest derives the identifier from the objects actual label or caption, removing any embedded spaces or special characters (such as accelerators). So, for example, the Save As label becomes the identifier SaveAs. Note Identifiers can contain single-byte international characters, such as and .
Identifier can be based on the index
If the object does not have a label or caption, SilkTest constructs an identifier by combining the class of the object with the objects index. The index is the objects order of appearance, from top left to bottom right, in relation to its sibling objects of the same class. For example, if a text field does not have a label or caption, and it is the first text field within its parent object, the default identifier is TextField1. Note that the identifier is arbitrary, and you can change the generated one to the unique name of your choice. See Modifying identifiers and tags on page 93.
Identifier can be changed
About tags
The tag is the actual name of the object, as opposed to the identifier, which is the logical name. SilkTest uses the tag to identify objects in the application under test when recording and when executing testcases. Testcases never use the tag to refer to an object; they always use the identifier.
84
Users Guide
4 RECORDING A TEST FRAME Window declaration syntax and structure Five kinds of tags
Unlike identifiers, tags are not arbitrary. There are five types of tags:
Tag type Description
Caption Prior text Index Window ID Location
The caption or label as it appears to the user. Closest static text above or to the left of the object. Prior text tags begin with the ^ character. The order of the object in relation to its sibling objects of the same class. Index tags begin with the # character. The GUI-specific internal ID of the object. Window ID tags begin with the $ character. The physical location (coordinates) of the object. Location tags begin with the @ character.
Note Not all types of objects have all five tags. Dialogs, for example, do not have window IDs, so they cannot have a Window ID tag. Example Here is what you would see if you record declarations for the Case Sensitive check box in the Text Editors Find dialog.
Users Guide
85
There are five possible tags for the check box:

Tag type Value Comment
Caption Prior text Index Window ID Location
Case sensitive ^Find What: #1 $1041 @(57,65) Find What is the nearest static text above or to the left of the check box The Case Sensitive check box is the first check box in the dialog
These are the possible tags that can be used by SilkTest to identify the Case Sensitive check box when recording or executing testcases.
What happens by default
You can record more than one tag for an object. Doing so makes scripts less sensitive to changes when the tests are run. For example, if you record a caption and a window ID for a control, then even if the caption on the control changes (such as the caption Case Sensitive changing to Case is significant), SilkTest can still find the control based on its window ID. This is particularly an issue in situations where captions change dynamically, such as in MDI applications where the window title changes each time a different child window is made active. By default, when you record window declarations, each object is given two tags: the caption (if there is one) and the Window ID (if there is one). Notice in the Record Declarations dialog shown above that two tags are checked in the Tag Information box: Caption and Window ID. For example, here is the default recorded declaration for the Case Sensitive check box:
CheckBox CaseSensitive multitag "Case sensitive" "$1041"
SilkTest specifies multiple tags in a declarations file using the multitag statement. In the previous example, the check box is declared with two tags: The string Case sensitive, which is its caption The string $1041, which is its Window ID
For complete information about the multitag statement, see multitag statement in online Help.
86
Users Guide
4 RECORDING A TEST FRAME Window declaration syntax and structure Specifying which tags to record
When you are recording declarations, you can select any combination of tags to record by selecting check boxes in the Tag Information group box in the Record Window Declarations dialog. You can record different tags for different objects. You can also specify which tags you want recorded by default. Procedure To change which tags are recorded by default for GUI objects: 1 2 Select Record/Window Declarations. The Record Window Declarations dialog displays. Click Options. The Record Window Declarations Options dialog displays.
3 4
Select and unselect the check boxes in the Default Multitags group box as appropriate. Click OK. The next time you record window declarations, SilkTest will use the tag types you selected by default. You can always override the defaults for a particular object.
Using class-specific multiple tags
You can specify which multiple-tag types to use for an individual class. For example, maybe you dont want window ID used with a particular class, even though you want window ID used with all other classes. You can specify this by including a setting statement in the declaration for the class. For more information, see winclass declaration in online Help.
If an object doesnt have a caption or if the caption is not unique
By default, SilkTest follows these steps to create a Caption tag for an object: 1 2 SilkTest uses the literal label or caption of the object, if there is one. If the object has a sibling object with the same label or caption, SilkTest appends the objects index number to the tag. The index number is the objects order of appearance in relation to other sibling objects of the same class, from top left to bottom right within the parent object.
Users Guide
87
For example, if a dialog has two objects labeled Find, the tag of the one nearest the top left of the dialog is Find[1] and the tag of the one nearest the bottom right of the dialog is Find[2]. 3 If the object does not have a label or caption, SilkTest uses the index number. For example, if a dialog contains two unnamed text fields, the text field closest to the upper left corner of the dialog has the tag #1, and the other has the tag #2. Changing tags Sometimes you need to change tags from what SilkTest named them by default. See Modifying identifiers and tags on page 93.
How SilkTest uses multiple tags at runtime
When running your testcases, the Agent tries to resolve each part of a multiple tag from top to bottom until it finds an object that matches. Consider this declaration:
CheckBox CaseSensitive multitag "Case sensitive" "#1"
When SilkTest encounters a reference to Find.CaseSensitive, it first looks for a check box whose caption is Case sensitive. If it finds one, it uses it. If it doesnt find one, it looks for the first check box in the dialog (because of the index tag #1). If there is one, SilkTest uses it. If none of the tags resolve, an exception is raised. For complete information about tag resolution, see multitag statement in online Help.
Turning off multipletag recording
Because multiple tags make your scripts less sensitive to changes at runtime, you will want to use them. They make your scripts play back more reliably. However, if your application has tags that contain the pipe character ( | ), you need to turn off multiple-tag support, because the multitag statement uses the pipe character to delimit the tags. Procedure To turn off multiple-tag recording. 1 2 3 Select Record/Window Declarations. The Record Window Declarations dialog displays. Click Options. Deselect the Record Multiple Tags check box. The check boxes in the Default Tag group box become radio buttons.
88
Users Guide
4 5
Select the tag type you want SilkTest to use by default. Click OK. Now when you record window declarations, SilkTest will default to the tag type you selected and record the tag in a tag statement. You can always override the default for a particular object. For more information, see tag statement in online Help.
About GUI specifiers

Where SilkTest can detect a difference from one platform to the next, it automatically inserts a GUI-specifier in a window declaration to indicate the platform, for example msw95 mac or motif. For a complete list of the valid GUI specifiers, see GUITYPE data type in online Help.
About the main window and menu declarations

The main window declaration
The following example shows the beginning of the default declaration for the main window of the Text Editor application:
window MainWin TextEditor multitag "Text Editor" "$D:\MYTESTS\TEXTED2.EXE"
As is true for all window declarations, the declaration for the main window is composed of a class, identifier, and tag.
Part of declaration Value for TextEditors main window
Class Identifier
MainWin TextEditor
Users Guide
89
Part of declaration
Value for TextEditors main window
Tag
Two components in the multiple tag: Text EditorThe applications caption executable pathThe window ID (the full path of the executable file that invoked the application)
The window statement
The main window declaration begins with the 4Test reserved word window. The term window is historical, borrowed from operating systems and window manager software, where every GUI object (for example main windows, dialogs, menu items, and controls) is implemented as a window. When you record the declaration for your applications main window and menus, the sCmdLine and wMainWindow constants are created. These constants allow your application to be started automatically when you run your testcases. The sCmdLine constant specifies the path to your applications executable. The following example shows sCmdLine constants for different operating environments:
msw const sCmdLine = "d:\mytests\texted2.exe" motif const sCmdLine = "textedit" mac const sCmdLine = "Centris610:Sample:Text Editor"
The sCmdLine and wMainWindow constants
The wMainWindow constant specifies the 4Test identifier for the main window of your application. For example, here is the definition for the wMainWindow constant of the Text Editor application on all platforms:
const wMainWindow = TextEditor The menu declarations
The following example from the Text Editor application shows the default main window declaration and a portion of the declarations for the File menu:
window MainWin TextEditor multitag "Text Editor" "$D:\MYTESTS\TEXTED2.EXE" . . . Menu File tag "File" MenuItem New multitag "New" "$100"
90
Users Guide
Note Menus do not have window IDs, but menu items do, so by default menus are declared with the tag statement while menu items are declared with the multitag statement. For more information, see About tags on page 84.
Declarations are nested
The declarations for the menus are nested (indented) within the declaration for the main window, and the declarations for the menu items are nested within their respective menus. This nesting denotes the hierarchical structure of the GUI, that is, the parent-child relationships between GUI objects. In the sample Text Editor application, MainWin is the parent of the File menu; the File menu is considered a child of the MainWin. Similarly, all the menu items are child objects of their parent, the File menu. A child object belongs to its parent object, which means that it is either logically associated with the parent or physically contained by the parent. Because child objects are nested within the declaration of their parent object, the declarations for the child objects do not need to begin with the reserved word window.
About dialog declarations

The following example from the Text Editor application shows the declarations for the Find dialog and its contained controls:
window DialogBox Find tag "Find" parent TextEditor StaticText FindWhatText multitag "Find What:" "$65535" TextField FindWhat multitag "Find What:" "$1152" CheckBox CaseSensitive multitag "Case sensitive" "$1041" StaticText DirectionText multitag "Direction" "$1072" RadioList Direction multitag "Direction" "$1056" PushButton FindNext multitag "Find Next" "$1" PushButton Cancel
Users Guide
91
4 RECORDING A TEST FRAME Window declaration syntax and structure multitag "Cancel" "$2" Declarations are nested The parent statement
The declarations for the controls contained by a dialog are nested within the dialogs declaration to show the GUI hierarchy. Although a dialog is not physically contained by the main window, as is true for menus, the dialog nevertheless logically belongs to the main window. Therefore, a parent statement within each dialog declaration is used to indicate that it belongs to the main window of the application.
About the generic message box declaration

When you create a new test frame, SilkTest also automatically creates a declaration for a generic message box, one that is valid for any message box in the application. Therefore, you do not have to record a declaration for every one of the message boxes (potentially hundreds) in your application. A message box is a dialog that has static text and pushbuttons, but no other controls. Typically, message boxes are used to prompt users to verify an action (such as Save changes before closing?) or to alert users to an error. Here is the declaration for the generic message box:
window MessageBoxClass MessageBox tag "~ActiveApp/[DialogBox]$MessageBox" PushButton OK tag "OK" PushButton Cancel tag "Cancel" PushButton Yes tag "Yes" PushButton No tag "No" StaticText Message motif tag "#2" tag "#1" Why the declaration is generic
The message box declaration is generic for three reasons: The dialogs tag specifies that its parent is the current active application. The most likely names for pushbuttons are accounted for: OK, Cancel, Yes, and No. The tag of the message is an index number, not the text of the message.
You can enhance the generic declaration. See Modifying the message box declaration on page 99.
92
Users Guide
4 RECORDING A TEST FRAME Modifying identifiers and tags
Modifying identifiers and tags

In some instances, you may want to modify the recorded identifiers or tags.
Why change the identifier
Because an identifier is a logical name, it is arbitrary and can be any name of your choice. Therefore, if you find the generated identifier too long or not descriptive enough, you can change it. By default, the GUI objects caption and index are used for the tag, because they are the most portable. In most cases, these are what you want to use. However, there are situations in which the default tag is not suitable. For complete information about using alternative tags, see the section How to choose a tag-string form in the multitag statement topic in online Help.
Why change the tags
Modifying in the Record dialog
You can modify the identifier or tag for a dialog as you are recording it. Procedure To modify a declaration in the Record Window Declarations dialog: 1 In the Window Declaration list box at the bottom of the dialog, click the line for the object containing the tag or identifier you want to change. The Window Detail group box in the upper left of the dialog updates to include the information from the line you clicked on. 2 To change the identifier, simply replace the existing identifier with one of your choice. To change the tag, select the tag types you want to include in the generated multitag. You can edit the contents of each tag type in the text fields in the Tag Information group box. The Window Declaration list box updates dynamically as you enter the new information. 3 When finished making modifications, press the Paste to Editor pushbutton. The declarations for the dialog are appended to the test frame file. Example of changing tag You might want to provide more than one caption for a control if the controls caption can change dynamically. For example, if a push button sometimes says Yes and sometimes says Continue, you could change the tag as shown here: Note that you separate different tag components using the pipe character ( | ).
Users Guide
93
4 RECORDING A TEST FRAME Mapping custom classes to standard classes
Modifying after pasting
If you need to modify the identifier or tag for the applications main window or menus, you do so after you have pasted the declarations to the test frame file, by editing the file directly.
Mapping custom classes to standard classes

While using the Record Window Declarations dialog, you will on occasion notice that the recorded class is CustomWin, indicating that SilkTest does not recognize the class of the object. The object is interpreted as a custom object. If the object is in fact a custom object, then you need to follow the procedures outlined in Chapter 17, Supporting Custom Objects, to be able to record and run testcases that interact with the custom object. However, if the object is not actually a custom object, but is instead a standard object that your applications developers have renamed, you can record and run testcases merely by establishing a class map between the renamed class and the standard 4Test class. Procedure To map a class when a declaration for a CustomWin appears in the Record Window Declarations dialog: 1 Press Ctrl+Alt. The declarations are frozen in the Window Declaration list box in the lower half of the Record Window Declarations dialog.
94
Users Guide
4 RECORDING A TEST FRAME Mapping custom classes to standard classes
In the Window Declarations list box, click on the line containing the declaration for the custom object. The line is highlighted and the declaration for the CustomWin appears in the Window Detail group box.
In the Window Detail group box, click Class Map. The Class Map dialog appears. The name of the custom class is displayed in the Custom Class text field.
4 5 6
In the Standard Class field, enter the name of the built-in 4Test class to which you are mapping the custom class. Click Add. Click OK.
When you resume recording, the object has the standard 4Test class. Any subsequent similar objects encountered will automatically be mapped to the correct 4Test class. You must modify any prerecorded declarations containing these objects to use the standard class.
Mapping and recording Manager Class OSF/Motif widgets

The Manager Class widgets of OSF/Motif (including XmFrame and XmForm) are by default mapped to the Ignore class, and are ignored when you record your test frame. If you wish to include these objects among those being recorded, you must remap these classes to themselves. To learn which widgets are being mapped to the Ignore class, check the Show Ignored Windows option in the Record Window Declarations Options dialog. Ignore-class widgets are displayed in the text field at the bottom of the Record Window Declarations dialog. As long as these widgets are classed as Ignore, they will not be pasted into your test frame.
Users Guide
95
4 RECORDING A TEST FRAME Specifying how a dialog is invoked
To enable the recording of these declarations, map the Custom Class of the Ignore class widget onto itself. For example, to un-ignore the class XmForm, enter XmForm in both the Custom Class and Standard Class windows of the Class Map dialog. You can also use Class Mapping to add custom windows to the Ignore class.
Specifying how a dialog is invoked

Two ways to invoke
4Test provides two equivalent ways to invoke a dialog: Use the Pick method to pick the menu item that invokes the dialog, as in:
TextEditor.File.Open.Pick ()
Use the Invoke method, as in:

Open.Invoke ()
While both are equivalent, using the Invoke method makes your testcases more maintainable. For example, if the menu pick changes, you only have to change it in your window declarations, not in any of your testcases.
The Invoke method
To use the Invoke method, the declaration for the dialog needs to specify the dialogs wInvoke variable, which contains the identifier of the menu item or button that invokes the dialog. For example:
window DialogBox Open tag "Open" parent TextEditor WINDOW wInvoke = TextEditor.File.Open
Defining a method for a GUI object

If you need to perform an action on an object, and the class does not provide a method for doing so, you can define your own method in the window declaration for the object. Then, in your scripts, you can use the method as though it were just another of the classs built-in methods. You can hand-code methods or record them. Procedure To record a method: 1 2 Position the insertion point on the declaration of the GUI object you want to add a method to. Select Record/Method. The Record Method dialog displays.
96 Users Guide
4 RECORDING A TEST FRAME Defining a method for a GUI object
3 4 5 6
Example
Name the method by typing the name or selecting one of the predefined methods: BaseState, Close, Invoke, or Dismiss. Record the actions that make up the method. Edit the 4Test statements that were recorded, if necessary. Paste the code to your include file.
For example, suppose you want to create a method named SetLineNum for a dialog named GotoLine, which does the following: Invokes the dialog Enters a line number Clicks the OK button
The following 4Test code shows how to add the definition for the SetLineNum method to the GotoLine dialogs declaration:
window DialogBox GotoLine tag "Goto Line" parent TextEditor const wInvoke = TextEditor.Search.GotoLine void SetLineNum (STRING sLine) Invoke () // open dialog Line.SetText (sLine) // populate text field // whose identifier is Line Accept () // close dialog, accept values
Then, to go to line 7 in the dialog, you use this method call in your testcases:
GotoLine.SetLineNum (7)
Users Guide
97
4 RECORDING A TEST FRAME Defining a data structure for an object Where to define the method
When you include a method definition in a GUI objects window declaration, you can only use the method on that one object. If you want to define a method that you can use on all GUI objects of a given class, you should define the method at the class level. For more information, see Adding methods to a class on page 306.
Defining a data structure for an object

As you design your testcases, you may want to associate data with individual objects, which can then be referenced inside testcases. You may find this preferable to declaring global variables or passing parameters to your testcases. The type of data you decide to define within a window declaration will vary, depending on the type of testing you are doing. Some examples include:
Example: a dialogs tab sequence
The default value that you expect the object to have when it displays The tab sequence for each of a dialog boxs child objects
The following declaration for the Find dialog contains a list that specifies the tab sequence of the dialogs children.
window DialogBox Find tag "Find" parent TextEditor LIST OF WINDOW lwTabOrder = {...} FindWhat CaseSensitive Direction Cancel
For more information about the syntax to use for lists, see LIST data type in online Help.
Use the member-of operator to access data
Use the member-of operator ( . ) to reference the data defined in a window declaration. For example, if a script needs to know which control should have focus when the Find dialog (declared in the preceding example) is first displayed, it can access this data from the window declaration with this expression:
Find.lwTabOrder[1]
Similarly, to set focus to the third control in the list:

Find.lwTabOrder[3].SetFocus ()
98
Users Guide
4 RECORDING A TEST FRAME Modifying the message box declaration
Modifying the message box declaration

When to modify the declaration
When SilkTest generates the window declarations for the main window of your application, it also includes a declaration for a generic object named MessageBox, as described in About the generic message box declaration on page 92. If your application contains message boxes that have extra pushbuttons or if your pushbuttons use different names, you need to add the declarations for those buttons to the declaration for the generic MessageBox object. If a message box contains a Test pushbutton, for example, you need to add the following lines to the recorded declaration:
PushButton Test tag "Test"
Example
Users Guide
99
4 RECORDING A TEST FRAME Modifying the message box declaration
100
Users Guide
5e rh C t p a
Designing and Recording Testcases

This chapter shows you how to use the powerful SilkTest recorder to automatically capture the actions you perform as you interact with your application, creating an automated testcase. The cornerstone of an automated test is the verification stage, in which the test verifies that the state of the application matches the expected (baseline) state. Using the recorder, you can record object-appropriate verification of your applications state, data, or appearance.
Introduction
What you will learn

Topic Page
Preparing to design and record testcases Test running the built-in recovery system Testcase design principles Defining test requirements Overview of recording the stages of a testcase Recording the setup stage Recording the verification stage Recording the cleanup stage and pasting the recording Saving a script file Using object files Recording from within a testplan How recorded commands uniquely identify GUI objects
102 105 106 108 109 112 112 132 133 134 136 138
Users Guide
101
5 DESIGNING AND RECORDING TESTCASES Preparing to design and record testcases
Topic
Page
Recording without window declarations Recording 4Test components
138 139
Preparing to design and record testcases

If you plan to test Web applications, make sure you enable the correct browser extension(s) on your target and host machine. For more information, see Chapter 3, Enabling Extensions for Applications Under Test and Getting Started: A Tutorial. If you do not plan to test Web applications, you must disable all browser extensions on the host machine.

By default, SilkTest enables all the browser extensions on your target machine during the installation procedure. To change the default settings or verify your current settings, you must invoke a utility called the Extension Enabler, as described in the following procedure. Note If you are running local teststhat is, your target and host are the same machineyou must still ensure that browser extensions are enabled appropriately by running the Extension Enabler on the host machine. Procedure To enable support for browsers on a target machine: 1 From the SilkTest program group, click Extension Enabler. The Extension Enabler dialog opens, as in this example:
102
Users Guide
2 3
Enable support for the browsers you want to use, if they arent already enabled. In the Primary Extension field for a browser, choose Enabled. Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Click OK to close the Extension Enabler dialog.
4
You enable support for browsers on the host machine using the Extensions dialog. Be advised that there is overhead associated with having more than one browser extension enabled, so do so only if you are actually testing more than one browser in an automated session. Procedure To enable support for browsers on a host machine: 1 2 Start SilkTest. Select Options/Extensions. The Extensions dialog opens. All the installed extensions are listed, including the browser extensions, as in this example:
Users Guide
103
Enable the browser extension(s) you want to use. In the Primary Extension field for a browser, choose Enabled. Disable any browser extensions you do not plan to use by choosing Disabled in the Primary Extension field and unchecking their other extensions. Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Note If you enable support for ActiveX in this dialog, you must make sure that it is enabled in the Extension Enabler dialog as well.
Click OK to close the Extensions dialog.
You can get information about the files used by an extension by selecting an extension and clicking the Details pushbutton.

If you testing non-Web applications, you must disable browser extensions on your host machine. This is because the recovery system works differently when testing Web applications than when testing non-Web applications. For more information, see Getting Started: A Tutorial. Procedure To disable support for browsers on a host machine: 1 2 Start SilkTest. Select Options/Extensions. The Extensions dialog opens.
104
Users Guide
5 DESIGNING AND RECORDING TESTCASES Test running the built-in recovery system
Disable any browser extensions you do not plan to use by choosing Disabled in the Primary Extension field and unchecking all check boxes in the Other Extensions columns. Click OK to close the Extensions dialog.
Test running the built-in recovery system

Before you begin to design and record testcases, you should first make sure that the built-in recovery system can close representative dialogs from your application. Although the recovery system is robust enough to be able to close almost any application window, some applications may have windows that close in an unconventional fashion.
Three types of dialogs should be tested
Here are the three kinds of dialogs you should test: A modal dialog (a dialog that locks you out of the rest of your application until you dismiss it) A non-modal dialog A non-modal dialog that causes the display of a confirmation dialog
Procedure To test the recovery systems ability to close your applications dialogs: 1 2 3 4 5 Start SilkTest. If you have not already done so, record a test frame for your application. For more information, see Chapter 4, Recording a Test Frame. Make sure your applications test frame file is listed in the Use Files field in the Runtime Options dialog (select Options/Runtime). Start your application and invoke a representative dialog. Select Run/Application State. The Run Application State dialog is displayed:
Select the DefaultBaseState application state and click Run.
Users Guide
105
5 DESIGNING AND RECORDING TESTCASES Testcase design principles
The DefaultBaseState routine is executed, which should close the dialog and any open windows, then display a results file. For example, the following figure shows the results file produced when the recovery system closes the Text Editors Open dialog:
If the built-in recovery system cannot close one of the three representative dialogs, you need to modify the recovery system so that it understands how to close the dialog. For more information, see Specifying new window closing procedures on page 293.
Testcase design principles

This section explains the methodology you use when you design and record a testcase.
Two kinds of testcases
There are two basic types of testcases, each with its own purpose. Level 1 tests, often called smoke tests or object tests, verify that an applications GUI objects function properly. For example, they verify that text fields can accept keystrokes and check boxes can display a check mark. Level 2 tests verify an application feature. For example, they verify that an applications searching capability can correctly find different types of search patterns.
You typically run Level 1 tests when you receive a new build of your application, and do not run Level 2 tests until your Level 1 tests achieve a specific pass/fail ratio. The reason for this is that unless your applications graphical user interface works, you cannot actually test the application itself.
A testcase has three stages
Each testcase that you record should have the following three stages. 1 It drives the application from the initial state to the state you want to test.
106
Users Guide
5 DESIGNING AND RECORDING TESTCASES Testcase design principles
It verifies that the actual state matches the expected (correct) state. (Your QA department might use the term baseline to refer to this expected state.) This stage is the heart of the testcase. It cleans up the application, in preparation for the next testcase, by undoing the steps performed in stage 1.
3
Each testcase is independent
Each testcase you record should perform its own setup in stage 1, and should undo this setup in stage 3, so that the testcase can be executed independently of every other testcase. In other words, the testcase should not rely upon the successful or unsuccessful completion of another testcase, and the order in which it is executed should have no bearing on its outcome. If a testcase relies on a prior testcase to perform some setup actions, and an error causes the setup to fail or, worse yet, the application to crash, all subsequent testcases will fail because they cannot achieve the state where the test is designed to begin.
A testcase has a single purpose
Each testcase you record should verify a single aspect of the application in stage 2. When a testcase designed in this manner passes or fails, its easy to determine specifically what aspect of the target application is either working or not working. If a testcase contains more than one objective, many outcomes are possible. Therefore, an exception may not point specifically to a single failure in the software under test but rather to several related function points. This makes debugging more difficult and time-consuming and leads to confusion in interpreting and quantifying results. The net result is an overall lack of confidence in any statistics that might be generated. Note But there are techniques you can use to do more than one verification in a testcase. See Performing more than one verification in a testcase on page 211.
A testcase starts from a base state
In order for a testcase to be able to function properly, the application must be in a stable state when the testcase begins to execute. This stable state is called the base state. The recovery system is responsible for maintaining the base state in the event the application fails or crashes, either during a testcases execution or between testcases. DefaultBaseState To restore the application to the base state, the recovery system contains a routine called DefaultBaseState that makes sure that The application is running and is not minimized All other windows (for example, dialogs) are closed The main window of the application is active
Users Guide
107
5 DESIGNING AND RECORDING TESTCASES Defining test requirements
If these conditions are not sufficient for your application, you can customize the recovery system. For more information on how the recovery system works and how it can be customized, see Chapter 15, Understanding the Recovery System.
Defining test requirements

When defining test requirements, the goal is to rigorously test each application feature. To do so, you need to decide which set of inputs to a feature will provide the most meaningful test results.
Example: a word processors search feature
For purposes of illustration, this section develops test requirements for the searching feature of the sample Text Editor application. This feature uses the Find dialog.
When a user enters the criteria for the search and clicks the Find Next pushbutton, the searching feature attempts to locate the string. If the string is found, it is selected (highlighted). Otherwise, an informational message is displayed.
What data does the feature expect
A user can enter three pieces of information in the Find dialog: The search can be case sensitive or insensitive, depending on whether the Case Sensitive check box is set or unset. The search can be forward or backward, depending on whether the Down or Up radio button is selected. The search can be for any combination of characters, depending on the value entered in the Find What text field.
Create meaningful data combinations
To organize this information, it is helpful to construct a table that lists the possible combinations of inputs. From this list, you can then decide which combinations are meaningful and should be tested. A partial table for the Find dialog is shown below:
Case Sensitive Direction Search String
Yes Yes
Down Down
Character Partial word (start)
108
Users Guide
5 DESIGNING AND RECORDING TESTCASES Overview of recording the stages of a testcase
Case Sensitive
Direction
Search String
Yes Yes Yes Yes Yes Yes Yes Yes
Down Down Down Up Up Up Up Up
Partial word (end) Word Group of words Character Partial word (start) Partial word (end) Word Group of words
Overview of recording the stages of a testcase

As explained in Testcase design principles on page 106, a testcase performs its actions in three stages. The following table illustrates these stages, describing in high-level terms the steps for each stage of a sample testcase that tests whether the Find facility is working.
Stage Example from Text Editor
1: Setup
Open a new document. Type text into the document. Position the text cursor either before or after the text, depending on the direction of the search. Select Find from the Search menu. In the Find dialog: - Enter the text to search for in the Find What field. - Select a direction for the search. - Make the search case sensitive or not. - Click Find Next to do the search. Click Cancel to close the Find dialog. Record a 4Test verification statement that checks that the actual search string found, if any, is the expected search string. Close the document. Click No when prompted to save the file.
2: Verify
3: Cleanup
Users Guide
109
Note for the testplan editor users After learning the basics of recording in this section, read about Recording from within a testplan on page 136, which makes recording easier by automatically generating for you the links that connect the testplan to the testcase.
Preparing to record
Procedure To invoke the Record Testcase dialog and set up the test: 1 Select Record/Testcase. a If a script file is not the active window, you are prompted for a file name.
If prompted, specify the name of either a new or an existing script file and click OK.
The Record Testcase dialog appears.
Type the name of your testcase in the Testcase Name field of the Record Testcase dialog. By default the name is Test1 (unless that name is already used in the current script file, in which case the default name is Test2,
110
Users Guide
and so on). Testcase names are not case sensitive; they can be any length and consist of any combination of alphabetic characters, numerals, and underscore characters. 3 Select DefaultBaseState in the Application State field to have the built-in recovery system restore the default base state before the testcase begins executing. Select another application state if you want the recovery system to perform an additional sequence of steps after the default base state is restored but before the testcase begins executing. For more information on application states, see Using application states on page 185. The application state you chose is made the default application state and is suggested in the Record Testcase dialog the next time you record a testcase. Note If you choose (None) as the application state, the recovery system will not be used at all. This means that you must manually put your application in the starting state before running the testcase. 4 If you do not want SilkTest to display the status window it normally shows during playback when driving the application to the specified base stateperhaps because the status bar obscures a critical control in the application you are testinguncheck the Show AppState status window check box. Click the Start Recording pushbutton. The following sequence of events occurs: Using the Record Status window
The Record Testcase dialog closes. Your application is started, if it was not already running. The editor window disappears from the display. The Record Status window appears.
As you move around the application while recording, the Record Status window dynamically updates to show the object that the recorder thinks the mouse is currently over or the control that has focus.
Users Guide
111
5 DESIGNING AND RECORDING TESTCASES Recording the setup stage
Use this information to make sure that the recorder has kept up with your actions (sometimes the recorder can fall behind if your systems resources are low). In particular, check the Record Status window before pressing Ctrl+Alt to do your verification in order to make sure you are verifying the correct object.
Recording the setup stage

Procedure To record the setup stage: 1 Interact with your application, driving it to the state that you want to test. As you interact with your application, your interactions are recorded in the Testcase Code field of the Record Testcase dialog, which is not visible. 2 To review what you have recorded, click the Done button in the Record Status window. The Record Testcase dialog is redisplayed, containing the 4Test code that has been recorded for you. To resume recording your interactions, click the Resume Recording button in the dialog. 3 To temporarily suspend recording, click the Pause Recording button on the Record Status window. You might want to halt recording to leave your desk or to prevent some interactions from being recorded.
Recording the verification stage

Testing applications involves verifying that the state of a GUI object is the same as the baseline state that you expect. You can: Test the characteristics of an object Capture all or part of a bitmap to verify the appearance of an object Use a method to verify an object
Procedure To record the verification stage: 1 To add a verification statement, position the mouse cursor over the object. Look at the Record Status window and make sure it is listing the object you want to verify. If so, press Ctrl+Alt. (On OSF/Motif, make the
112
Users Guide
5 DESIGNING AND RECORDING TESTCASES Recording the verification stage
window you want to verify active and press Ctrl+Alt. How you make a window active on Motif depends on how you set the keyboardFocusPolicy resource.) The Verify Window dialog appears over your application window.
The Window field, in the top-left corner of the dialog, displays the name of the object you were pointing at when you pressed Ctrl+Alt. 2 If the name in the Window field is incorrect, press Cancel to close the dialog and return to the application. Point to the object you want to verify and press Ctrl+Alt again.
Motif note: if Ctrl+Alt doesnt work
If your application under test is running under OSF/Motif using an X Window emulator on a PC and Ctrl+Alt does not work, do the following: 1 2 Quit SilkTest. Create an environment variable named QAP_VERIFY_KEY and assign it the value 1. In the C shell, do this:
setenv QAP_VERIFY_KEY 1
In the Bourne shell, do this:

QAP_VERIFY_KEY=1; export QAP_VERIFY_KEY
In the Korn shell, do this:

export QAP_VERIFY_KEY=1
With this environment variable set, when you restart SilkTest, you can press F5 to display the Verify Window dialog.
Users Guide
113
5 DESIGNING AND RECORDING TESTCASES Recording the verification stage About the Verify Window dialog
The Verify Window dialog has three tabs. The dialog has these tabs by default.
Tab Purpose For more information
Property Bitmap
Verifies a characteristic of the object Captures all or part of a bitmap, if you want to verify the objects appearance Verifies the object using a method that has been defined for the class
See Verifying using properties on page 115. Verifying an objects bitmap on page 126. Verifying using methods on page 128.
Method
Attribute tab On non-Windows platforms, the Verify Window dialog has an Attribute tab instead of a Property tab. You use the Attribute tab on nonWindows platforms to verify a characteristic of the object. For information on verification using attributes, see Verifying object attributes on page 130. About attributes and properties Properties and attributes in this context are similarthey both are used to verify a characteristic of the object. However, properties are more encompassing, more flexible, and easier to use. For example, using attributes you can only verify one attribute at a time (or verify every attribute for an object and all its children); using properties you can verify selected properties of an object and any or all of its children at the same time. Under Windows, you can choose to verify an object using attributes instead of properties, though you will seldom choose attributes, except for compatibility with previous releases of SilkTest. Procedure To verify using attributes under Windows: 1 2 3 Select Options/Recorder. The Recorder Options dialog appears. Deselect the Verify Using Properties check box. Note You will not be able to deselect this check box if you have enabled enhanced support for Visual Basic which requires properties for verification. 4 Click OK.
114
Users Guide
With Verify Using Properties deselected, the next time you go to verify an object, the Verify Window dialog will have an Attribute tab, instead of a Property tab. For information on verification using attributes, see Verifying object attributes on page 130.
Verifying using properties

You will perform most of your verifications using properties. Each object has many characteristics, or properties. For example, as shown in the following illustration, dialog boxes have these verification properties: Caption, Children, DefaultButton, Enabled, Focus, Rect, and State.
Caption is the text that appears in the dialog boxs title bar; Children is a list of all the objects contained in the dialog box, DefaultButton is the button that is invoked when you press Enter, and so on. In your testcases, you can verify the state of any of these properties. You can also, in the same testcase, verify properties of children of the selected object. In the preceding illustration, notice that the child objects in the Find dialog box (such as the text field FindWhat and the check box CaseSensitive) are also selected for verification. By recording verification statements for the values of one or more of an objects properties, you can determine whether the state of the application is correct or in error when you run your testcases.
Users Guide
115
5 DESIGNING AND RECORDING TESTCASES Recording the verification stage Properties can be verified as a set
To make your testing easier, properties are organized into sets. Predefined sets Here are the predefined property sets:
This property set Includes properties that describe
Children Control State Menu State
Objects within the selected object, such as pushbuttons in a dialog box The state of controls, for example, whether a control is enabled The state of a menu, for example, whether its enabled or checked
Moveable Window State The state of a moveable window, for example, whether its enabled or the control that has focus Selection Style Value Range Values (default) Window Location and Size The currently selected row or current selection in an editable field Style variations for controls and objects Information that governs the range of possible values for controls and objects The current value of a control or object, for example, the text in a text field The position and size of objects on the screen
The property sets listed above are the general built-in property sets. If you have enabled an extension to provide enhanced support for testing an application built with a particular development environment, there might be additional property sets. For more information, see the online Help for the extension. If you are testing a Web application, there are additional property sets. For more information, see Getting Started: A Tutorial. Configuring property sets You can configure property sets to suit your needs, even combining frequently used property sets into a new larger property set. For more information, see Configuring your own property sets on page 121.
116
Users Guide
5 DESIGNING AND RECORDING TESTCASES Recording the verification stage How to verify a objects properties
Procedure To verify properties of an object: 1 Select Record/Testcase to begin recording a testcase (or select Record/ Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify. When you are ready to record a verification statement, position the mouse pointer over the object you want to verify. Look at the Record Status window and make sure it is listing the object you want to verify. If so, press Ctrl+Alt. To verify more than one object within a window, position the cursor on the title bar. The Capturing Properties window appears. Then the Verify Window dialog appears over your application window. The Window field (top) displays the identifier of the object to be verified; if the identifier is not correct, click Cancel and repeat this step.
Here are some points to note about the Property tab: The Windows to Verify list box (left) displays the class and the identifier of all the objects whose properties have been captured. Indentation denotes the hierarchy. A selected check box (left margin) means that the object will be verified. By default, all objects are checked and the first object is selected. The Properties to Verify list box (right) displays each property of the selected object and its current value. A selected check box (left margin) means that the property will be verified. By default, the properties of the selected property set (shown in the Property Set drop-down list box) are checked.
Users Guide
117
The Property Value field displays the value of the selected property. You can edit the value in this field if it is not what you want to verify against. The value specified in this field is the value you expect at runtime, that is, the baseline value.
3 4
Choose the object(s) to verify. To verify all or most objects, click the Check All pushbutton and then uncheck individual check boxes. Choose the properties to verify in one of the following ways: Select an object and check one or more check boxes in the Properties to Verify list box. Repeat this for all the properties of all objects that you want to verify. Select a set of properties from the Property Set drop-down list box. The list box displays the predefined property sets (Values is the default), and any property sets youve defined yourself. When you select a property set, all properties in that property set are selected and all properties not in that property set are deselected. You can select only one property set at a time.
5 6
Click OK to close the Verify Window dialog. If you are writing a complete testcase, record the cleanup stage and paste the testcase into the script. If you have added a verification statement to an existing testcase, paste it into your script and close the Record Actions dialog.
VerifyProperties method statement
A VerifyProperties method statement is added to your script. The VerifyProperties method verifies the selected properties of an object and its children. For more information about VerifyProperties, see the online Help.
Fuzzy verification
There are situations when SilkTest cannot see the full contents of a control (such as a text field) because of the way that the application paints the control on the screen. One known case is text fields in cells in Dev2000Tables. For example, consider a text field in a Dev2000Table whose contents are wider than the display area. In some situations (but not in others), the application clips the text to fit the display area before drawing it, meaning that SilkTest only sees the contents that are visible; not the entire contents. Consequently, when you later do a VerifyProperties against this text field, it may fail inappropriately. For example, say the true contents of the text field are 29 Pagoda Street, but only 29 Pagoda displays. Depending on exactly how the test is created and run, the expected value might be 29 Pagoda whereas the value seen at runtime might be 29 Pagoda Street (or vice versa). So the test would fail, even though it should pass.
118
Users Guide
To work around this problem, you can use fuzzy verification, where the rules for when two strings match are loosened. Using fuzzy verification, the expected and actual values do not have to exactly match. The two values are considered to match when one of them is the same as the first or last part of the other one. Specifically, VerifyProperties with fuzzy verification will pass whenever any of the following functions would return TRUE (where actual is the actual value and expected is the expected value): MatchStr (actual + "*", expected) MatchStr ("*" + actual, expected) MatchStr (actual, expected + "*") MatchStr (actual, "*" + expected)
(In string comparisons, * stands for any zero or more characters.) For example, all the following would pass if fuzzy verification is enabled:
Actual value Expected value
29 Pagoda oda Street 29 Pagoda Street 29 Pagoda Street
29 Pagoda Street 29 Pagoda Street 29 Pagoda oda Street
Enabling fuzzy verification You enable fuzzy verification by using an optional second argument to VerifyProperties, which has this prototype:
VerifyProperties (WINPROPTREE WinPropTree [, FUZZYVERIFY FuzzyVerifyWhich])
where the FUZZYVERIFY data type is defined as:

type FUZZYVERIFY is BOOLEAN, DATACLASS, LIST OF DATACLASS
So, for the optional FuzzyVerifyWhich argument you can either specify TRUE or FALSE, one class name, or a list of class names.
FuzzyVerifyWhich value Result
FALSE (default)
Fuzzy verification is disabled.
Users Guide
119
FuzzyVerifyWhich value
Result
One class
Fuzzy verification is enabled for all objects of that class. Example window.VerifyProperties ({}, Dev2000Table) enables fuzzy verification for all Dev2000Tables in window (but no other object).
List of classes
Fuzzy verification is enabled for all objects of each listed class. Example window.VerifyProperties ({}, {Dev2000Table, TextField}) enables fuzzy verification for all Dev2000Tables and text fields in window (but no other object).
TRUE
Fuzzy verification is enabled only for those objects whose FuzzyVerifyProperties member is TRUE. To set the FuzzyVerifyProperties member for an object, add the following line within the object's declaration:
FUZZYVERIFY FuzzyVerifyProperties = TRUE
Example If in the application's include file, the DeptDetails Dev2000Table has its FuzzyVerifyProperties member set to TRUE:
window ChildWin EmpData . . . Dev2000Table DeptDetails FUZZYVERIFY FuzzyVerifyProperties = TRUE
And the test has this line:

EmpData.VerifyProperties ({...}, TRUE)
Then fuzzy verification is enabled for the DeptDetails table (and other objects in EmpData that have FuzzyVerifyProperties set to TRUE), but no other object.
Fuzzy verification takes more time than standard verification, so only use it when necessary. For more information, see VerifyProperties method in online Help.
Defining your own verification properties
You can also define your own verification properties. For more information, see Defining new verification properties on page 308.
120
Users Guide
Configuring your own property sets

A property set consists of a list of properties and the class associated with each property. A number of property sets is predefined for your convenience. You can also:
Where property sets are stored
Create a new property set from scratch Create a new property set by combining existing sets Edit an existing set by adding or deleting properties or renaming the set Delete a property set
All property sets reside in the file named in the Data File for Property Sets field in the Recorder Options dialog.
If you... Your property set file is
Selected enhanced support for Visual Basic Did not select enhanced support for Visual Basic
vbprpset.ini propset.ini
The default file location is your SilkTest installation directory. To make sure that all testers in your group have access to the same property sets file, place the file on a shared drive and specify the full path in the Data File for Property Sets field. Note Extensions to SilkTest can also define property sets, which are defined in different files. See the documentation for your particular extension in online Help for more information about its property sets.
How to create a new property set
Procedure To create a new property set: 1 Select Options/Property Sets or the Define pushbutton on the Verify Window dialog.
Users Guide
121
The Property Sets dialog opens. The Property Sets list box displays all existing property sets.
Click the New pushbutton. The New Property Set dialog appears:
Specify a name for the new property set in the Name field. Property set names are not case sensitive; they can be any length and consist of any combination of alphabetic characters, numerals, and underscore characters. Specify a class in the Class field and then a property of that class in the Property field. For more information, see Specifying a class-property pair on page 123. Click Add. The class-property pair is added to the list box. Note The class or property name is not validated here, so type carefully. Invalid names are ignored at runtime. If you make a mistake, select the class-property pair and click Edit.
122
Users Guide
Repeat steps 4 and 5 for as many class-property pairs as you want to add. Delete any class-property pairs you dont want to include by selecting them and pressing Remove. Once the list of classes and properties is correct, click OK. The New Property Set dialog closes and the new property set is displayed in the Property Sets list box.
Click Close to dismiss the Property Sets dialog.
Specifying a class-property pair There are several ways to specify classproperty pairs. They can be specified as a full class name and a full property name. For example, to specify the State property for the CheckBox class, you would enter CheckBox in the Class field and State in the Property field. You can use the * wildcard character for partial matches. The asterisk matches zero or more characters. For example, specifying * as a class name matches all classes. Specifying Text* as a class name matches all classes that begin with the string Text. You can apply the rule of inheritance to property sets; that is, the properties of a class are inherited by its child classes. For example, specifying the Enabled property and the Control class as a pair means that the Enabled property of all classes descended from Control are also implicitly included in the property set.
How to create a property set by combining sets
You can only specify one property set at a time in the Verify Window dialog. So if you find that you use several property sets frequently, you might want to combine them into one set. The new property set represents the union of the combined sets. All constituent sets remain intact. Procedure To combine two property sets: 1 Select Options/Property Sets or the Define pushbutton on the Verify Window dialog. The Property Sets dialog opens, as shown on page 121. 2 Click the Combine pushbutton. The Combine Property Sets dialog appears:
Users Guide
123
3 4 5
Specify a name for the new property set in the Name field. Select at least two property sets from the Property Sets to Combine list box and click OK. The Combine Property Sets dialog closes and the new property is displayed in the Property Sets list box, as are the constituent sets. Note If you modify any of the constituent sets, the combined set will be modified as well.
How to edit a property set
You can modify an existing property set at any time to rename it or to edit the class-property pairs that compose it. Procedure To edit an existing property set: 1 Select Options/Property Sets or the Define pushbutton on the Verify Window dialog. The Property Sets dialog opens, as shown on page 121. 2 Select a property set from the Property Sets list box and click Edit. The Edit Property Set dialog opens. The following figure shows how the dialog looks when the Values property set is selected.
124
Users Guide
Take any of the following actions:

To Action and result
Edit the property set name Add a class-property pair
Edit the name in the Name field. Specify a class in the Class field. Specify a property for the class in the Property field. (For more information, see Specifying a class-property pair on page 123.) Click Add. The class-property pair is added to the list box.
Delete a class-property pair Select a class-property pair and click Remove. The pair is deleted from the list box. Edit a class-property pair Select a class-property pair and click Edit. The class and property appear in the text fields at the bottom of the dialog and the Add pushbutton becomes Replace. Modify the class, property, or both, and click Replace. The class-property pair appears in the list box.
When youre finished, click OK. The Edit Property Set dialog closes and the Property Set dialog is once again visible.
5
How to delete a property set
Click OK to close the Property Set dialog.
Procedure To delete a property set: 1 Select Options/Property Sets, or the Define pushbutton on the Verify Window dialog. The Property Sets dialog opens. 2 Select the name of the property set you want to delete from the Property Sets list box and then click Remove. You are prompted to confirm the deletion. 3 4 Click Yes to delete the property set. Click Close to dismiss the dialog.
Users Guide
125
Verifying an objects bitmap

A bitmap is a picture of some portion of your application. Verifying a bitmap is usually only useful when the actual appearance of an object needs to be verified to validate application correctness. For example, if you are testing a drawing or CAD/CAM package, a testcase might produce an illustration in a drawing region which you want to compare to a baseline. Other possibilities include the verification of fonts, color charts, and certain custom objects. Bitmap tips When comparing bitmaps, keep the following in mind: Bitmaps are not portable between GUIs. The formats are:
Platform File Format
Macintosh PC Class OSF/Motif
.PICT .BMP .XPM
A bitmap comparison will fail if the image being verified does not have the same screen resolution, color, window frame width, and window position when the testcase is run on a different machine than the one on which the baseline image was captured. Make sure that your testcase sets the size of the application window to the same size it was when the baseline bitmap was captured. Capture the smallest possible region of the image so that your test is comparing only what is relevant. If practical, do not include the windows frame (border), since this may have different colors and/or fonts in different environments.
Procedure To verify the GUI objects appearance using a bitmap: 1 Select Record/Testcase to begin recording a testcase (or select Record/ Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify. When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, then press Ctrl+Alt. The Verify Window dialog appears over your application window. 3 4 Display the Bitmap tab. Select the region to update: Entire Window, Client Area of Window (that is, without scroll bar or title bar), or Portion of Window.
126
Users Guide
Note Client Area of Window is not supported on OSF/Motif. 5 In the Bitmap File Name field, enter the full path of the bitmap file that will be created. The default path is based on the current directory. The default file name for the first bitmap is bitmap.bmp (on OSF/Motif, it is bitmap.xpm). Use the Browse pushbutton if you need help choosing a new path or name. 6 Click OK. If you selected Entire Window or Client Area of Window, the bitmap is captured and you return to your test application. If you select Portion of Window, position the cursor at the desired location to begin capturing a bitmap. While you press and hold the left mouse button, drag the mouse to the screen location where you want to end the capture. Release the mouse button. 7 Complete your testcase.
Bitmap Agent options Various bitmap options determine bitmap verification behavior:
Option
OPT_BITMAP_MATCH_COUNT
Description
The number of successive snapshots that must be the same for the bitmap to be considered stable. The time interval between snapshots to use for ensuring the stability of the image. The total time allowed for a bitmap to become stable The number of pixels of difference below which two bitmaps are judged equivalent. This option is not supported on OSF/Motif.
OPT_BITMAP_MATCH_INTERVAL
OPT_BITMAP_MATCH_TIMEOUT
OPT_BITMAP_PIXEL_TOLERANCE
To globally set these options for a group of scripts, use the Agent Options dialog. To locally set these options within a script or testcase, use the SetOption method, for example:
Agent.SetOption (OPT_BITMAP_MATCH_COUNT, 3)
Users Guide
127
Verifying using methods

Each class has a set of methods associated with it, including built-in verification methods. You can verify an objects state using one of these built-in verification methods or by using other methods in combination with the built-in Verify function.
Using a verification method
A classs verification methods always begin with Verify. For example, a TextField has the following verification methods; VerifyPosition, VerifySelRange, VerifySelText, and VerifyValue. Procedure To verify an object using a verification method: 1 Select Record/Testcase to begin recording a testcase (or select Record/ Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify. When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, and press Ctrl+Alt. The Verify Window dialog appears over your application window. 3 Display the Method tab. The methods for the selected class are listed on the left. Select the Include Inherited check box to see methods that the class inherits. 4 5 6 Select the verification method from the list. If the method takes arguments, they are displayed. Fill in any arguments as needed and click OK. Complete your testcase.
For example, here is a testcase that verifies that the text in the TextField Replace.FindWhat is myText. It uses the built-in verification method VerifyValue.
testcase VerifyMethodTest () TextEditor.Search.Replace.Pick () Replace.FindWhat.VerifyValue ("myText") Replace.Cancel.Click () Using the Verify function
You can use the built-in Verify function to verify that two values are equal and generate an exception if they are not. Typically, you use Verify to test something that doesnt map directly to a built-in property or method. Verify has the following syntax:
128
Users Guide
Verify (aActual, aExpected [, sDesc]) aActual aExpected sDesc The value to verify. ANYTYPE. The expected value. ANYTYPE. Optional. A message describing the comparison. STRING.
Usually, the value to verify is obtained by calling a method for the object being verified; you can use any method that returns a value. Procedure To verify an object using the Verify function: 1 Select Record/Testcase to begin recording a testcase (or select Record/ Actions if you want to record a verification statement in an existing testcase) and drive your application to the state you want to verify. When you are ready to record a verification statement, position the mouse cursor over the object you want to verify, and press Ctrl+Alt. The Verify Window dialog appears over your application window. 3 Display the Method tab. The methods for the selected class are listed on the left. Select the Include Inherited check box to see methods that the class inherits. 4 Select the method that will return the expected value; provide any needed arguments. You can specify a built-in method or a user-defined method (as long as it returns a value). Note To learn how to create custom methods, see Adding methods to a class on page 306. 5 6 7 8 Click OK. You return to the test application. Complete your testcase. Paste the testcase into your script. In the editor, wrap the Verify function around the method that returns the expected value as follows: Make the method call the first argument, specify the expected value as the second argument, and provide an error message string optionally as the third argument.
See the following for an example. Example Say you want to verify the number of radio buttons in the Direction RadioList in the Replace dialog in the Text Editor. There is no property or method you can directly use to verify this. But there is a method
Users Guide
129
for RadioList, GetItemCount, which returns the number of radio buttons. You can use the method to provide the actual value, then specify the expected value in the script, as follows: When doing the verification, position the mouse pointer over the RadioList and press Ctrl+Alt. Select the Method tab in the Verify Window dialog, and select the GetItemCount method. Click OK to close the Verify Window dialog, and complete your testcase. Paste it into a script. You now have the following script:
testcase VerifyFuncTest () TextEditor.Search.Replace.Pick () Replace.Direction.GetItemCount () Replace.Cancel.Click ()
Now use the Verify function to complete the verification statement. Change the line
Replace.Direction.GetItemCount ()
to
Verify (Replace.Direction.GetItemCount (), 2)
That is, the call to GetItemCount (which returns the number of radio buttons) becomes the first argument to Verify. The expected value, in this case, 2, becomes the second argument. Your completed script is:
testcase VerifyFuncTest () TextEditor.Search.Replace.Pick () Verify (Replace.Direction.GetItemCount (), 2) Replace.Cancel.Click ()
Verifying object attributes

Each kind of GUI object in an application has a variety of characteristics, called attributes. For example, a text field has the following attributes: Caret position, which is the current position of the text insertion cursor, in (line, column) format. For example, a value of {1,1} means that the text insertion cursor is positioned on line 1, column 1. Enabled, which is the current enabled status of the text field, either TRUE or FALSE.
130
Users Guide
Selected range, which is the beginning and ending position of the text string currently selected in the field, in (line, column) format. For example a value of {1,12,1,16} means that the selected text begins on line 1, column 12 and ends on line 1, column 16. Selected Text, which is the string that is currently selected, if any, in the text field. Text, which is the entire contents of the text field.
By recording verification statements for the values of one or more of an objects attributes, you can determine whether the state of the application is correct or in error when you run your testcases. That is: did the feature you are testing have the expected result? Note Attributes have been essentially rendered obsolete and have been replaced by properties. See Verifying using properties on page 115. Procedure To verify an attribute of a GUI object: 1 2 Select Options/Recorder. Drive your application to the test state and press Ctrl+Alt. The Verify Window dialog opens with the Attribute tab displayed.
The list box on the left shows the attributes for the current object. 3 Select an attribute from the list box. The current value of the attribute (that is, the value that exists when you are recording) is shown in the Attribute Value field.
Users Guide
131
5 DESIGNING AND RECORDING TESTCASES Recording the cleanup stage and pasting the recording
If the current value of the attribute is not the value you want to test for at runtime, edit the attribute value field. The value specified in this field is the value you expect at runtime, that is, the baseline value. Click OK to accept the attribute and its value. The Verify Window dialog is closed, and the Record Status window is opened.
The testcase will verify that the object has the attribute value selected. If not, an error is written to the results file. Verifying all attributes By selecting the Verify All Attributes check box, you can record a test that verifies the state, contents, and value of a GUI object and any objects it contains. This is commonly called a smoke test or a Level 1 test. A smoke test uses the VerifyEverything method to verify every aspect of a particular GUI object. Procedure To verify everything about a GUI object: 1 2 Select the Verify All Attributes check box. Click OK. The Verify Window dialog is closed, and the Record Status window is opened. Defining additional attributes If you need to, you can define and add your own attributes to the built-in hierarchy of GUI classes. For more information, see Defining new attributes on page 311.
Recording the cleanup stage and pasting the recording

1 After performing the verification, continue to interact with your application. This is the cleanup stage. For example, in the sample testcase, cleanup means closing the document window without saving it. 2 When you have finished recording your testcase or just want to see what you have recorded, press Done on the Record Status window. The Record Testcase window is redisplayed. The Testcase Code field contains your interactions written as 4Test code.
132
Users Guide
5 DESIGNING AND RECORDING TESTCASES Saving a script file
Review the code and take the following actions:

If Then
All the information in the Click OK. SilkTest closes the Record window is complete and what Testcase dialog and places the new testcase you want. in your script file. The 4Test code is not what you expected. Edit the code in the Testcase Code field.
The testcase name is not what Edit the name in the Testcase Name field. you want. The application state is not the one you want. Delete the code in the Testcase Code field, select a new application state from the drop-down list and click Resume Recording to rerecord the testcase. Click Resume Recording. The Record Testcase window is reopened. You can continue to record your interactions.
The testcase is not finished.
4
How the application state is recorded
Save the script file, as described in the next section.
If you chose DefaultBaseState as the application state, the testcase is recorded in the script file as:
testcase testcase_name ()
If you chose another application state, the testcase is recorded as:

testcase testcase_name () appstate appstate_name
If you chose (none) as your application state, the testcase is recorded as:
testcase testcase_name () appstate none The recording statement
When you paste a recorded testcase (or other recorded actions, such as when you use Record Actions) into a script, SilkTest indents the code under a recording statement to facilitate playback. For more information about the recording statement, see online Help.
Saving a script file

Procedure To save a script file, select File/Save. If it is a new file, you are prompted for its name and location. When saving a file, SilkTest does the following:
Users Guide
133
5 DESIGNING AND RECORDING TESTCASES Using object files
Saves a source file, giving it the .t extension; the source file is an ASCII text file, which you can edit Saves an object file, giving it the .to extension; the object file is a binary file that is executable, but not readable by you
Example If you name a script file mytests and save it, you will end up with two files: the source file mytests.t, in the location you specify, and the object file mytests.to. Procedure To save a new version of a scripts object file when the script file is in view-only mode, select File/Save Object File.
About object files
Object files are described in the next section.
Using object files

In order for SilkTest to run a script or include file that is in source form, it must compile it, which can be time-consuming. Object files, on the other hand, are ready to run. SilkTest can read and write object files for scripts and include files. When you save a script or include file, a source file and an object file are saved. Object files are not platform-specific; you can use them on all platforms that SilkTest supports. The object file for a script is saved with a .to extension, for an include file with a .ino extension.
Advantages of object files
Object files have these advantages: Because object files are ready to run, they dont need to be recompiled if the source file hasnt changed. This can save you a lot of time. If your object file is more recent than your source file, the source file does not need to be recompiled each time the file is first opened in a session; the object file is used as is. You can distribute object files without having to distribute the source equivalents. So if you have built some tests (and include files) that you want to distribute but dont want others to see the sources, you can distribute only the object files. Since an object file cannot be run directly, you need to define the code you want to hide in an include file, which will be compiled into a .ino object file, and then call those functions from an ordinary script file. You then distribute the .t script file and the compiled .ino include file. Users can open and run the script file as usual, through File/Run.
134
Users Guide
5 DESIGNING AND RECORDING TESTCASES Using object files
Example Heres a simple example of how you might distribute object files so that others cannot see the code. In file test.inc, place the definition of a function called TestFunction. (When you save the file, the entire include file is compiled into test.ino.)
TestFunction () ListPrint (Desktop.GetActive ())
In the file test.t use the test.inc include file. SilkTest will load the .ino equivalent. Call TestFunction, which was defined in the include file
use "test.inc" main () TestFunction () // call the function
Distribute test.t and test.ino. Users can open test.t and run it but do not have access to the actual routine, which resides only in compiled form in test.ino.
How object files are used
Object files are always used if they are available. When you open a script file or an include file, the corresponding object file is loaded as well, if there is one. If the object file is not older than the source file, the source file will not be recompiled; the script is ready to run. If the source file is more recent, the source file is recompiled before the script is run. If you then later save the source, a new object file will be saved automatically. If a file is loaded during compilation (that is, if a file is included in another file that is being compiled), only the object file will be loaded, if it exists and is newer than the corresponding source file.
Where object files are located
By default, an object file is read from and written to the same directory as its corresponding source file. But you can specify different directories for object files. Procedure To specify where object files should be written to and read from: 1 2 Select Options/Runtime. The Runtime Options dialog is displayed. Specify a directory in the Objfile Path field, as follows: Leave the field empty if you want to store object files in the same directory as their corresponding source files Specify an absolute path if you want to store all object files in the same directory Specify a relative path if you want object files to be stored in a directory relative to the directory containing the source files
Users Guide
135
5 DESIGNING AND RECORDING TESTCASES Recording from within a testplan
Click OK.
Object files will be saved in the location you specify here. In addition, SilkTest will try to find object files in these locations. If it fails to find an object file, it will look in the directory containing the source file. Examples Specifying d:\obj in the Objfile Path field tells SilkTest to read and write all object files in the d:\obj directory, regardless of where the source files are located. Specifying obj in the Objfile Path field tells SilkTest to read and write an object file in the directory obj that is a subdirectory of the directory containing the source file. In this scenario, each directory of source files will have a different directory of object files. For example, if a source file is in d:\src, its corresponding object file would be read from and written to d:\src\obj. Specifying multiple directories You can specify several directories in the Objfile Path field if you want. New files will be written to the first directory specified. SilkTest will search the directories in the order that you specified them to find existing files and will subsequently resave existing files in the same directory where it found them.
Recording from within a testplan

You can record a testcase while writing a testplan. The advantage of doing so is that the testplan editor automatically inserts the script and testcase statements into the plan once the recording is finished, linking the plan to the 4Test code. Procedure To link to a script and testcase by recording a testcase: 1 Place the insertion cursor at the end of a test description or a group description. For example, the following figure shows how to position the cursor to link the Case sensitive, forward, character search test description:
136
Users Guide
5 DESIGNING AND RECORDING TESTCASES Recording from within a testplan
Select Record/Testcase. The Record Testcase Script dialog prompts you to name a script file to contain the testcase. Note You wont be prompted for a script file if there is a script defined at a higher level and inherited by the testcase you are now recording. If there is, the testcase will be put in that script.
If prompted, select an existing script from the list or enter the name of a new script in the File Name field, then click OK. SilkTest displays the Record Testcase dialog. Enter the name for the testcase and optionally select an application state to be run before the recording starts. Click the Start Recording pushbutton. The Recording Status dialog displays. The dialog flashes the word Recording for the duration of the session.
4 5
When finished recording the actions that comprise the testcase, click the Done pushbutton in the Recording Status dialog. The status dialog closes, and the Record Testcase dialog reappears. Click the Paste to Editor pushbutton. The Record Testcase dialog closes, the testcase is inserted into the script file, and the script and testcase statements are added to the testplan on a new line and are indented appropriately. Note If the script file is inherited by the testcase you are recording, only the testcase statement is pasted.
Users Guide
137
5 DESIGNING AND RECORDING TESTCASES How recorded commands uniquely identify GUI objects
How recorded commands uniquely identify GUI objects

Fully qualified identifiers
When you record testcases, SilkTest uses the window declarations in the test frame file to construct a unique identifier, called a fully qualified identifier, for each GUI object. The fully-qualified identifier consists of the identifier of the object, combined with the identifiers of the objects ancestors. In this way, the 4Test commands that are recorded can manipulate the correct object when you run your testcases. If all identifiers were unique, this would not be necessary. However, because it is possible to have many GUI objects with the same identifier (for example, the ever-present OK pushbutton), a method call needs to specify as many of the objects ancestors as are required to uniquely identify it. The following table shows how fully qualified identifiers are constructed:
GUI object Fully-qualified identifier Example
Main Window The main windows identifier TextEdit.SetActive () Dialog Control Menu item The dialogs identifier The identifiers of the dialog and the control The identifiers of the main window, the menu, and the menu item Find.SetActive () Find.Cancel.Click () TextEditor.File.Open.Pick ()
The fully qualified identifier for main windows and dialogs does not need to include ancestors because the declarations begin with the keyword window.
Recording without window declarations

If you record a testcase against a GUI object for which there is no declaration (or if you want to write a testcase from scratch against such an object), SilkTest requires a special syntax to uniquely identify the GUI object because there is no identifier.
Dynamic instantiation
This special syntax is called a dynamic instantiation and is composed of the class and tag of the object. For example, if there is not a declaration for the Find dialog of the Text Editor application, the syntax required to identify the object looks like this:
MainWin("Text Editor|$D:\MYPROGS\TEXTED2.EXE").DialogBox("Find")
138
Users Guide
5 DESIGNING AND RECORDING TESTCASES Recording 4Test components
The general syntax of this kind of identifier is: class(tag).class(tag). ... Note To create the dynamic tag, the recorder uses the multiple-tag settings that are stored in the Record Window Declarations dialog. In the example shown above, the tag for the Text Editor contains its caption as well as its window ID. For more information, see About tags on page 84.
Recording 4Test components

If you want to hand-write some or most of your 4Test code, or if you want to add individual lines to an existing testcase, you can use the following recording tools:
Three additional recording tools
Record/Actions Record/Window Identifiers Record/Window Locations Record/Class
Record/Actions
For example, when youre working with a script, you might want to leave the Record Actions dialog open. Any time you want to verify a GUI object, you can point to the object in your application and verify it. You can also use the dialog to write a syntactically correct 4Test statement based on your manual interaction with your application. This eliminates the need to search through the documentation for the correct method and its arguments. Once the statement is recorded, the Paste to Editor button inserts the statement to your script. For reference information on Record/Actions, see page 488.
Record/Window Identifiers
Similar to the Actions command, Record/Window Identifiers records the fully qualified name of the GUI object youre pointing at, which you can then insert into your script. This eliminates the need to bring up your test frame file to find the correct identifier for the object. For reference information on Record/Window Identifiers, see page 499. It can be useful to know the position of certain objects, for example objects that are drawn (like tools on a toolbar) or drawing regions (in a CAD/CAM package, for example). To record the location of an object, use the Record/ Window Locations commands. For reference information on Record/ Window Locations, see page 501.
Record/Window Locations
Users Guide
139
5 DESIGNING AND RECORDING TESTCASES Recording 4Test components Record/Class
If you are using ActiveX, Visual Basic, or Java classes (controls) that are not predefined, you can record the classes for use in your tests. For more information about recording classes, see online Help.
140
Users Guide
6e rh C t p a
Running Tests and Interpreting Results

You can run testcases individually, as a script, as a group of scripts (called a suite), or from a testplan if you are running the testplan editor. Whenever you run tests, SilkTest generates a results file, which indicates how many tests passed and how many failed, describes why tests failed, and provides summary information. You can invoke comparison tools from within the results file that pinpoint exactly how the runtime results differ from your known baselines. Testplan results files offer additional features, such as the ability to generate a Pass/Fail report or compare different runs of the testplan. This chapter contains the following sections:
Topic Page
Introduction
What you will learn
Preparing to run tests Running tests Results files Using the results file to find errors Fixing errors in a script Managing results file information
141 144 148 152 155 156
Preparing to run tests

If you plan to test Web applications, make sure you enable the correct browser extension(s) on your target and host machine. For more information, see Chapter 3, Enabling Extensions for Applications Under Test and Getting Started: A Tutorial.
Users Guide
141
6 RUNNING TESTS AND INTERPRETING RESULTS Preparing to run tests
If you do not plan to test Web applications, you must disable all browser extensions on the host machine.

By default, SilkTest enables all the browser extensions on your target machine during the installation procedure. To change the default settings or verify your current settings, you must invoke a utility called the Extension Enabler, as described in the following procedure. Note If you are running local teststhat is, your target and host are the same machineyou must still ensure that browser extensions are enabled appropriately by running the Extension Enabler on the host machine. Procedure To enable support for browsers on a target machine: 1 From the SilkTest program group, click Extension Enabler. The Extension Enabler dialog opens, as in this example:
2 3
Enable support for the browsers you want to use, if they arent already enabled. In the Primary Extension field for a browser, choose Enabled. Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Click OK to close the Extension Enabler dialog.
4
You enable support for browsers on the host machine using the Extensions dialog. Be advised that there is overhead associated with having more than one browser extension enabled, so do so only if you are actually testing more than one browser in an automated session.
142
Users Guide
6 RUNNING TESTS AND INTERPRETING RESULTS Preparing to run tests
Procedure To enable support for browsers on a host machine: 1 2 Start SilkTest. Select Options/Extensions. The Extensions dialog opens. All the installed extensions are listed, including the browser extensions, as in this example:
Enable the browser extension(s) you want to use. In the Primary Extension field for a browser, choose Enabled. Disable any browser extensions you do not plan to use by choosing Disabled in the Primary Extension field and unchecking their other extensions. Select any other extensions that the browser needs for testing such as ActiveX or fault trapping. We recommend that you do not turn on fault trapping until you really need it. Note If you enable support for ActiveX in this dialog, you must make sure that it is enabled in the Extension Enabler dialog as well.
Click OK to close the Extensions dialog.
You can get information about the files used by an extension by selecting an extension and clicking the Details pushbutton.

If you are testing non-Web applications, you must disable browser extensions on your host machine. This is because the recovery system works differently when testing Web applications than when testing non-Web applications. For more information, see Getting Started: A Tutorial.
Users Guide
143
6 RUNNING TESTS AND INTERPRETING RESULTS Running tests
Procedure To disable support for browsers on a host machine: 1 2 3 Start SilkTest. Select Options/Extensions. The Extensions dialog opens. Disable any browser extensions you do not plan to use by choosing Disabled in the Primary Extension field and unchecking all check boxes in the Other Extensions columns. Click OK to close the Extensions dialog.
Running tests
You can run testcases individually, as a script, as a group of scripts (called a suite), or from a testplan if you are running the testplan editor.
Preparing to run tests
If you plan to test Web applications, make sure you enable the correct browser extension(s). If you do not plan to test Web applications, you must disable all browser extensions. This is because the recovery system works differently when testing Web applications than when testing non-Web applications. See Enabling browser extensions on page 142 and Disabling browser extensions on page 143 for more information. For information on how SilkTests recovery system works for web applications, see Getting Started: A Tutorial.
Running a testcase
Procedure To run a testcase: 1 2 Make sure that the testcase you want to run is in the active window. Select Run/Testcase. The Run Testcase dialog appears. The Testcase listbox displays all the testcases contained in the current script.
144
Users Guide
Select a testcase, specify arguments (if necessary) in the Arguments field, separating multiple arguments with commas, and click Run. SilkTest runs the testcase and generates a results file.
Running testcases on platforms that do not support multiple tags
Multiple tags are supported by Windows 95, Windows 98, and Windows NT Agents. If you are running testcases using other Agents (such as on Motif), you can run scripts that use declarations with multiple tags by doing one of the following: Checking the Disable Multiple Tag Feature check box in the Agent Options dialog (Compatibility tab). Specifying the following in your script:
Agent.SetOption (OPT_MULTIPLE_TAGS, FALSE)
After you have turned off multiple-tag support, 4Test discards all segments of a multiple tag except the first one.
Stopping a running testcase
Procedure To stop a running testcase before it completes: If your test application is on a target machine other than the host machine, select Run/Abort. If your test application is running on your host machine, press Shift+Shift.
Grouping scripts in a suite
After you have created a number of script files, you might want to collect them into a test suite. A suite is a file that names any number of scripts. Instead of running each script individually, you run the suite, which executes in turn each of your scripts and all the testcases they contain. Suite files have a .s extension. Procedure To create a suite: 1 Select File/New. The New dialog is displayed.
Users Guide
145
2 3
Select the Suite radio button and click OK. An untitled suite file opens Enter the names of the script files in the order you want them executed. For example, the following suite file executes the find.t script first, the goto.t script second, and the open.t script third.
find.t goto.t open.t
4
Running a script or suite
Select File/Save to save the file.
Procedure To run the currently active script or suite: 1 2 Make sure the script or suite you want to run is in the active window. Select Run/Run. SilkTest runs all the testcases in the script or suite and generates a results file. Procedure To run a script or suite that is not currently open: 1 2 3 Select File/Run. Select the script or suite name from the Run dialog and browse the directories to select the name of the script or suite file you want to run. Click OK. SilkTest runs all the testcases in the script or suite and generates a results file.
Running a testplan
SilkTest provides two commands for running a testplan. They are Run All Tests and Run Marked Tests. For information on how to mark tests, see Marking a testplan on page 243. Note You can also run a single testcase without marking it. See the Run/Testcase command, which is described on page 144. Prerequisites Before running a testplan, make sure that The window declarations file for the testplan is correctly specified in the Runtime Options dialog. The testplan is in the active window.
Procedure To run the entire testplan, select Run/Run All Tests. SilkTest runs each testcase in the plan and generates a results file.
146
Users Guide
Procedure To run marked tests, select Run/Run Marked Tests. SilkTest runs each marked test and generates a results file. How subplans are handled If your testplan is structured as a master plan and associated subplans, SilkTest automatically opens any closed subplans before running. For more information on using subplans, see Dividing a testplan into a master plan and subplans on page 259.
About saving files
SilkTest always saves the suite, script, or testplan before running it if you made any changes to it since the last time you saved it. By default, SilkTest also saves all other open modified files whenever you run a script, suite, or testplan. To prevent this automatic saving of other open modified files, deselect the Save Files Before Running check box in the General Options dialog. See Options/General... on page 477.
Passing arguments to a script
You can pass arguments to a script. For example, you might want to pass in the number of iterations to perform or the name of a data file. All functions and testcases in the script have access to the arguments. Passing arguments There are three ways to pass arguments to a script: You can specify them in the Arguments field in the Runtime Options dialog (Options/Runtime from the menu bar). Note The Arguments field in the Run Testcase dialog is used to pass arguments to a testcase, not to an entire script. You can specify them in a suite file after a script name, such as:
find.t arg1 arg2
You can provide arguments when you invoke SilkTest from the command line. See Chapter 26, Command Line Syntax.
All arguments are passed in as strings, separated by spaces, such as:

Bob Emily Craig
If an argument is more than one word, enclose it with quotation marks. For example, the following passes in three arguments:
"Bob H" "Emily M" "Craig J"
Processing arguments You use the GetArgs function to process arguments passed into a script. GetArgs returns a list of strings, with each string being one of the passed arguments. Any testcase or function in a script can call GetArgs to access the arguments.
Users Guide
147
6 RUNNING TESTS AND INTERPRETING RESULTS Results files
Examples The following testcase simply prints a list of all the passed arguments:
testcase ProcessArgs ( ) LIST OF STRING lsArgs lsArgs = GetArgs ( ) ListPrint (lsArgs)
You can also process the arguments individually. The following testcase prints the second argument passed:
testcase ProcessSecondArg ( ) LIST OF STRING lsArgs lsArgs = GetArgs ( ) Print (lsArgs[2])
The following testcase adds the first two arguments:

testcase AddArgs () LIST OF STRING lsArgs lsArgs = GetArgs ( ) NUMBER nArgSum nArgSum = Val (lsArgs[1]) + Val (lsArgs[2]) Print (nArgSum)
Note that the Val function was used to convert the arguments (which are always passed as strings) into numbers. Specifying arguments 10 20 30 results in the following:
Script scr_args.t (10, 20, 30) - Passed Passed: 1 test (100%) Failed: 0 tests (0%) Totals: 1 test, 0 errors, 0 warnings Testcase AddArgs - Passed 30
Results files
When you run a testcase, script, suite, or testplan, SilkTest creates a results file. The following figure shows a results file for a script (find.t). The menu bar includes the Results menu, which allows you to manipulate the results file and locate errors. The Results menu appears only when the active window displays a results file
148
Users Guide
Definition of results file
A results file provides information about the execution of the testcase, script, suite, or testplan. By default, the results file has the same name as the executed script, suite, or testplan, but with a .res extension (for example, find.res). To change the default name and directory of the results file, edit the Runtime Options dialog. For more information, see Options/Runtime... on page 482.
Overall summary
By default, the results file displays an overall summary at the top of the file, including the name of the script, suite, or testplan; the machine the tests were run on; the number of tests run; the number of errors and warnings; actual errors; and timing information. The overall summary for find.t is shown in the preceding figure. Procedure To hide the overall summary, click on the summary and select Results/Hide Summary. (You can change the default behavior by deselecting the Show Overall Summary check box in the Runtime Options dialog. See page 482.)
Individual test summary
You can see a summary for an individual test in a results file. For a script or suite results file, the individual test summaries contain timing information and errors or warnings. For a testplan results file, the individual test summaries contain the same information as in the overall summary plus the name of the testcase and script file. Procedure To see an individual summary: 1 2 Click on a testcase line in a suite or script results file, or click on a test description in a testplan results file. Select Results/Show Summary. The following figure shows the individual summary for a test in a testplan results file.
Users Guide
149
Hierarchical format of testplan results file
The format for the rest of a testplan results file follows the hierarchy of test descriptions that were present in the testplan. Test statements in the testplan that are preceded by a pound sign (# ) as well as comments (using the comment statement) are also printed in the results file, in context with the test descriptions.
Results file colors

Default colors
By default, these results file elements are displayed in the following colors on color monitors:
Results file element Default color/icon
Error messages and warnings Warnings only Test descriptions of executed tests Test descriptions of unexecuted tests Other descriptive lines
Red plus sign (bold on black-and-white monitor) Purple plus sign Dark blue Grayed out Black
150
Users Guide
6 RUNNING TESTS AND INTERPRETING RESULTS Results files Customizing colors
You can modify the colors for these elements in the results file in the Editor Colors dialog. For more information, see Using color to indicate structure on page 54.
Multiple sets of results

By default, the results file displays the most current execution of the script, suite, or testplan. However, by default, SilkTest saves the last five sets of results for each script, suite, or testplan executed. To change the default number of results sets that SilkTest keeps, edit the History Size field in the Runtime Options dialog, described on page 484.
Displaying another results set
Procedure To display a different set of results: 1 Select Results/Select. The Select Results dialog appears. The most current results set is displayed first. 2 Select the set of results you want to see and click OK.
Adding comments to a results set
You can attach comments to individual results sets to record useful information about the test run. Procedure To attach a comment to a result set: 1 2 3 Open the results file. Select Results/Select. The Select Results dialog displays. Select the results set you want to attach a comment to and type its comment in the Comment text field at the bottom of the dialog. The comment appears in the Comment column in the list of result sets. 4 Click OK. The comments display in the various dialogs that list results sets (such as the Extract Results and Delete Results dialogs).
Deleting a results set
Procedure To delete a set of results: 1 Select Results/Delete. The Delete Results dialog appears. The most current results set is displayed first. 2 Select the set of results you want to delete and click OK.
Users Guide
151
6 RUNNING TESTS AND INTERPRETING RESULTS Using the results file to find errors
Using the results file to find errors

You can expand the text of an error message yourself or have SilkTest find the error messages for you.
Clicking on the red plus sign
Procedure To expand an error message to reveal the cause of an error, click on the red plus sign preceding the message. In addition, you see the call stack, which is the list of 4Test functions executing at the time the error occurred. In the following example of a results file, the error occurred in VerifySelText, which was called from Case_Back_Char on line 39 of find.t.
Using Find Error
Procedure To find and expand the next error or warning message in the results file, select Edit/Find Error. Tip To skip warning messages and find error messages only, deselect Find Error Stops at Warnings check box in the Runtime Options dialog. You can also use the Find, Find Next, and Go to Line commands on the Edit menu to navigate through a results file.
Navigating to errors in the script
SilkTest provides several ways to move from the results file to the actual error in the script: Double-click in the margin next to an error line to go to the script file that contains the 4Test statement that failed. Click on an error message and select Results/Goto Source.
152
Users Guide
6 RUNNING TESTS AND INTERPRETING RESULTS Using the results file to find errors
Navigating to errors in the testplan What the box icon means
Click on an error message and press Enter.
To navigate from a testplan test description in a results file to the actual test in the testplan, click on the test description and select Results/Goto Source. Some expanded error messages are preceded by a box icon and three asterisks, as shown in the following figure.
What happens when you click on the box icon depends on the error message. If the error message relates to an applications appearance, as in Bitmaps have different sizes, SilkTest opens the bitmap tool for your platform. If the error message relates to an applications behavior, as in Verify selected text failed, SilkTest opens the Difference Viewer.
The bitmap tool compares baseline and results bitmaps, whereas the Difference Viewer compares actual and expected values for a given testcase.
Finding application appearance errors

When you click on a box icon followed by a bitmap-related error message, SilkTest starts the bitmap tool, reads in the baseline and result bitmaps, and opens a Differences window and Zoom window.
Users Guide
153
6 RUNNING TESTS AND INTERPRETING RESULTS Using the results file to find errors Bitmap tool
The following figure shows the bitmap tool. The baseline bitmap for the Font dialog in the Text Editor is compared to the result bitmap, which shows the Font dialog when the testcase was run. The Differences window shows the area of difference between the two bitmaps, that is, the different selection for Font Style.
The baseline bitmap shows Regular as the font style.
The Differences window shows the differences between the baseline and result bitmap The result bitmap shows Bold as the font style.
The bitmap tool supports several comparison commands, which let you closely inspect the differences between the baseline and results bitmaps. These commands are described in Comparing bitmaps on page 412. In particular, see Zooming in on the differences on page 414.
Finding application logic errors

To evaluate application logic errors, SilkTest provides the Difference Viewer, which you can invoke by clicking on the box icon following an error message relating to an applications behavior.
Difference Viewer
Clicking on the box icon invokes the Difference Viewers double-paned display-only window (shown in the following figure). It lists every expected (baseline) value in the left pane and the corresponding actual value in the right pane. SilkTest highlights all occurrences where expected and actual values differ. On color monitors, differences are marked with red, blue, or green lines, which denote different types of differences, for example, deleted, changed, and added items. The highlight is not visible in the black-and-white
154
Users Guide
6 RUNNING TESTS AND INTERPRETING RESULTS Fixing errors in a script
figure below; however, in reviewing the text, you can see, for example, that SilkTest expects the Case Sensitive check box to be unchecked (FALSE), while the actual value found during testcase execution was checked (TRUE). When you have more than one screen of values or are using a black-andwhite monitor, use Results/Next Result Difference to find the next difference. Use Update Expected Values, described next, to resolve the differences.
Fixing errors in a script

Updating expected values
You might notice upon inspecting the Difference Viewer or an error message in a results file that the expected values are not correct. For example, when the caption of a dialog changes and you forget to update a script that verifies that caption, SilkTest will log errors when you run the testcase. To have your testcase run cleanly the next time, you can modify the expected values with the Update Expected Value command. Note that this command updates data within a testcase, not data passed in from the testplan. (For more information on testplan data, see Chapter 10, Adding Data to a Testplan.) Procedure To fix incorrect values in a script: 1 2 Make the results file active. Select Results/Update Expected Value. SilkTest replaces the expected values in the script with the actual values found at runtime. 3 Optionally, select Run/Testcase in order to run the test and confirm that it now passes.
Users Guide 155
6 RUNNING TESTS AND INTERPRETING RESULTS Managing results file information Debugging tools
You might need to use the debugger to explore and fix errors in your script. In the debugger, you can use the special commands available on the Breakpoint, Debug, and View menus. For more information, see Chapter 7, Using the Debugger.
Marking failed testcases
When a testplan results file shows testcase failures, you might choose to fix and then rerun them one at a time. You might also choose to rerun the failed testcases at a slower pace, without debugging them, simply to watch their execution more carefully. Procedure To have SilkTest identify the failed testcases, select Results/ Mark Failures in Plan when the results file is active. SilkTest marks all the failed testcases and makes the testplan the active file.
Managing results file information

You can: Store the results in an unstructured ASCII text file for printing or viewing Export the results to a structured file for further processing Send the results to SilkRadar for further processing Merge two testplan results files Compare two different testplan results files Generate a Pass/Fail report from a testplan results file Compact a results file
Storing results in an unstructured ASCII format

As a QA engineer, you might need to prepare a status report based on one or more sets of results or you might want to print the errors you found after a day of testing. SilkTest allows you to extract the information you want in an unstructured ASCII text format and send it to a printer, store it in a file, or look at it in an editor window. Note To learn how to save your results in a structured text file that an application such as a spreadsheet can process, see the next section.
156
Users Guide
6 RUNNING TESTS AND INTERPRETING RESULTS Managing results file information
Procedure To store results in unstructured ASCII format: 1 Select Results/Extract. The Extract Results dialog appears, as shown in the following figure.
2 3
Select a radio button in the Extract To group box indicating the destination of the extracted output: Window (default), File, or Printer. Select one or more check boxes in the Include group box indicating which optional text, if any, to extract. (This optional text is in addition to the output selected in the Expand group box.) The choices are: Output Text, which is the output of print statements. Error Text, which includes text generated by the LogError and ExceptLog functions, such as exactly where the error occurred and a full description. Summary Text, which is a brief description of which tests passed and failed and the number of errors produced by failed tests.
Select a radio button in the Expand group box indicating which units to extract information about. Select Scripts, Scripts and Testcases (default), or Anything with Errors. Select one or more results sets from the Results to Extract group box. Click OK.
If the destination is Then
5 6
Window File
The extracted results are displayed in the active window Specify a directory in the Extract to File dialog. By default the text file has the same name as the script and a .txt extension
Users Guide
157
If the destination is
Then
Printer
Specify the margins, headers, and footers in the Extract to Printer dialog.
Exporting structured information

You can export the results to a structured (delimited) text file that is suitable for importing into a spreadsheet or other application for further processing. Note For information on exporting results to SilkRadar, see the next section. Procedure To export results to a structured file for further manipulation: 1 2 3 Select Results/Export. The Export Results dialog appears. Specify which fields you want to export to the file. Specify how you want the fields delimited in the file. The default is to comma delimit the fields and put quotations marks around strings. You can pick another built-in delimited style listed in the Export Format dropdown list box or select Custom and specify your own delimiters. Select the Write Header check box to include header information in the file that includes the name of the results file, which fields were exported, and how the fields were delimited. Specify which results sets you want to export. The default is the results set that is currently displayed in the results window. Specify the file name. By default, SilkTest suggests the name results-file.rex (for results export). Click OK. SilkTest saves the information in a delimited text file. You can import that file into an application that can process delimited files, such as a spreadsheet.
5 6 7
Sending results to SilkRadar

SilkRadar is the defect-tracking product from Segue that you can use to create and manage bug reports, enhancement requests, and documentation issues for your application.
158
Users Guide
SilkRadar is integrated with SilkTest. You can associate individual SilkTest tests with defects stored in SilkRadar and have SilkRadar process the defects based on the results of the tests. You can pass your test results to SilkRadar in two ways: Send the results directly to SilkRadar. This is the easiest way to pass the results if you are running both SilkRadar and SilkTest. Export the results to a .rex file for importing later in SilkRadar.
Procedure To send the results directly to SilkRadar: 1 2 3 4 Start SilkRadar. In SilkTest, select Results/Send to SilkRadar. The Send Results to SilkRadar dialog appears. Specify the results set you want to send. Click OK. SilkTest activates SilkRadar and opens the SilkTest Results Import dialog, passing the results for further processing. For more information about processing SilkTest results in SilkRadar, see the SilkRadar documentation. Procedure To export results to SilkRadar using a .rex file: 1 Select Results/Export as described in Exporting structured information on page 158. The Export Results dialog appears. 2 3 Specify the file to export the results to. You must use the .rex extension. Specify the following information, which is needed by SilkRadar:
In this category Do this
Fields to Export
Select these fields: Testplan, Script, Testcase, Test Data, Error Count, Error Text, Start Date/Time, Elapsed Time. Select Comma Delimited, Quoted Strings. Select Write Header.
Export Format
4 5
Select the results set to export. Click OK. The .rex file is created.
Users Guide
159
In SilkRadar, import the .rex file.
For more information about processing SilkTest results in SilkRadar, see the QA Radar documentation.
Merging testplan results sets

Results files consist of a series of results setsone set for each testplan run. You can merge different results sets in a results file. Merging results sets is useful when: Sections of the testplan are run separately (either by one person or by several people) and you need to create a single report on the testing process. That is, you want one results set that includes the different runs. The testplan is updated with new tests or subplans and you want a single results set to reflect the execution of the additional tests or subplans.
Procedure To merge two results sets in a results file: 1 2 3 Open the results file. Use Results/Select to open the results set into which you want to merge another results set. Select Results/Merge. The Merge Results dialog is displayed. All the results sets in the current file are listed, except the set that is currently open.
4
What happens
Select the results set you want to merge into the currently opened results set and click OK.
SilkTest combines the two results sets by merging the results set you selected in the Merge Results dialog into the currently open results set. The open results set is altered; no additional results set is created. The date and time of the altered results set reflect the more recent test run.
160
Users Guide
For example, say you ran a section of the testplan consisting of 20 tests yesterday and ran a different section of the testplan consisting of 10 tests today. The merged results set would have today's date and would consist of the results of 30 tests. Note SilkTest uses the test descriptions as well as the script, testcase, and testdata statements to identify and locate the various cases in the testplan and in the results set. When test results overlap in the two results set that were merged, the more recent run is used. If you change a test description between runs or modify the statements, SilkTest might be unable to find the test when you try to merge results. SilkTest places these so-called orphaned tests at the top of the results set.
Comparing different runs of a testplan

Compare Two Results command
The Compare Two Results command allows you to quickly note only the results that have changed from a prior run, without having to look at the same errors over again. The command identifies differences based on the following criteria: A test passes in one testplan run and fails in the other A test fails in both runs but the error is different A test is executed in one testplan run but not in the other Note SilkTest uses the test descriptions as well as the test statements to identify and locate the various cases in the testplan. Therefore, if you change the descriptions or statements between runs, SilkTest will not be able to find the test when you run Compare Two Results. Procedure To find differences between testplan executions: 1 2 Make the results set you want to compare to another results set the active window. Select Results/Compare Two Results. The Compare Two Results dialog appears.
Users Guide
161
Select a results set from the list box and click OK. When the results set reappears, a colored arrow is positioned in the left margin for every test that is different. A red arrow indicates that the difference is due to the pass/fail state of the test changing. A magenta arrow indicates that the difference is due to the addition or removal of the test in the compared test run.
Select Results/Next Result Difference to search for the next difference or select Results/Next Error Difference to search for the next difference that is due to the change in a pass/fail state of a test.
Generating a testplan Pass/Fail report

Introduction
A Pass/Fail report lists the number and percentage of tests that have passed during a given execution of the testplan. The report can be subtotaled by an attribute, for example, by Developer. (For more information on attributes, see Chapter 11, Categorizing and Marking Testplans.) You can optionally: Print the report. Chart the report. Export the report to a comma-delimited ASCII file
How to generate a Pass/Fail report
Procedure To generate a Pass/Fail report on the active results file: 1 2 Make sure the testplan results file you want to report on is active. Select Results/Pass/Fail Report. The Pass/Fail Report dialog appears, as shown in the following figure.
162
Users Guide
3 4
If you want, select an attribute to report on from the Subtotal by Attribute drop-down list. Click Generate. The report is displayed in the list box. The following figure shows a Pass/Fail report subtotaled on the Developer attribute.
Take one of the following actions:

If you want to Then
Subtotal the report by a different attribute
Select a different attribute in the Subtotal By Attribute drop-down list box, then click Generate.
Users Guide
163
If you want to
Then
Print the report
Click Print. The Print Pass/Fail Report dialog appears. You can set the margins, headers and footers, print quality, and fonts for the report. To change the font, click Font. To change the printer setup, click Setup. When you have finished setting these options, click OK to print the report.
Chart the report
Display the Chart tab. See Generating a testplan Pass/Fail chart on page 164 for more information. Click Export. The Export Pass/Fail Report dialog appears. Specify the full path of the file and click OK.
Write the report to a comma-delimited ASCII file
You can open the file in a spreadsheet application that accepts comma-delimited data.
Including results of manual tests
You can mark manual tests as having passed or failed in the Update Manual Tests dialog (see Documenting manual tests in the testplan on page 65). The Pass/Fail report includes in its statistics the manual tests that you have documented as having passed or failed. By default, whenever you generate a report, it will include information on the tests run for that results file, plus the current results of any manual tests specified in the testplan. If the manual test results are subsequently updated, the next time you generate the report, it will incorporate the latest manual results, which might not be what you want. Instead, you might want the report to use a snapshot of manual results, not the most recent manual results. You do this by merging the results of manual tests into the results file. Procedure To merge results of manual tests: 1 2 3 Make the results file for the testplan active. Select Results/Merge. The Merge Results dialog displays, listing all other runs of your testplan, plus the results of your manual tests.
164
Users Guide
Select Results of manual tests and click OK. The latest statistics for your manual tests are merged into the results file. If you later generate a report using these results, the merged manual results will be used, not the manual results that are current.
Generating a testplan Pass/Fail chart

You can chart a generated Pass/Fail reportthat is, produce report information as a graph, or you can directly graph the testplan results information without a preexisting report.
How to generate a Pass/Fail chart
Procedure To produce a Pass/Fail chart: 1 Select the Chart tab. If you have already generated a report, a chart of the generated report is displayed, as shown in the following figure.
Users Guide
165
You might need to resize the window so there is enough room to display the chart well. If you have not generated a report, as described in Generating a testplan Pass/Fail report on page 162, a default chart is displayed, which allows you to modify chart parameters before actually generating the chart.
Take one of the following actions:

If you want to Then
Change basic Click Setup. The Chart Settings dialog is displayed. charting properties
To change the chart type, select an option from the Chart Type drop-down list box. SilkTest provides bar charts, line charts, and area charts. Click Apply to update the chart and leave the Chart Settings dialog open. You can also choose whether the chart is threedimensional, is stacked (for bar charts), and displays a legend, which describes the data being charted. SilkTest displays a model that represents how the chart will look based on current settings. Add the results from another execution of the testplan to the chart Click Select. The Select Results dialog is displayed, listing recent runs of the current testplan. Note SilkTest keeps a history of results for each testplan. The number of results it keeps is determined by the value for History Size in the Runtime Options dialog. Select the results you want to add to the chart. The results from the selected execution of the testplan will be added to the results currently charted. You can use this feature to compare two different runs of the same tests to spot problem areas. You can chart todays results, then click Setup and select yesterdays results to have both appear on one chart.
166
Users Guide
If you want to
Then
Move a part of the chart
Click the part you want to move, such as the title, legend, or footnote (the text that displays below the chart). The area is selected. Drag it with the mouse. Click Print. The Print Pass/Fail Chart dialog displays. You can specify a header or footer. Click OK to print the chart. Press the right mouse button anywhere on the displayed chart. Select Copy from the popup menu. The chart is placed on the clipboard. You can paste it into another application.
Print the chart
Copy the chart to the clipboard
Change advanced See Changing advanced properties next. charting properties Generate the chart Once you are satisfied with the chart parameters, click Generate. The Pass/Fail chart is displayed.
Changing advanced properties
Usually you can get the chart you want using the default and basic charting properties. But if you want more customization, you can modify just about any property in the chart. For example, you can: Change the text that appears for the title and footnote Change the font used for any text in the chart Specify the location for the title, legend, and footnote Change the colors used for the data Change the size and spacing of the bars in bar charts Add borders and shading to the background (backdrop) of any area
Procedure To fully customize a chart: 1 2 Generate the Pass/Fail report and select the Chart tab. Click the area of the chart that you want to customize. The area is selected. The following figure shows the titled selected.
Users Guide
167
Double-click the selected area. A dialog opens, displaying the properties for the selected area. Tip You can also press the right mouse button anywhere on a chart and select the area you want to modify from a popup menu. For example, the following dialog displays title properties. Like most of the formatting dialogs, it has several tabsin this case, a tab for the backdrop (background), text, font, and location.
4 5
Make your changes. For information about the properties, click Help. Click Apply to apply the changes without closing the dialog. Click OK to close the dialog.
168
Users Guide
Compacting results files

As results files grow after repeated testing, a lot of unused space can accumulate in the files. You can remove the unused space by selecting Results/Compact, thereby reducing the file size.
Users Guide
169
170
Users Guide
7e rh C t p a
Using the Debugger

You will find out about many of the errors or inconsistencies in your scripts when SilkTest automatically raises an exception in response to them. Some problems, however, cause a script to work in unexpected ways, but do not generate exceptions. You can use SilkTest debugger to solve these kinds of problems. Using the debugger, you can step through a script a line at a time and stop at specified breakpoints, as well as examine local and global variables and enter expressions to evaluate. But the SilkTest debugger is more than just a tool for fixing scripts. You can also use the debugger to help find problems in your application, using the debugging facilities to step through the application slowly so you can determine just where a problem occurs.
What you will learn

Topic Page
Starting the debugger Setting breakpoints Executing a script in the debugger Viewing the debugging transcript Viewing the call stack Viewing other script elements Evaluating expressions Running, resetting, and terminating execution Tips on debugging
172 173 175 177 178 178 179 180 180
Users Guide
171
7 USING THE DEBUGGER Starting the debugger
Starting the debugger

There are several ways to enter the debugger, as shown in the following table:
If you want to debug Then
The script in the active window Another script
Select Run/Debug. SilkTest enters the debugger and pauses; it does not set a breakpoint. Select File/Debug and select the script file from the Debug dialog. SilkTest enters the debugger and pauses; it does not set a breakpoint. With a script active, select Run/Testcase, select a testcase from the Run Testcase dialog, and click the Debug pushbutton. SilkTest enters the debugger and sets a breakpoint at the first line of the testcase. Select Run/Application State, select an application state from the Run Application State dialog, and click the Debug pushbutton. SilkTest enters the debugger and sets a breakpoint at the first line of the application state definition.
A testcase
An application state
When you enter the debugger, you can execute the script under your control (but note that you cannot edit a script when you are in the debugger). The following figure shows the debugging environment.
172
Users Guide
7 USING THE DEBUGGER Setting breakpoints
Debugger menus
In debugging mode the menu bar includes three additional menusDebug, Breakpoint, and View. Debug menu commands allow you to control the scripts flow. Breakpoint menu commands add or remove a breakpoint. View menu commands display different elements of the running script (for example, local and global variables, the call stack, and breakpoints) and evaluate expressions.
Exiting the debugger
You can leave the debugger whenever execution is stopped. Procedure To exit the debugger, select Debug/Exit.
Setting breakpoints
One useful way to debug a script is to pause it, observe its behavior and check its state, then restart it. This is useful when you are not sure what lines of code are causing a problem. The debugger lets you stop execution on any line by setting breakpoints. A breakpoint is a line in the script where execution stops, so that you can check the scripts status. During debugging, you can set breakpoints on any executable line where you want to check the call stack (described in Viewing the call stack on page 178), examine the values in one or more variables, or just see what a script has done so far. You cannot set breakpoints on blank lines or comment lines.
Users Guide
173
7 USING THE DEBUGGER Setting breakpoints Adding a breakpoint
You can add breakpoints in any of these ways.

To set a breakpoint on Then
The first line of a function (or 1 Select Breakpoint/Add to open the Add testcase) Breakpoint dialog. 2 Double-click on a module name to have the functions declared in that module listed in the Function list box. 3 Double-click on a function name to set a breakpoint on the first line of that function. Any line in a function (or testcase) Place the cursor on the line where you want to set a breakpoint and select Breakpoint/ Toggle. or Double-click in the left margin of the line. A specific line in a script 1 Select Breakpoint/Add to open the Add Breakpoint dialog. 2 In the Breakpoint field, type the number of the line on which you want to set a breakpoint. (For example, entering 8 sets a breakpoint on the eighth line of the script.) 3 Click OK.
A breakpoint is denoted as a large red bullet.
Breakpoints
174
Users Guide
7 USING THE DEBUGGER Executing a script in the debugger Seeing a list of breakpoints
Procedure To view a list of all the breakpoints in a script, select View/ Breakpoints. The list shows the name of the file each breakpoint is in, and the number of the line it is set on or the name of the function if the breakpoint was set on the first line of the function.
Deleting breakpoints
You can delete breakpoints in any of three ways.

To delete Then
All breakpoints
1 Select Breakpoint/Delete All. 2 Click Yes to confirm the deletions.
An individual breakpoint Place the cursor on the line where the breakpoint is set and select Breakpoint/Toggle. or Double-click in the left margin of the line One or more breakpoints 1 Select Breakpoint/Delete to open the Delete Breakpoint dialog. 2 Select one or more breakpoints from the list box and click OK.
Setting a temporary breakpoint
Select Debug/Run To Cursor to set a temporary breakpoint (indicated by a hollow red circle in the margin) on the line containing the cursor. SilkTest immediately runs the script, stopping at the current line. The breakpoint is cleared after it is hit.
Executing a script in the debugger

Once you have set one or more breakpoints, you can start executing your script. Procedure To start debugging a script, select Debug/Run. SilkTest runs the script until it hits the first breakpoint, an error occurs, or the script ends. The line you are stopped on is indicated by a blue triangle.
Users Guide
175
7 USING THE DEBUGGER Executing a script in the debugger
Line stopped at
Procedure To continue execution, do one of the following: Select Debug/Continue. SilkTest runs the script until it hits the next breakpoint, an error occurs, or the script ends. Select Debug/Step Into, Debug/Step Over, or Debug/Finish Function to run a smaller chunk of your script, as described next.
Stepping into and over functions

Sometimes the key to locating a bug in your code is to divide the script up into discrete functions, and debug each function separately. One good way to do this is with the Step Into, Step Over, and Finish Function commands on the Debug menu (also available on the toolbar). These commands let you run and test functions individually.
Finish Function Step Into Step Over Step Into command
You use the Step Into command to execute the next line in a script. If the line contains a function call, control passes into the function. You can use Step Into to step through the function one line at a time, executing each line in turn as you go.
176
Users Guide
7 USING THE DEBUGGER Viewing the debugging transcript Step Over command
Step Over also executes the next line in a script. However, if the line is a function call, SilkTest executes the function without stopping. You use Step Over to speed up debugging if you know a particular function is bug-free. You use Finish Function to execute the script until the current function returns. The focus will be at the line where the function returns. Try using Finish Function in combination with Step Into to step into a function and then run it.
Finish Function command
Viewing the debugging transcript

When you debug a script, SilkTest records error information and output from print statements in a transcript, not in a results file. Procedure To view the transcript, select View/Transcript when execution is stopped. The transcript is displayed in a new window. You can save its contents in a text file by selecting File/Save.
Sending commands to the test application
The Transcript window has an Execute field that you can use to send commands to the application you are testing. You can type in any command that would be valid in a script and click Execute. For example, you might want to print the value of a variable or the contents of a window. You can have SilkTest record all the methods that a script has invoked in the transcript. Each entry includes the method name and the arguments passed into the method. This information can be useful for debugging because it tells you exactly what GUI functions were actually called by the running script. Procedure To enable viewing the trace listing, check the Print Agent Calls option (and if you want, the Print Tags with Agent Calls option) in the Runtime Options dialog before running a script. Procedure To check the Agent trace during debugging, select View/ Transcript when execution pauses. In addition to error information and output from print statements, the transcript also lists all methods called by the script.
Including the called methods
Users Guide
177
7 USING THE DEBUGGER Viewing the call stack
Viewing the call stack

The call stack is a description of all the function calls that are currently active in the script you are debugging. By viewing the call stack, you can trace the flow of execution, possibly uncovering errors that result when a script's flow of control is not what you intended. Procedure To view the current call stack, select View/Call Stack The call stack is shown in a new window. Tip To return to the script being debugged, press F6 or select View/ Module and select the script from the displayed list.
Viewing other script elements

In addition to information about the breakpoints, output, and call stack, SilkTest also gives you access to information about a running scripts variables and modules.
Viewing and setting variables

Viewing variables
When debugging, you can examine variables. Procedure To view a list of all the local variables that are in scope (accessible) from the current line, including their values, select View Local Variables. Procedure To view a list of global variables, select View/Global Variables. The variables and their values are listed in a new window.
178
Users Guide
7 USING THE DEBUGGER Evaluating expressions
If a variable is uninitialized, SilkTest labels it "<unset>." If a variable has a complex value, like an array, SilkTest might need to display its result in collapsed form. Use View/Expand Data and View/Collapse Data (or doubleclick on the plus icon) to manipulate the display (this was done in the preceding figure, which shows a record). Tip To return to the script being debugged, press F6 or select View/ Module and select the script from the displayed list.
Setting variables
While viewing variables, you can also change their values to test various scenarios. Procedure To change the value of an active variable, select the variable and type its new value in the Set Value field. When you resume execution, the new values will be used.
Viewing a list of modules

Procedure To see a list of modules used by the script being debugged: 1 Select View/Module. The View Module dialog displays a list of modules. The list includes all the modules SilkTest loads at startup (that is, the modules loaded by startup.inc, including winclass.inc), so you can set breakpoints in GUI functions, window class declarations, and so forth. 2 Double-click on a module's name to view it in a debug window.
Evaluating expressions
When a script reaches a breakpoint, you can evaluate expressions. Procedure To evaluate expressions: 1 2
Specifying expressions
Select View/Expression. The Expression window is displayed. Type an expression into the input area and press Enter to view the result.
If you type an identifier name, the result is the value that variable currently has in the running script. If you type a function name, the result is the value the function returns. Any function you specify must return a value, and must be in scope at the current line.
Users Guide
179
7 USING THE DEBUGGER Running, resetting, and terminating execution
Properties and methods for a class are valid in expressions, as long as the declaration for the class they belong to is included in one of the modules used by the script being debugged. If an expression evaluates to a complex value, like an array, SilkTest may display its result in collapsed form. Use View/Expand Data or View/Collapse Data (or double-click on the plus icon) to manipulate the display.
Running, resetting, and terminating execution

Procedure To run the script you are debugging, select Debug/Run. The script runs until a breakpoint is hit, an error occurs, or it terminates. Procedure To reset a script, use Debug/Reset. This frees memory, frees all variables, and clears the call stack. The focus will be at the first line of the script. Procedure To abort execution of a running script: Press Shift-Shift when running a script on the same machine. Select Debug/Abort when running a script on a different machine.
Procedure To exit the debugger, select Debug/Exit.
Tips on debugging
This section describes some common bugs and how to find them. It also includes some ideas for designing and testing scripts with error detection in mind.
Troubleshooting common problems

Typographical errors
It is very easy to make typographical errors that the 4Test compiler cannot catch. If a line of code does nothing, this might be the problem. When you write a function that uses global variables, make sure that each variable has an appropriate value when the function exits. If another function uses the same variable later, and it has an unexpected value on entry to the function, an error could occur.
Global variables with unexpected values
180
Users Guide
7 USING THE DEBUGGER Tips on debugging
To check that a variable has a reasonable value on entry to a function, set a breakpoint on the line that calls the function and use the command View/ Global Variables to check the variable's value.
Uninitialized variables
SilkTest does not initialize variables for you. So if you have not initialized a variable on entry to a function, it will have the value <unset>. It is better to explicitly give a value to a variable than to trust that another function has already initialized it for you. Also, remember that 4Test does not keep local variables around after a function exits; the next time the function is called, its local variables could be uninitialized. If you are in doubt about whether a variable has a reasonable value at a particular point, set a breakpoint there and use View/Global Variables or View/Local Variables to check the variable's value.
Global and local variables with the same name
It is usually not good programming practice to give different variables the same names. If a global and local variable with the same name are in scope (accessible) at the same time, your code can only access the local variable. To check for repeated names, use View/Local Variables and View/ Global Variables to see if two variables with the same name are in scope simultaneously.
Incorrect values for loop variables
When you write a for loop or a while loop, be sure that the initial, final, and step values for the variable that controls the loop are correct. Incrementing a loop variable one time more or less than you really want is a common source of errors. To make sure a control loop works as you expect, use Debug/Step Into to step through the execution of the loop one statement at a time, and watch how the value of the loop variable changes using View/Local Variables.
Checking the precedence of operators
The order in which 4Test applies operators when it evaluates an expression may not be what you expect. Use parentheses, or break an expression down into intermediate steps, to make sure it works as expected. You can use View/ Expression to evaluate an expression and check the result. A break statement transfers control of the script out of the innermost nested for, for each, while, switch, or select statement only. In other words, break exits from a single loop level, not from multiple levels. Use Debug/Step Into to step through the script one line at a time and ensure that the flow of control works as you expect. To check for infinite loops, step through the script with Debug/Step Into. To check for code that never executes, step through the script with Debug/ Step Into.
Incorrect use of break statements
Infinite loops Code that never executes
Users Guide
181
7 USING THE DEBUGGER Tips on debugging
Designing and testing with debugging in mind

Here are some suggestions for designing and testing a script that will facilitate debugging it later: Plan for debugging (and robustness) when youre designing the script, by having your functions check for valid input and output, and inform you in some way if problems occur. Test each function as you write it, by building it into a small script that calls the function with test arguments and performs some operation that lets you know it works. Or use the debugger to step through the execution of each function individually after you have coded all (or part) of the script. Test each routine with the full range of valid data values, including the highest and lowest valid values. This is a good way to find errors in control loops. Test each routine with invalid values; it should reject them without crashing. Test each routine with null (empty) values. Depending on the purpose of the script, it might be useful if a reasonable default value were provided when input is incomplete.
182
Users Guide
Enhancing Your Tests

II tP r a
In this part

Chapter Page
Chapter 8, Generalizing a Testcase Chapter 9, Handling Exceptions Chapter 10, Adding Data to a Testplan Chapter 11, Categorizing and Marking Testplans Chapter 12, Working With Large Testplans
185 209 221 235 255
Users Guide
183
184
Users Guide
8e rh C t p a
Generalizing a Testcase
The testcases you record are written in 4Test, a fourth-generation language (4GL) designed specifically for testing GUI applications. 4Test provides a rich set of statements, keywords, functions, and operators that allow you to generalize your recordings to make them more robust.
What you will learn

Topic Page
Using application states Using data-driven testcases Testing applications that have been made Year 2000 compliant
185 190 195
For more information on any of the 4Test statements and reserved words discussed in this chapter, see the online Help.
Using application states

When testing an application, youre likely to have a number of testcases that have identical setup steps. For example, each testcase that exercises the sample Find dialogs forward, case-sensitive searching ability has the same Setup stage.
Example
Each testcase needs to Open a new document file Type text into the document Position the insertion point at the top of the file Select Find from the Search menu
Users Guide
185
8 GENERALIZING A TESTCASE Using application states

Application states
Select the forward (down) direction for the search Make the search case sensitive
Rather than record the same steps over and over again, it is easier to record the steps as an application state and then associate the application state with the relevant testcases. An application state is the state you want your application to be in after the base state is restored but before you run one or more testcases. By creating an application state, you are creating reusable code that saves space and time. Furthermore, if you need to modify the Setup stage, you change it once, in the application state routine.
Application states can be chained together
A testcase can have, at most, one application state associated with it. However, that application state may itself be based on another previously defined application state. For example, assume that The testcase Find is associated with the application state Setup The application state Setup is based on the application state OpenFile The application state OpenFile is based on the built-in application state, DefaultBaseState
SilkTest would execute the programs in this order: 1 2 3 4

Behavior of an application state based on NONE
DefaultBaseState application state OpenFile application state Setup application state Find testcase
If an application state is based on the keyword NONE, SilkTest executes the application state twice: when the testcase with which it is associated is entered and when the testcase is exited. On the other hand, if an application state is based on DefaultBaseState, SilkTest executes the application state only when the associated testcase is entered. The following example code defines the application state InvokeFind as based on the NONE keyword and associates that application state with the testcase TestFind.
Appstate InvokeFind () basedon none xFind.Invoke () print (hello) testcase TestFind () appstate InvokeFind print (In TestFind)
186
Users Guide
8 GENERALIZING A TESTCASE Using application states xFind.Exit.Click ()
When you run the testcase in SilkTest, in addition to opening the Find dialog, closing it, and reopening it, the testcase also prints:
hello In TestFind hello
The testcase prints hello twice because SilkTest executes the application state both as the testcase is entered and as it is exited.
A base state is an application state
As this example illustrates, the built-in routine DefaultBaseState is itself an application state. The DefaultBaseState application state is responsible for restoring the application to the base state in the event the application fails or is corrupted during testcase execution or between testcases. For more information, see Chapter 15, Understanding the Recovery System. If a testcase is based on a single application state, that application state must itself be based on DefaultBaseState in order for the testcase to use the recovery system. Similarly, if a testcase is based on a chain of application states, the final link in the chain must be DefaultBaseState. In this way, SilkTests built-in recovery system is still able to restore the application to its base state when necessary.
Base the final application state on DefaultBaseState
Recording an application state

You define an application state before recording the testcase(s) associated with it. As with testcases, you can write an application state routine from scratch or you can use the Application State command on the Record menu. Procedure To record an application state: 1 Open the file in which you want to place the application state. This can either be the test frame file for the application or the script file where the associated testcases are defined. If you put the application state in the test frame file, it will be available to all testcases. If you put it in the script file, it will be available only to testcases in that script file. Open the application you want to test. Select Record/Application State. The Record Application State dialog appears.
2 3
Users Guide
187
4 5 6
Type the name of your new application state in the Application State Name field. Select an application state from the Based On drop-down list. Press the Start Recording pushbutton. SilkTest closes the Record Application State dialog and displays the Record Status window. The Status field flashes Recording.
7 8
Drive your application to the state you want to record. At any point, you can record a verification by pressing Ctrl+Alt. When you have finished recording an application state, press Done on the Record Status window. SilkTest redisplays the Record Application State dialog. The Application State Code field contains all the 4Test code youve just recorded. You can take the following actions:
If Then
All the information in the window is complete and what you expect. You want to alter the code.
Click Paste to Editor. SilkTest closes the Record Application State dialog and places the new application state in your file. Edit the Application State Code field.
The application state name Edit the name in the Application State Name is not what you want. field.
188
Users Guide
If
Then
The application state on which this application state is based is not the one you want. The application state routine is not finished.
Example
Delete the code in the Application State Code field, select a new application state from the drop-down list, and click Resume Recording to rerecord the application state. Click Resume Recording. SilkTest opens the Record Status window.
Here is a sample application state that performs the setup for all forward case-sensitive searches in the Find dialog:
appstate Setup () basedon DefaultBaseState TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys ("Test Case<Home>") TextEditor.Search.Find.Pick () Find.CaseSensitive.Check () Find.Direction.Select ("Down")
Testing the application state

Before you run a testcase that is associated with an application state, you should make sure the application state compiles and runs without error. Procedure To test an application state: 1 Make active the window that contains the application state and select Run/Application State. The Run Application State dialog appears, shown in the following figure.
2 3
Select the application state you want to run and click Run. If there are compilation errors, SilkTest displays an error window. Fix the errors and rerun the application state.
Users Guide
189
8 GENERALIZING A TESTCASE Using data-driven testcases
Using data-driven testcases

SilkTest uses the term data-driven testcase to refer to a testcase that is passed data. To understand the motivation for creating data-driven testcases, take a look at the following testcase, which is not data-driven:
Example of a nondata-driven testcase testcase FindTest () TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys ("Test Case<HOME>") TextEditor.Search.Find.Pick () Find.FindWhat.SetText ("Case") Find.CaseSensitive.Check () Find.Direction.Select ("Down") Find.FindNext.Click () Find.Cancel.Click () DocumentWindow.Document.VerifySelText (<text>) Case TextEditor.File.Close.Pick () MessageBox.No.Click ()
The chief disadvantage of this kind of testcase is that it tests only one out of the many possible sets of input data to the Find dialog. Therefore, to adequately test the Find dialog, you must record or hand-write a separate testcase for each possible combination of input data that needs to be tested. In even a small application, this creates a huge number of testcases, each of which must be maintained as the application changes.
Creating a data-driven testcase

Procedure To create a data-driven testcase: 1 2 3 Record a testcase that tests a single combination of data. Identify the data type of each recorded data value. Data values appear inside the parentheses at the end of a 4Test statement. Define a record data type for the data being factored out of the testcase. This type definition can be placed at the top of the script file or in the test frame. Add a record variable to the testcases list of parameters. The list of parameters is specified between the parentheses after the testcase name. Replace each data value in the testcase with the appropriate field from the record. In 4Test, as in other languages, fields in records are indicated with the dot (.) operator.
4 5
190
Users Guide
8 GENERALIZING A TESTCASE Using data-driven testcases Example
The following example shows how to generalize the testcase shown above to make it data-driven. First, we analyzed the testcase to see what data it used. The testcase has five recorded STRING values: Test Case <HOME> Case Down Case
Next, we defined a record that consisted of the data elements used by the testcase:
type SEARCHINFO is record STRING sText // Text to type in document window STRING sPos // Starting position of search STRING sPattern // String to look for BOOLEAN bCase // Case-sensitive or not STRING sDirection // Direction of search STRING sExpected // The expected match
Note that the record contains an additional data value, a BOOLEAN value that indicates whether or not to check the Case Sensitive check box. Then we modified the declaration of the testcase so that it is passed a record of type SEARCHINFO:
testcase FindTest (SEARCHINFO Data)
Note that the record variable name (Data) is preceded by the record type name (SEARCHINFO). Finally, we revised the testcase to use the defined data:
testcase FindTest (SEARCHINFO Data) TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText+Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern) Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () Find.Cancel.Click () DocumentWindow.Document.VerifySelText ({Data.sExpected}) TextEditor.File.Close.Pick () MessageBox.No.Click ()
Users Guide
191
Note that the command that checks the Case Sensitive check box has been changed to use the SetState method instead of the recorded Check method, so that it can receive the BOOLEAN parameter bCase. Note also that the following statement in the rewritten testcase uses the {} operator (list constructor operator).
DocumentWindow.Document.VerifySelText({Data.sExpected})
This is because the VerifySelText command expects a list when verifying a multi-line text field, not a single value. Note See the online Help for information on any of the 4Test syntax features used in this section.
Whats next
Once you have defined your data-driven testcase, you need to pass data to it, as described next.
Passing data to a testcase

Once you have defined your data-driven testcase, you pass data to it, as follows: If you are not using the testplan editor, you pass data from a scripts main function, as described in this section. If you are using the testplan editor, you do not pass data to a testcase from the scripts main function. Instead, you embed the data in the testplan and the testplan editor passes the data when you run the testplan. Therefore, the testplan editor users should skip this section and instead follow the instructions in Adding Data to a Testplan on page 221.
Using a main function in the script
Although most of the script files you create contain only testcases, in some instances, you need to add a function named main to your script. You can use the main function to pass data to testcases as well as control the order in which the testcases in the script are executed. When you run a script file using Run/Run: If the script file contains a main function, the main function is executed, then execution stops. Only testcases and functions called by main will be executed, in the order in which they are specified in main. If the script does not contain a main function, the testcases are executed from top to bottom.
The following template shows the structure of a script that contains a main function that passes data to a data-driven testcase:
192
Users Guide
8 GENERALIZING A TESTCASE Using data-driven testcases main () // 1. Declare a variable to hold current record // 2. Store all data for testcase in a list of records // 3. Call the testcase once for each record in the list Example of using main
Using this structure, the following example shows how to create a script that defines data records and then calls the sample testcase once for each record in the list:
type SEARCHINFO is record STRING sText // Text to type in document window STRING sPos // Starting position of search STRING sPattern // String to look for BOOLEAN bCase // Case-sensitive or not STRING sDirection // Direction of search STRING sExpected // The expected match main () SEARCHINFO Data list of SEARCHINFO lsData = {...} {"Test Case", "<END>", "C", TRUE, "Up", "C"} {"Test Case", "<END>", "Ca", TRUE, "Up", "Ca"} // additional data records can be added here for each Data in lsData FindTest (Data) testcase FindTest (SEARCHINFO Data) TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern) Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () Find.Cancel.Click () DocumentWindow.Document.VerifySelText ({Data.sExpected}) TextEditor.File.Close.Pick () MessageBox.No.Click ()
When the script is run
When you select Run/Run, the main function is called and the FindTest testcase will be executed once for every instance of Data in lsData (the list of SEARCHINFO records). In the script shown above, the testcase will be run twice. Here is the results file that is produced:
Script findtest.t - Passed Passed: 2 tests (100%) Failed: 0 tests (0%) Totals: 2 tests, 0 errors, 0 warnings
Users Guide
193
Testcase FindTest ({"Test Case", "<END>", "C", TRUE, "Up", "C"}) - Passed Testcase FindTest ({"Test Case", "<END>", "Ca", TRUE, "Up", "Ca"}) - Passed
Notice that with data-driven testcases, SilkTest records in the results file the parameters that are passed in.
Using FileReadInfo to read external data
In this sample data-driven testcase, the testcase data is stored in a list within the script itself. It is also possible to store the data externally and read records into a list using the FileReadValue function. For more information on this function, see online Help.
Testing an application with invalid data

Note This section assumes that you have already read Using datadriven testcases on page 190. To thoroughly test an application feature, you need to test the feature with invalid as well as valid data. For example, the sample Text Editor application displays a message box if a user specifies a search string in the Find dialog that doesnt exist in the document. To account for this, you can create a data-driven testcase, like the following, that verifies that the message box appears and has the correct message:
type SEARCHINFO is record STRING sText // Text to type in document window STRING sPos // Starting position of search STRING sPattern // String to look for BOOLEAN bCase // Case-sensitive or not STRING sDirection // Direction of search STRING sExpected // The expected match STRING sMessage // The expected message in message box testcase FindInvalidData (SEARCHINFO Data) TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern) Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () MessageBox.Message.VerifyValue (Data.sMessage) MessageBox.OK.Click () Find.Cancel.Click ()
194
Users Guide
8 GENERALIZING A TESTCASE Testing applications that have been made Year 2000 compliant TextEditor.File.Close.Pick () MessageBox.No.Click ()
The VerifyValue method call in this testcase verifies that the message box contains the correct string. For example, the message should be Cannot find Ca if the user enters Ca into the Find dialog and the document editing area does not contain this string.
Using do...except to handle an exception
The VerifyValue method, like all 4Test verification methods, raises an exception if the actual value does not match the expected (baseline) value. When this happens, SilkTest halts the execution of the testcase and transfers control to the recovery system. The recovery system then returns the application to the base state. (For more on the recovery system, see Chapter 15, Understanding the Recovery System.) However, suppose you dont want SilkTest to transfer control to the recovery system, but instead want to trap the exception and handle it yourself. For example, you might want to log the error and continue executing the testcase. To do this, you can use the 4Test do...except statement and related statements, which allow you to handle the exception yourself. For more information, see Chapter 9, Handling Exceptions.
Testing applications that have been made Year 2000 compliant

You might have to change some of your applications over the upcoming months to be Year 2000 compliant. Specifically, you will probably want to change date fields in your application from two-digit years (such as 98) to four-digit years (such as 1998). But you will want your existing tests (which process two-digit years) to continue to runwithout modificationagainst your converted application. SilkTest supports this requirement. By performing the following steps, you will be able to run the same tests that you have developed for your non-Year 2000compliant application (which expects two-digit years) against the application that you have converted to be Year 2000 compliant (which expects four-digit years). SilkTest will automatically convert dates when needed to conform to what your updated, Year 2000compliant application is expecting. To do this you: 1 2 Enable Year 2000 date transformation in SilkTest. Define the rules that SilkTest should use in transforming dates for Year 2000 compliance.
Users Guide
195
8 GENERALIZING A TESTCASE Testing applications that have been made Year 2000 compliant
Run your existing tests against your Year 2000compliant application. SilkTest will automatically convert the dates your tests send to the application to what the application is expecting. Similarly, when running your tests, SilkTest will transform the expected values in your existing verifications so they match the updated expected values in your Year 2000compliant application. SilkTest will log a message whenever it transforms a date when running a script.
Enabling Year 2000 date transformation

Procedure To enable Year 2000 date transformation: 1 Select Options/Runtime. The Runtime Options dialog displays.
Select the Enable Y2K Date Transformation Rules check box. With this check box set, SilkTest will transform dates according to rules that you define, as described next. If this check box is not set, SilkTest will not perform any automatic date transformations, even if rules have been defined.
196
Users Guide
Defining the transformation rules

Once you have enabled date transformation, you define the rules that tell SilkTest how to transform the dates: You specify the date formats it should look for in your scripts, then specify how the dates should be transformed so they match what your Year 2000compliant application is expecting. Procedure To define the transformation rules: 1 2 Select Options/Runtime. The Runtime Options dialog displays. Click the Y2K Rules button. The Y2K Date Transformation Rules dialog displays.
3 4 5 6
Defining a rule
Define a rule, as described next. Click Add. Define additional rules if you want. Click OK.
Fill in the following fields to define a transformation rule:

Field Description
Input Date Format
Specifies how the dates are specified in your scripts. You can select one of the predefined formats in the list or define your own. See Specifying date formats below.
Users Guide
197
Field
Description
Input Year Threshold
Specifies whether a two-digit date is transformed into a 19xx date or a 20xx date. Dates whose years are less than or equal to the threshold are transformed into 20xx dates; dates whose years are greater than the threshold are transformed into 19xx dates. Specify 0 to ignore this field.
Minimum Year
Dont transform the date if the date after transformation would be less than the year specified here. Specify a four-digit date; specify 0 to ignore this field. Dont transform the date if the date after transformation would be greater than the year specified here. Specify a four-digit date; specify 0 to ignore this field. Specifies how the Year 2000complaint application expects the dates. You can select one of the predefined formats in the list or define your own. See Specifying date formats below. Number of years to add to the transformed date. You can use aging so that your tests can test different dates without your having to modify the scripts. Specify 0 to ignore this field.
Maximum Year
Output Date Format
Output Year Aging
Specifying date formats
Date formats consist of date masks, which specify how the date is represented, and delimiters. The date masks Date import and output formats in transformation rules support the following masks (which are a subset of the masks supported by the FormatDateTime function):
Mask Description
yy yyyy m mm
Two-digit year Four-digit year Month with one or two digits (112) Month with exactly two digits (0112)
198
Users Guide
Mask
Description
mmm mmmm d dd ddd dddd
Abbreviated month name (Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, or Dec); case insensitive Full month name (January, February, and so on); case insensitive Day of month with one or two digits (131) Day of month with exactly two digits (0131) Abbreviated day of the week (Sun, Mon, Tue, Wed, Thu, Fri, or Sat); case insensitive Full day of the week (Monday, Tuesday, and so on); case insensitive
Note Though it is unlikely that you will need any masks other than those listed above, date output formats can use any of the masks supported by FormatDateTime. Valid delimiters In addition to the date masks, date formats can include the following delimiters: Space / - : , . (matches one or more spaces) (forward slash, hyphen, colon, comma, or period)
Any other character in the format generates an error.

Examples of date formats
Here are some examples of date formats you can use in the transformation rules:
Format Matches these date strings
m/d/yy mm/dd/yy m-d-yy yy/m/d mmddyy mm/dd/yyyy yyyy/m/d mmm d, yyyy
9/18/97, 09/18/97, 9/18/30 09/18/97 (but not 9/18/97) 9-18-97, 09-18-97 97/9/18, 97/09/18 091897 9/18/1997, 09/18/1997, 9/18/1930, 9/18/2030 1997/9/18, 1997/09/18 Sep 18, 1997
Users Guide
199
How SilkTest uses the rules

This is what happens by default when you have enabled date transformation and defined one or more rules:
Sending dates to the application
Based on the rules you have defined, SilkTest will transform what it sends to the application through any of the following methods: TypeKeys for any object SetText for a ComboBox or TextField Note SilkTest doesnt change what gets recorded in a test. For example, if you send a date with a two-digit year to an application, it is that two-digit date that gets recorded in the test, though the application actually receives the transformed four-digit date (if an appropriate transformation rule applied).
Getting dates from the application
Similarly, SilkTest will transform dates that come back from the application through the following method: GetText for a ComboBox, DynamicText, StaticText, or TextField
Verifying properties
Each time VerifyProperties is called, SilkTest traverses the entire property tree (the WinPropTree argument to the function) and converts all the expected values using the defined rules. The transformed dates are used as the true expected values in the verifications. When the methods listed above are called, SilkTest does the following: 1 2 It compares the string argument to all the input formats that have been defined in any of the rules. If the argument matches an input format, the string is transformed to the output format, subject to the values specified in the rule for Input Year Threshold, Minimum Year, Maximum Year, and Output Year Aging. If a string doesnt match an input format, it is not transformed. SilkTest uses the same criterion when transforming dates coming back from the application as when transforming dates sent to the application: If the date coming back through GetText matches an input format, it is transformed. 3 It sets or gets the text in the application using the transformed date and notifies you in the results file that it transformed a date. Note SilkTest does all this transformation in memory at runtime. Your scripts are not changed.
How the dates are transformed
200
Users Guide
8 GENERALIZING A TESTCASE Testing applications that have been made Year 2000 compliant Finding all the dates in your tests
If you want SilkTest to find all dates in your tests but not transform them, create a rule where the output format is the same as the input format. SilkTest will log a message whenever it encounters such a date, but wont transform it.
Examples of date transformation

Consider the following rule:
That rule says to transform dates in the form m/d/yy (such as 9/18/97) to the form mm/dd/yyyy (such as 09/18/1997). Dates whose years are less than or equal to 30 are transformed to 20xx. Here are some tests that illustrate the date transformation (it uses the Text Field window in the test application shipped with SilkTest):
Example 1: Date conversion testcase GetandSetText1 () STRING sReturnedText TextFieldWindow.Invoke () Print ("Entering the value '9/18/97'") TextFieldWindow.TheTextField.SetText ("9/18/97") Print ("Getting ...") sReturnedText = TextFieldWindow.TheTextField.GetText () Print (sReturnedText) TextFieldWindow.Close () // Result: // Entering the value '9/18/97'
Users Guide
201
8 GENERALIZING A TESTCASE Testing applications that have been made Year 2000 compliant // *** Y2K Notification: date value '9/18/97' has been changed to '09/18/1997' in TextFieldWindow.TheTextField.SetText // Getting ... // 09/18/1997
Note the following about this test: The argument to SetText is 9/18/97. That date is of the input form m/d/yy so it matches the rule. Therefore, the date is transformed to the form mm/dd/yyyy. In this case, the date is set to 09/18/1997, as shown in the return from GetText. SilkTest includes a notification message that the date has been transformed.
Example 2: Date conversion using threshold
testcase GetandSetText2 () STRING sReturnedText TextFieldWindow.Invoke () Print ("Entering the value '9/18/20'") TextFieldWindow.TheTextField.SetText ("9/18/20") Print ("Getting ...") sReturnedText = TextFieldWindow.TheTextField.GetText () Print (sReturnedText) TextFieldWindow.Close () // Result: // Entering the value '9/18/20' // *** Y2K Notification: date value '9/18/20' has been changed to '09/18/2020' in TextFieldWindow.TheTextField.SetText // Getting ... // 09/18/2020
Note the following about this test:

Example 3: No date conversion
The argument to SetText is 9/18/20, also of the input form m/d/yy, so it matches the rule. In this case, the year (20) is below the threshold in the rule (30), so the date is converted to 2020, instead of 1920.
testcase GetandSetText3 () STRING sReturnedText TextFieldWindow.Invoke () Print ("Entering the value '9-18-97'") TextFieldWindow.TheTextField.SetText ("9-18-97") Print ("Getting ...") sReturnedText = TextFieldWindow.TheTextField.GetText ()
202
Users Guide
8 GENERALIZING A TESTCASE Testing applications that have been made Year 2000 compliant Print (sReturnedText) TextFieldWindow.Close () // // // // Result: Entering the value '9-18-97' Getting ... 9-18-97
Note the following about this test: The argument to SetText is 9-18-97. This does not match an input format, so the date is not transformed.
If you want to transform dates of this form, you can define a rule using the input format m-d-yy.
Specifying where date transformation occurs

SilkTest provides two keywords for specifying where in the object class hierarchy date transformation should occur:
Y2KRules
Y2KRules: Specifies the precedence for applying transformation rules. Can also be used to disable all Y2K transformations. Y2KRulesInherited: Applies data transformation rules or disables Y2K transformations selectively for particular objects or classes.
Y2kRules sets the precedence for applying transformation rules for example, when to apply global transformation rules specified in Runtime Options as opposed to rules specified selectively for particular classes or objects. You must add the Y2KRules declaration within the AnyWin class in winclass.inc (located in the directory where you installed SilkTest). The syntax for setting Y2KRules is:
setting Y2KRules = "value" | NULL
Users Guide
203
You can declare Y2KRules as follows:

setting Y2KRules = Action
UseContainer
Applies Y2K transformations specified selectively for particular objects or classes (via Y2KRulesInherited declarations). If no selective transformations are specified, uses the date transformation rules that you have defined in Runtime Options. If you have not enabled transformation rules in Runtime Options, no transformations are applied
Applies date transformation rules you have defined in Runtime Options. If you have not enabled transformation rules in Runtime Options, no transformations are applied.
rule
Applies rule globally to all objects in your application, overriding all selective rules. For information about how to specify rule, see the description of the TransformDate function in the online Help.
NULL
Disables all Y2K transformations.
You can also use the Y2KRules declaration to apply rules to individual child controls. The Y2KRules construct applies the rule only to the control in which it is declared. If you want to apply a rule to a control and all of its children, use the Y2KRulesInherited construct.
Y2KRulesInherited
Y2KRulesInherited allows you to specify transformation rules selectively for particular objects or classes. You must declare Y2KRulesInherited rules within the declarations of specific objects or classes. The rule you specify will apply to the object and all of its children. Rules set for specific child controls override any rules they might inherit. For example, if you want to specify the same Y2K transformation rule for all text fields on a dialog, add the Y2KRulesInherited rule in the declaration for the dialog. If you later want one of the text fields to apply a different transformation rule, add a second Y2KRulesInherited rule in the declaration for that text field. This rule will override the inherited one. The syntax for setting Y2KRulesInherited is:
setting Y2KRulesInherited = "rule" | NULL
204
Users Guide
You can declare Y2KRulesInherited as follows:

setting Y2KRulesInherited = Action
Applies date transformation rules you have defined in Runtime Options. If you have not enabled transformation rules in Runtime Options, no transformations are applied.
rule
Applies rule to a class or object and all of the objects children. For information about how to specify rule, see the description of the TransformDate function in the online Help.
NULL
Disables Y2K transformations for a particular class or object. Transformations will also be disabled for the objects child controls unless you defined a rule using Y2KRules at a level in the class hierarchy that applies to one of the controls.
Default behavior
When you first install SilkTest, your winclass.inc file contains the following declaration within the AnyWin class:
setting Y2KRules = "UseContainer"
This declaration tells SilkTest to perform Y2K transformations for all windows, according to the precedence rules described for Y2KRules.
Examples
Suppose you have a dialog box called ShowDates that contains two text fields. Initially, you want the dates in all of its text fields to be transformed to the format mmmm dd, yyyy. To enforce this behavior, place the following statement inside the ShowDates declaration:
setting Y2KRulesInherited="mm/dd/yy|0|0|0|mmmm dd, yyyy|0"
Later, you realize that the dates in one of the text fields, called TheSingleLine, must be transformed to the format mm-dd-yyy for a particular report. To meet this requirement, add the following statement inside the declaration for TheSingleLine:
setting Y2KRules="mm/dd/yy|0|0|0|mm-dd-yyyy|0"
Users Guide
205
This new rule will override the inherited rule for TheSingleLineTextField text field. Note that we use Y2KRules because we are applying this rule to a single control. If TheSingleLineTextField had child controls that needed to inherit this rule, we would have used the Y2KRulesInherited construct instead. Heres how these declarations appear in the test frame file:
Suppressing warning messages when transformation occurs

By default, SilkTest logs a message whenever a string is interpreted as a date. There are two situations: When SilkTest transforms a date. For example:
*** Y2K Notification: date value '09/18/97' has been changed to '09/18/1997' in TextFieldWindow.TheTextField.SetText
When SilkTest reads a string that matches an input format in a rule that does not specify a transformation (that is, the output format is the same as the input format). For example:
*** Y2K Notification: date value '9/18/97' was found in TextFieldWindow.TheTextField
206
Users Guide
This default behavior is the result of the following statement in the AnyWin declaration in winclass.inc:
setting Y2KWarnings = TRUE
If you dont want the warnings logged in the results file, change the statement to:
setting Y2KWarnings = FALSE
either for the AnyWin class in winclass.inc or for any other specific declaration.
Users Guide
207
208
Users Guide
9e rh C t p a
Handling Exceptions
This chapter describes techniques you can use to handle exceptions (errors) that are generated when you run your scripts.
What you will learn

Topic Page
Default error handling Using do...except Performing more than one verification in a testcase Adding to the default error handling Trapping the exception number Writing an error-handling function Programmatically logging an error Defining your own exceptions Enabling fault trapping
209 210 211 212 214 214 216 216 217
Default error handling

If a testcase fails (for example, if the expected value doesnt match the actual value in a verification statement), SilkTest by default calls its built-in recovery system, which: Terminates the testcase Logs the error in the results file Restores your application to its default base state in preparation for the next testcase
Users Guide
209
9 HANDLING EXCEPTIONS Using do...except
These runtime errors are called exceptions. They indicate that something did not go as expected in a script. They can be generated automatically by SilkTest, such as when a verification fails, when there is a division by zero in a script, or when an invalid function is called. You can also generate exceptions explicitly in a script. However, suppose you dont want SilkTest to transfer control to the recovery system when an exception is generated, but instead want to trap the exception and handle it yourself. To do this, you use the 4Test do...except statement.
Using do...except
Using do...except you can handle exceptions locally, instead of passing control to SilkTests built-in error handler (which is part of the recovery system). The statement has the following syntax: do statements except statements If an exception is raised in the do clause of the statement, control is immediately passed to the except clause, instead of to the recovery system. If no exception is raised in the do clause of the statement, control is passed to the line after the except clausethe statements in the except clause are not executed.
... do control passes into except clause exception raised here except statements ...
... do control passes to first statement after except clause no exception raised here except statements ...
210
Users Guide
9 HANDLING EXCEPTIONS Performing more than one verification in a testcase
Consider this simple testcase:

testcase except1 (STRING sExpectedVal, STRING sActualVal) do Verify (sExpectedVal, sActualVal) Print ("Verification succeeded") except Print ("Verification failed")
This testcase uses the built-in function Verify, which generates an exception if its two arguments are not equivalent. In this testcase, if sExpectedVal equals sActualVal, no exception is raised, Verification succeeded is printed, and the testcase terminates. If the two values are not equal, Verify raises an exception, control immediately passes to the except clause (the first Print statement is not executed), and Verification failed is printed. Here is the result if the two values one and two are passed to the testcase:
Testcase except1 ("one", "two") - Passed Verification failed
Note that testcase passes and the recovery system is not called, because you handled the error yourself. (In Adding to the default error handling on page 212, you will learn how you can handle the error yourself and call the recovery system too.) You handle the error in the except clause. You can include any 4Test statements, so you could, for example, choose to ignore the error, write information to a separate log file, log the error in the results file, and so on. The following sections describe some typical uses of do...except.
Performing more than one verification in a testcase

Usually, testcases have only one verification statement. If the verification fails, an exception is raised and the testcase terminates. But sometimes for ease of maintenance, you might want to have more than one verification statement in a testcase. Consider the following:
testcase MultiVerify () TextEditor.Search.Find.Pick () Find.VerifyCaption ("Find") Find.VerifyFocus (Find.FindWhat) Find.VerifyEnabled (TRUE) Find.Cancel.Click ()
Users Guide
211
9 HANDLING EXCEPTIONS Adding to the default error handling
That testcase contains three verification statements. However, if, for example, the first verification (VerifyCaption) fails, an exception is raised and the testcase terminates. The second and third verifications never happen. To verify more than one thing in a testcase, you can trap all but the last one in a do...except statement, such as:
testcase MultiVerify2 () TextEditor.Search.Find.Pick () do Find.VerifyCaption ("Find") except ExceptLog () do Find.VerifyFocus (Find.FindWhat) except ExceptLog () Find.VerifyEnabled (TRUE) Find.Cancel.Click ()
Here, all the verifications will happen each time the testcase is run. If either of the first two fails, the 4Test function ExceptLog is called. That function logs the error information in the results file, then continues execution of the script.
Adding to the default error handling

You can also use do...except to perform some custom error handling, then use the reraise statement to pass control to the recovery system as usual. Consider this situation: The Text Editor application displays a message box if a user searches for text that doesnt exist in the document. So you can create a data-driven testcase that verifies that the message box appears and has the correct message. (That testcase is shown in Testing an application with invalid data on page 194.) But suppose you want to determine if the Text Editor application is finding false matches, that is, if it is selecting text in the document before displaying the message box. That means that you want to do some testing after the exception is raised, instead of immediately passing control to the recovery system, as shown in this example:
testcase Negative (SEARCHINFO Data) STRING sMatch TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern)
212
Users Guide
9 HANDLING EXCEPTIONS Adding to the default error handling Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () do MessageBox.Message.VerifyValue (Data.sMessage) except sMatch = DocumentWindow.Document.GetSelText () if (sMatch != "") Print ("Found " + sMatch + " not " + Data.sPattern) reraise MessageBox.OK.Click () Find.Cancel.Click () TextEditor.File.Close.Pick () MessageBox.No.Click ()
As the example shows, following the do keyword is the verification statement, and following the except keyword are the 4Test statements that handle the exception. The exception-handling statements in this example do the following: 1 2 Call the GetSelText method to determine what text, if any, is currently selected in the document. If the return value from the GetSelText method is not an empty string, it means that the application found a false match. If this is the case, then print the false match and the search string to the results file. Reraise the exception to transfer control to the recovery system and terminate the testcase.
3
reraise
The reraise statement raises the most recent exception again and passes control to the next exception handler (in the preceding testcase, it passes control to the built-in recovery system). We used reraise in the preceding testcase because if the exception-handling code does not explicitly reraise the exception, flow of control passes to the next statement in the testcase. So this example performs a test after an exception is raised, prints a statement to the results file if text was selected, then calls the recovery system, which terminates the testcase, logs the error, and restores the test application to its default base state.
Users Guide
213
9 HANDLING EXCEPTIONS Trapping the exception number
Trapping the exception number

Each built-in exception has a name and a number (they are defined as an enumerated data type, EXCEPTION). For example, the exception generated when a verify fails is E_VERIFY (-13700), and the exception generated when there is a division by zero is E_DIVIDE_BY_ZERO (-11500). Note All exceptions are defined in 4test.inc, in the directory where you installed SilkTest. They are described in the topic Exception values in online Help. You can use the ExceptNum function to test for which exception has been generated and, perhaps, take different actions based on the exception. You would capture the exception in a do...except statement then check for the exception using ExceptNum. For example, if you want to ignore the exception E_WINDOW_SIZE_ INVALID, which is generated when a window is too big for the screen, you could do something like this:
do Open.Invoke () except if (ExceptNum () != E_WINDOW_SIZE_INVALID) reraise
If the exception is not E_WINDOW_SIZE_INVALID, the exception is reraised (and passed to the recovery system for processing). If the exception is E_ WINDOW_SIZE_INVALID, it is ignored.
Writing an error-handling function

If you want to customize your error processing, you will probably want to write your own error-handling function, which you can reuse in many scripts. For example, you might want to print the text associated with the exception as well as the function calls that generated the exception. The following testcase illustrates this.
testcase VerifyTest () STRING sTestValue = "xxx" STRING sExpectedValue = "yyy" CompValues (sExpectedValue, sTestValue) CompValues (STRING sExpectedValue, STRING sTestValue) do
214
Users Guide
9 HANDLING EXCEPTIONS Writing an error-handling function Verify (sExpectedValue, sTestValue) except ErrorHandler () ErrorHandler () CALL Call LIST OF CALL lCall lCall = ExceptCalls () Print (ExceptData ()) for each Call in lCall Print("Module: {Call.sModule}", "Function: {Call.sFunction}", "Line: {Call.iLine}")
Note the following about this testcase: It calls the user-defined function CompValues, passing two arguments. CompValues uses Verify to compare its arguments. If they are not equal, an exception is automatically raised. If an exception is raised, CompValues calls a user-defined function, ErrorHandler, which handles the error. This is a general function that can be used throughout your scripts to process errors the way you want. ErrorHandler uses two built-in exception functions, ExceptData and ExceptCalls.
ExceptData
All built-in exceptions have message text associated with them. ExceptData returns that text. ExceptCalls returns a list of the function calls that generated the exception. You can see from ErrorHandler above, that ExceptCalls returns a LIST OF CALL (CALL is a built-in data type that is a record with three elements: sFunction, sModule, and iLine). ErrorHandler processes each of the calls and prints them in the results file. Note SilkTest also provides a function ExceptPrint, which combines the features of ExceptCalls, ExceptData, and ExceptNum.
ExceptCalls
The results file
Running this testcase returns the following in the results file:

Testcase VerifyTest - Passed *** Error: Verify value failed - got "yyy", expected "xxx" Module: Function: Verify Line: 0 Module: except.t Function: CompValues Line: 121 Module: except.t Function: VerifyTest Line: 112
Users Guide
215
9 HANDLING EXCEPTIONS Programmatically logging an error
The second line is the result of printing the information from ExceptData. The rest of the lines show the processing of the information from ExceptCalls. And note that this testcase passes, because the error was handled locally and not reraised.
Programmatically logging an error

Notice that some of the testcases in this chapter passed, even though an error occurred, because they used their own error handler and did not specify to log the error. If you want to handle errors locally and generate an error (that is, log an error in the results file), you can do any of the following: After you have handled the error, reraise it using the reraise statement and let the default recovery system handle it Call any of the following functions in your script:
Function Action
LogError (string)
Writes string to the results file as an error (displays in red or italics, depending on platform) and increments the error counter. This function is called automatically if you dont handle the error yourself.
LogWarning (string) Same as LogError, except it logs a warning, not an error. ExceptLog ( ) Calls LogError with the data from the most recent exception.
Defining your own exceptions

In addition to using built-in exceptions, you can define your own exceptions and generate them using the raise statement. Consider the following testcase:
testcase raiseExample () STRING sTestValue = "xxx" STRING sExpected = "yyy" TestVerification (sExpected, sTestValue)
216
Users Guide
9 HANDLING EXCEPTIONS Enabling fault trapping TestVerification (STRING sExpected, STRING sTestValue) if (sExpected == sTestValue) Print ("Success!") else do raise 1, "{sExpected} is different than {sTestValue}" except print ("Exception number is {ExceptNum()}") reraise
The TestVerification function tests two strings. If they are not the same, they raise a user-defined exception using the raise statement.
raise
The raise statement takes one required argument, which is the exception number. All built-in exceptions have negative numbers, so you should use positive numbers for your user-defined exceptions. raise can also take an optional second argument, which provides information about the exception; that information is logged in the results file by the built-in recovery system or if you call ExceptLog. In the preceding testcase, raise is in a do...except statement, so control passes to the except clause, where the exception number is printed, then the exception is reraised and passed to the recovery system, which handles it the same way it handles built-in exceptions. Here is the result of the testcase:
Testcase raiseExample - 1 error Exception number is 1 yyy is different than xxx Occurred in TestVerification at except.t(31) Called from raiseExample at except.t(25)
Note that since the error was reraised, the testcase failed.
Enabling fault trapping

Under Windows you can have SilkTest trap system errors (general protection faults). This means that if the application under test crashes, SilkTest can raise an exception, record information about the systems state, close the application, then continue executing your tests. You enable fault trapping by setting various Agent options. For more information on the available options, see Compatibility options on page 469.
Users Guide
217
9 HANDLING EXCEPTIONS Enabling fault trapping Windows 95, Windows 98, and Windows NT
Fault trapping is available on Windows 95, Windows 98, and Windows NT. This is especially important for Windows NT users, because unexpected program terminations are likely to cause the entire system to hang. Tip We recommend that you keep fault trapping turned off until you need to turn it on. Setting on the host and target machines Fault trapping for the specific application under test is enabled through the Extensions dialog and the Extension Enabler dialog. Once you enable fault trapping, you can also selectively turn on/off a number of fault trapping options. Prerequisite The following procedure assumes that you have already configured the dialogs for testing your application. Procedure To enable fault trapping on the host and target machines: 1 2 3 4 5 6 7 8 9 Open the Extensions Enabler dialog, available from the Silk or QualityWorks program group, on a target machine. Select the Fault Trap check box for the application you want to test. If you are testing on more than one target machine, repeat steps 12 for every target machine. On the host machine start SilkTest. Select Options/Extensions to open the Extensions dialog. Select the Fault Trap check box for the application you want to test. Click the Fault Trap pushbutton to open the Fault Trap Options dialog. These options are described on page 474. Turn on/off options as you wish and click OK. Click OK to close the Extensions dialog. Note If your application was running while you were configuring the dialogs, you must close your application and restart it. You must also restart the Agent on the target machine. Setting at runtime Alternatively, you can specify fault trapping in a script as follows: 1 Specify the names of one or more executable programs to trap faults for using the OPT_TRAP_FAULTS_APPS option. Specify the entries in a string argument to SetOption, separating multiple entries with commas or spaces. For example:
Agent.SetOption (OPT_TRAP_FAULTS_APPS, "myapp.exe")
10 Start your test application.
218
Users Guide
9 HANDLING EXCEPTIONS Enabling fault trapping
Turn on fault trapping:

Agent.SetOption (OPT_TRAP_FAULTS, TRUE)
Note that in some instances, when an exception in an application occurs during the last action of a testcase, that fault is not considered part of the testcase, and therefore the recovery system terminates the script in the usual manner. In order to ensure continuation of the script, add a Sleep (1) statement after the action that precipitated the fault, thereby trapping the fault within the testcase.
Users Guide
219
9 HANDLING EXCEPTIONS Enabling fault trapping
220
Users Guide
0rh 1e C t p a
Adding Data to a Testplan

One way to eliminate redundancy in a testplan is to generalize a testcase so that it can receive many combinations of data from the testplan. This kind of testcase, as described in Chapter 8, Generalizing a Testcase, is called a data-driven testcase. Linking a testplan to a data-driven testcase alters in two ways the basic testplan structure introduced in Chapter 2, Creating Testplans: You add the testcase statement at the group description level. You place the test data directly in the testplan.
Overview
Benefits
The benefits are twofold: By recording a single testcase for a group, you make your scripts easier to maintain. By placing data at different levels in the plan, you create a matrix of data combinations that makes it easy to assess coverage.
What you will learn

Topic Page
Linking a testplan to a data-driven testcase Specifying data that is unique to a single test description Specifying data that is shared by multiple tests Converting the data in a results file into a plan
222 223 226 232
Users Guide
221
10 ADDING DATA TO A TESTPLAN Linking a testplan to a data-driven testcase
Linking a testplan to a data-driven testcase

Two ways to link a plan to a testcase
To link a group of test descriptions in the plan with a data-driven testcase, add the testcase statement to the group description level. There are two ways to do this: Use the Testplan Detail dialog to automate the process. Enter the testcase statement manually. Note For details on each of these methods, see Linking the testplan to scripts and testcases on page 59. For example, consider the data-driven testcase FindTest, which takes a record of type SEARCHINFO as a parameter:
type SEARCHINFO is record STRING sText // Text to type in document window STRING sPos // Starting position of search STRING sPattern // String to look for BOOLEAN bCase // Case-sensitive or not STRING sDirection // Direction of search STRING sExpected // The expected match testcase FindTest (SEARCHINFO Data) TextEditor.File.New.Pick () DocumentWindow.Document.TypeKeys (Data.sText + Data.sPos) TextEditor.Search.Find.Pick () Find.FindWhat.SetText (Data.sPattern) Find.CaseSensitive.SetState (Data.bCase) Find.Direction.Select (Data.sDirection) Find.FindNext.Click () Find.Cancel.Click () DocumentWindow.Document.VerifySelText ({Data.sExpected}) TextEditor.File.Close.Pick () MessageBox.No.Click ()
222
Users Guide
10 ADDING DATA TO A TESTPLAN Specifying data that is unique to a single test description
The following testplan is associated with the FindTest testcase (the testcase statement is highlighted for emphasis). The statement occurs at the Find dialog group description level, so that each of the eight test descriptions in the group can call the testcase, passing it a unique set of data:
This figure shows the testdata statement, which is used to specify the data that a test description passes to a data-driven testcase. The testdata statement is described in detail in the next section.
Specifying data that is unique to a single test description

Two ways to add a testdata statement
If a data value is unique to a single test description, it should be placed in the plan at the same level as the test description, using the testdata statement. There are two ways to add the testdata statement: Use the Testplan Detail dialog to automate the procedure. Type the statement directly into the plan.
Entering data in the Testplan Detail dialog
Procedure To use the Testplan Detail dialog to enter the testdata statement: 1 Place the insertion point at the end of the test description. For example, the following figure shows how to place the insertion cursor to add the testdata statement for the Character search test description:
Users Guide
223
Note If a testdata statement is not associated with a test description, the compiler will generate an error. 2 Select Testplan/Detail. The Testplan Detail dialog opens, with the Text Execution tab displayed.
To provide context, the multi-line list box at the top of the dialog displays the line in the testplan that the cursor was on when the dialog was invoked, indicated by the black arrow icon. If the testcase and script associated with the current test description are inherited from a higher level in the testplan, they are shown in blue; otherwise, they are shown in black.
224
Users Guide
Enter the data in the Test Data field, separating each data element with a comma. For example:
Tip Remember, if the testcase expects a record, you need to enclose the list of data with the list constructor operator (the curly braces); otherwise, SilkTest interprets the data as individual variables, not a record, and will generate a data type mismatch compiler error. 4 Click OK. The Testplan Detail dialog closes, and the testdata statement and data values are entered in the plan.
Typing data directly into the testplan
Procedure To enter the testdata statement manually: 1 Open up a new line after the test description and indent the line one level. For example, to manually add the testdata statement for the Character search test description:
Enter the testdata statement, using this syntax: a If the testcase expects one or more variables: testdata: data [,data]
Users Guide
225
10 ADDING DATA TO A TESTPLAN Specifying data that is shared by multiple tests
If the testcase expects a record, use the same syntax as above, but open and close the list of record fields with curly braces: testdata: {data [,data]} where data is any valid 4Test expression.
Be sure to follow the testdata keyword with a colon. If you enter the keyword correctly, the statement appears in dark red, the default color. Otherwise, the statement appears in either blue or black, indicating the compiler is interpreting the line as a description. For example:
Specifying data that is shared by multiple tests

If you have many related testcases, using the technique shown above with the testdata statement can result in a lot of redundant data, which creates a maintenance burden. For example, look at the testplan in the figure on page 223. The testcase, FindTest, takes a record. Using testdata, you have to supply the entire record in each individual test, even though some of the data is common to several tests. In order to eliminate that redundancy, you can factor out the data that is common to a group of tests and define it at a level in the testplan where it can be shared by the group. To do this, you define symbols and assign them values.
Definition of a symbol
A symbol represents a piece of data in a data-driven testcase. It is like a 4Test identifier, except that its name begins with the $ character. For example, consider the following testplan:
226
Users Guide
The testplan in the figure uses six symbols: $Text is the text to enter in the document window. $Position is the position of the insertion point in the document window. $Pattern is the pattern to search for in the document window. $Case is the state of the Case Sensitive check box. $Direction is the direction of the search. $Expected is the expected match.
The symbols are named in the parameter list to the FindTest testcase, within the parentheses after the testcase name.
testcase: FindTest ({ $Text, $Position, $Pattern, $Case, $Direction, $Expected })
Note that the symbols are only named in the parameter list; they are not assigned values. The values are assigned at either the group or test description level, depending on whether the values are shared by several tests or are unique to a single test. For example, in the preceding figure, each test description assigns its own unique values to the $Pattern and the $Expected symbols. The remaining four symbols are assigned values at a group description level: The $Text symbol is assigned its value at the Find dialog group description level, because all eight tests of the Find dialog enter the text SilkTest into the document window of the Text Editor application.
Users Guide
227
The $Case symbol is assigned the value TRUE at the Case sensitive group description level and the value FALSE at the Case insensitive group description level. The $Direction symbol is assigned the value Down at the Forward group description level, and the value Up at the Backward group description level. The $Position symbol is assigned the value <HOME> at the Forward group description level, and the value <END> at the Backward group description level.
Because the data that is common is factored out and defined at a higher level, it is easy to see exactly what is unique to each test.
The same symbol can have several local values
If a symbol is defined at a level in the plan where it can be shared by a group of tests, each test can assign its own local value to the symbol, overriding whatever value it had at the higher level. You can tell whether a symbol is locally assigned by using the Testplan Detail dialog: Locally assigned symbols appear in black. Symbols that inherit their values appear in blue.
Specifying symbols as arguments in a testcase statement
Procedure To specify symbols as arguments when entering a testcase statement in the Testplan Detail dialog: 1 Place the insertion cursor in the testplan at the location where the testcase statement is to be inserted. For example:
Select Testplan/Detail.
228
Users Guide
The Testplan Detail dialog opens. For example:
Enter the name of a data-driven testcase, followed by the argument list enclosed in parenthesis. If the testcase expects a record, and not individual values, you must use the {} operator (list constructor operator). For example:
Click OK.
Users Guide
229
The Testplan Detail dialog is dismissed, and the testcase statement is inserted into the testplan. For example:
Assigning values to symbols

Placing a symbol name in the argument list of a testcase statement only specifies the name of the symbol; you also need to define the symbol and assign it a value at either the group or testcase description level, as appropriate. If you do not know the value when you are initially writing the testplan, assign a question mark (?) to avoid getting a compiler error when you compile the testplan; doing so will also cause the tests to be counted as incomplete when a Completion report is generated.
Assigning values in the Testplan Detail dialog
Procedure To define symbols in the Testplan Detail dialog: 1 Place the insertion cursor in the plan where you need to assign a value to a symbol. For example:
Select Testplan/Detail.
230
Users Guide
The Testplan Detail dialog opens. For example:
Select the Symbols tab. The Symbols tab is displayed, as shown in the following figure:
Enter the symbol definition in the text field to the left of the Add pushbutton. For example:
Note that you do not need to enter the $ character; the testplan editor takes care of this for you when it inserts the definitions into the testplan.
Users Guide
231
Click Add. The symbol is added to the list box above the Add text field, for example:
Define additional symbols in the same manner, and then click OK when finished. The Testplan Detail dialog closes, and the symbol definition(s), including the $ character, are entered in the plan. For example:
How inheritance is indicated
The Testplan Detail dialog uses color and font to indicate whether a symbols value is inherited or assigned locally. Locally assigned symbols appear in black, and symbols that inherit their value appear in blue. You can also define symbols and assign them values by typing them into the testplan, using this syntax: $symbolname = symbolvalue where symbolname is any valid 4Test identifier name, prefixed with the $ character and symbolvalue is any string, list, array, or the ? character (which indicates an undefined value).
Assigning values manually
232
Users Guide
10 ADDING DATA TO A TESTPLAN Converting the data in a results file into a plan
For example, the following statement defines a symbol named Color and assigns it the STRING value Red:
$Color = "Red"
For more information, see Appendix A, Testplan Editor Keywords.
Converting the data in a results file into a plan

The testplan editor provides a convenient way to convert a results file into a testplan. This is useful when converting old suite-based tests into testplans. Procedure To convert a results file to a testplan: 1 Open a results file that was generated by SilkTest, not one generated by the testplan editor from a testplan. For example:
2 3
Select Results/Convert to Plan. The testplan editor displays the Convert Results to Plan dialog. Select the results file you want to convert (typically the most recent) and click OK. The testplan editor converts the results file to a testplan.
Because the results file was for a script, not a testplan, there are no group or testcase descriptions in the testplan, only test data. Therefore, the testplan editor uses the # symbol so that when this testplan is run, the testdata statement will double as description. The # symbol can be used
Users Guide
233
10 ADDING DATA TO A TESTPLAN Converting the data in a results file into a plan
with any testplan editor statement so that the statement will double as description. For more information on the syntax of the testplan editor statements, see Appendix A, Testplan Editor Keywords.
234
Users Guide
1e rh C t p a
Categorizing and Marking Testplans

Topic Page
What you will learn
Overview Defining and modifying attributes and values Assigning attributes and values to a testplan Marking a testplan Creating and modifying testplan queries
235 237 240 243 246
Overview
You might want to work with only selected testcases at any one time, for example, to run only those tests that exercise a particular area of the application or to report on only the tests that were assigned to a particular QA engineer.
Definition of mark
To work with selected tests rather than the entire testplan, you denote or mark those tests in the testplan. Marks are temporary; you can remove them at any time, and they last only as long as the current work session. By choice You can mark an individual test description, a group description, or the entire plan simply by selecting it and then choosing the appropriate marking command on the Testplan menu.
Ways of marking a testplan
Users Guide
235
11 CATEGORIZING AND MARKING TESTPLANS Overview
By query You can also mark a testplan according to a certain set of characteristics it possesses. This is called marking by query. You build a query based on one or more specific test characteristics its script file, data, symbols, or attributes and then mark those tests that match the criteria set up in the query. For example, you might want to mark all tests that live in the find.t script and that were created by the developer named Emily. If you name and save the query, you can reapply it in subsequent work sessions without having to rebuild the query or manually remark the tests that youre interested in working with. By test failure After running a testplan, the generated results file might indicate test failures. You can mark these failures in the plan by selecting Results/Mark Failures in Plan. You then might fix the errors and rerun the failed tests. (For information on results files, see Results files on page 148.)
Definition of attributes
Attributes are site-specific characteristics that you can define for your testplan and assign to test descriptions and group descriptions. You can use test attributes to build queries. By assigning attributes to parts of the testplan, you can: Group tests in the plan to distinguish them from the whole testplan Report on the testplan based on a given attribute value Run parts of the testplan that have a given attribute value
How attributes are used
For example, you might define an attribute called Engineer that represents the set of QA engineers that are testing an application through a given testplan. You might then define values for Engineer like Bob, Emily, Craig, and Zoe, the individual engineers who are testing this plan. You can then assign the values of Engineer to the tests in the testplan. Certain tests are assigned the value of Emily, others the value of Craig, and so on. You can then run a query to mark the tests that have a given value for the Engineer attribute. Finally, you can run just these marked tests. Attributes are also used to generate reports. For example, to generate a report on the number of passed and failed tests for Engineer Craig, simply select this value from the Pass/Fail Report dialog. You do not need to mark the tests or build a query in this case. Note This chapter does not describe how to run parts of a testplan, nor does it describe how to generate a report using attributes. These are described in Chapter 6, Running Tests and Interpreting Results.
236
Users Guide
11 CATEGORIZING AND MARKING TESTPLANS Defining and modifying attributes and values
Defining and modifying attributes and values

Attributes are used to categorize tests, so that you can reference them as a group. Attributes can also be incorporated into queries, which allow you to mark tests that match the querys criteria. Marked tests can be run as a group.
Predefined attributes
The testplan editor has three predefined attributes.

Attribute Description
Developer Component Category
Specifies the group of QA engineers who developed the testcases called by the testplan. Specifies the application modules to be tested in this testplan. Specifies the kinds of tests used in your QA Department, for example, Smoke Test.
What you can do
You can define up to 254 attributes. You can also rename the predefined attributes. Here are the rules for naming attributes: Attribute names can be up to 11 characters long Attribute and value names are not case sensitive
Naming rules
Attributes reside in the initialization file
Attributes and values as well as queries are stored by default in testplan.ini (the initialization file is specified in the Data File for Attributes and Queries field in the General Options dialog). The testplan.ini file is located in the SilkTest installation directory Make sure that all the QA engineers in your group use the same initialization file.
How to define attributes and their values
You can define up to 254 attributes. Procedure To define an attribute and its values: 1 Select Testplan/Define Attributes.
Users Guide
237
The Define Attributes dialog appears.
Click New. The New Attribute dialog appears.
3 4
Name the attribute. Select its type.

Type Description
Normal Edit
You specify values when you define the attribute. Users of the attribute in a testplan pick one value from the list. You dont specify values when you define the attribute. Users type their own values when they use the attribute in a testplan. Like Normal, except that users can pick more than one value.
Set
Note Attributes of type Edit and Set are supported on Windows only. They are ignored in testplans on non-Windows platforms. 5 6 Click OK. You return to the Define Attributes dialog. If you have defined an Edit type attribute, you are done. Click OK to close the dialog. If you are defining a Normal or Set type attribute, provide the allowable values for the attribute:
238
Users Guide
Type a value in the text field and click Add. Repeat this step for each value. The values display in the Values list in alphabetical order.
b
How to modify existing attributes and values
Click OK to close the dialog.
You can modify the definition of attributes. Procedure To modify the definition of an attribute: 1 2 Select Testplan/Define Attributes. Select the attribute you want to modify, then:
To... Action and result
Rename an attribute Edit its name in the Name field. Assign a new value to the attribute Modify a value Type the value in the text field at the bottom right of the dialog. Click Add. The value is added to the list of values. Select the value from the Values list box. Click Edit. The value appears in the text field at the bottom right of the dialog and the Add pushbutton is renamed Replace. Modify the value and click Replace. Select the value from the Values list box and click Remove. The text field is cleared and the value is removed from the Values list box. Click Delete.
Delete a value
Delete an attribute
Users Guide
239
11 CATEGORIZING AND MARKING TESTPLANS Assigning attributes and values to a testplan
When youre finished, take one of the following actions:

To... Action and result
Have your changes take effect Cancel all changes
Click OK. The dialog closes. The attributes and values are saved in the initialization file specified in the General Options dialog. Click Cancel. The dialog closes.
Important Modifying attributes and values through the Define Attributes dialog has no effect on existing attributes and values already assigned to the testplan. You must make the changes in the testplan yourself.
Assigning attributes and values to a testplan

Two ways to assign attributes and values
Attributes and values have no connection to a testplan until you assign them to one or more tests using an assignment statement. To add an assignment statement, you can do one of the following: Type the assignment statement yourself directly in the testplan. Use the Testplan Detail dialog.
What an assignment statement looks like
An assignment statement consists of the attribute name, a colon, and a valid attribute value, in this format: attribute-name: attribute value For example, the assignment statement that associates the Searching value of the Module attribute to a given test would look like:
Module: Searching
Attributes of type Set are represented in this format: attribute-name: attribute value; attribute value; attribute value; ...
Where to place the assignment statement
Whether you type an assignment statement yourself or have the Testplan Detail dialog enter it for you, the position of the statement in the plan is important.
To have an assignment statement apply to Place it directly after the
An individual test A group of tests
test description group description
240
Users Guide
11 CATEGORIZING AND MARKING TESTPLANS Assigning attributes and values to a testplan How to assign an attribute via the Testplan Detail dialog
Procedure To assign an attribute from the Testplan Detail dialog: 1 Place the cursor in the testplan where you would like the assignment statement to appear, either after the test description or the group description. Select Testplan/Detail to invoke the Testplan Detail dialog. Click on the Test Attributes tab. The Test Attributes tab is displayed.
2 3
The arrow in the list box at the top of the dialog identifies the test description at the cursor position in the testplan, for example, Case sensitive in the preceding figure. The attribute will be added to this test description. The Test Attributes tab lists all your current attributes at this level of the testplan. 4 Do one of the following: 5 If the attribute is of type Normal, select a value from the list. If the attribute is of type Set, select one or more values from the list. If the attribute is of type Edit, type a value.
Click OK.
Users Guide
241
11 CATEGORIZING AND MARKING TESTPLANS Assigning attributes and values to a testplan
The dialog closes and the assignment statement(s) are placed in the testplan.
Adding or removing members of a Set attribute
Tests can be assigned more than one value at a time for attributes whose type is Set. For example, you might have a Set variable called RunWhen with three values: UI, regression, and smoke. You can assign any combination of these three values to a test or group of tests. Separate each value with a semicolon. You can use the + or operator to add or subtract elements to what were previously assigned. Use + to add members Consider this example:
RunWhen: UI; regression Test 1 testcase: t1 RunWhen: + smoke Test 2 testcase: t2
In this example, Test 1 has the values UI and regression. The statement
RunWhen: + smoke
adds the value smoke to the previously assigned values, so Test 2 has the values UI, regression, and smoke. Use - to remove members Consider this example:
RunWhen: UI; regression Test 1 testcase: t1
242
Users Guide
11 CATEGORIZING AND MARKING TESTPLANS Marking a testplan RunWhen: - regression Test 2 testcase: t2
In this example, Test 1 has the values UI and regression. The statement
RunWhen: - regression
removes the value regression from the previously assigned values, so Test2 has the value UI. Note the following about using + and : You must follow the + or with a space. You can add or remove any number of elements with one statement. Separate each element with a semicolon. You can specify + element(s) even if no assignments had previously been made; the result is that the element(s) are now assigned. You can specify element(s) even if no assignments had previously been made; the result is that the sets complement is assigned. Using the previous example, specifying
RunWhen: - regression
when no RunWhen assignment had previously been made results in the values UI and smoke being assigned.
Marking a testplan
Marks are temporary denotations that allow you to work with selected tests in a testplan. Marks can be removed at any time, and they last only as long as the current work session.
Black stripe denotes a mark
You can recognize a marked testcase by the black stripe in the margin. The following figure shows two marked tests in the outline4 testplan.
Users Guide
243
11 CATEGORIZING AND MARKING TESTPLANS Marking a testplan
Marking commands are on the Testplan and Results menus
The Testplan menu contains seven marking commands Mark, Mark All, Unmark, Unmark All, Mark by Query, Mark by Named Query, and Find Next Mark. In addition, the Results menu contains the Mark Failures in Plan command. (The Results menu appears only when a results file is in the active window. For more information on results files, see Results files on page 148.) The following table describes each of the commands.
Use this marking command To
Testplan/Mark Testplan/Mark All Testplan/Unmark Testplan/Unmark All Testplan/Mark by Query
Mark the selected test(s). Mark every test in the plan. Unmark the selected test(s). Unmark every test in the plan. Run a query based on a set of criteria. Optionally name the query. For more information, see Creating and modifying testplan queries on page 246.
244
Users Guide
11 CATEGORIZING AND MARKING TESTPLANS Marking a testplan
Use this marking command
To
Testplan/Mark by Named Query
Build a query based on a set of criteria. Name the query. Edit, combine, and delete the query. Save and run the query, or save the query without running it.
For more information, see Creating and modifying testplan queries on page 246. Testplan/Find Next Mark Results/Mark Failures in Plan Locate the next set of marks in the plan. Mark all failed tests and make the testplan the active window.
Marking commands on the tool bar
Four of the commands are also available on the tool bar:
Find Next Mark Unmark All Unmark Mark How to mark one or more tests
Procedure To mark a single test, place the cursor on the test description and select Testplan/Mark. The test description and all the statements are marked. Procedure To mark a group of related tests, place the cursor on the group description and select Testplan/Mark. The group description, its associated statements, and all test descriptions and statements subordinate to the group description are marked. Procedure To mark two or more adjacent tests and their subordinate tests, select the test description of the adjacent tests and select Testplan/Mark. The testplan editor marks the test descriptions and statements of each selected test and any subordinate tests.
How the marking commands interact
When you apply a mark using the Mark command, the new mark is added to existing marks. When you mark tests via the query marking commands, the testplan editor by default clears all existing marks before running the query. Mark by Named Query supports sophisticated query combinations, and it would not make
Users Guide
245
11 CATEGORIZING AND MARKING TESTPLANS Creating and modifying testplan queries
sense to retain previous marks. However, Mark by Query, which allows onetime-only queries, lets you override the default behavior and retain existing marks. To retain existing marks, deselect the Unmark All Before Query check box in the Mark by Query dialog.
Printing marked tests
Procedure To print the testplan showing only marked tests, select File/Print and make sure the Print Marked Only check box is selected in the Print dialog, as well as any other options you want. Select OK to print the listing.
Creating and modifying testplan queries

Introduction
A testplan query is used to mark all tests that match a user-selected set of criteria, or test characteristics. A query comprises one or more of the following criteria: Testplan execution: script file, testcase name, or test data Test attributes and values Symbols and values
Query criteria
Test attributes and symbols must have been previously defined to be used in a query. For more information on defining symbols, see Typing data directly into the testplan on page 225. Example: You want to mark all tests in the testplan that meet all three of these criteria: They are defined in the script file find.t, the value of the Developer attribute is Dave, and the value of the symbol $TestLevel is 2.
Named versus unnamed queries
Queries can be named or unnamed. However, unnamed queries can be run only once. If you name the query, you can have the testplan editor run it in the same or subsequent work sessions without having to rebuild the query or manually remark the tests that youre interested in rerunning or reporting on. Testplan/Mark by Query or Testplan/Mark by Named Query both create queries; however, Mark by Named Query provides extra features, like the ability to combine queries or to create a query without running it immediately. If the query-creation function and the query-running function are distinct in your company, then use Mark by Named Query. If you intend to run a query only once, or run a query while keeping existing marks, then use Mark by Query.
Two query marking commands
246
Users Guide
The following table highlights the differences between the two commands.
Mark by Query Mark by Named Query
Builds a query based on criteria you select and runs query immediately. Name is optional, but note that only named queries are saved and can be rerun at any time (in the Mark by Named Query dialog). Cannot edit or delete a query. Cannot combine queries.
Builds a new query based on criteria you select. Can run query at any time. Name is required. Query is saved.
Can edit or delete a query. Can combine queries into a new query.
Lets you decide whether or not to clear Clears existing marks before running existing marks before running new new query. query. Unmarks by default.
Queries reside in the initialization file
Named queries are stored by default in testplan.ini (the initialization file is specified in the Data File for Attributes and Queries field in the General Options dialog). The testplan.ini file is in the SilkTest installation directory. Make sure that all the QA engineers in your group use the same initialization file.
How to create a new query
You can create a new query through either Testplan/Mark by Query or Testplan/Mark by Named Query. Procedure To create a new query: 1 Open the testplan and any associated subplans.
Users Guide
247
Select Testplan/Mark by Query or Testplan/Mark by Named Query.

If you chose Mark by Named Query
If you chose Mark by Query
The Mark by Query dialog appears. It has three tabs: Test Execution, Test Attributes, Symbols.
The Mark by Named Query dialog appears. The Testplan Queries list box displays existing queries.
Specify a name in the Query Name (Optional) field if you want to save the query. Deselect the Unmark All Before Query check box if you want to retain existing marks. By default, the marks are cleared.
Click the New pushbutton. The New Testplan Query dialog appears. It has the same tabs as the Mark by Query dialog (shown at left). Specify a name for the query in the Query Name field.
Which criteria do you want to include in the query? To include a script, testcase, or test data, use the Test Execution tab, shown below. Use the Script and Testcase pushbuttons to select a script and testcase, or type the full specification yourself.
Note: To build a query that marks only manual tests, enter the keyword manual in the Testcase field. Note: The wildcard characters * (asterisk) and ? (question mark) are supported for partial matches: * is a placeholder for 0 or more characters, and ? is a placeholder for 1 character.
248
Users Guide
Example 1: If you type find_5 (* in the Testcase field, the query would search all the testcase statements in the plan and mark those test descriptions that matched as well as all subordinate descriptions to which the matching testcase statement applied; that is, those where the find_5 testcase passed in data. Example 2: If you type find.t in the Script field, the query would search all script statements in the plan and mark those test descriptions that matched exactly as well as all subordinate descriptions to which the matching script statement applied; that is, those in which you had specified find.t exactly. It would not match any script statements in which you had specified a full path. To include existing attributes and values in the query, use the Test Attributes tab (shown in the following figure).
To include one or more existing symbols and values, use the Symbols panel (shown in the following figure). Type the information and click Add. The symbol and value are added to the list box Note: Do not type the dollar sign ($) prefix before the symbol name. Also, use ? (question mark) to indicate an unset value. For example, Mysymbol = ? in a query would mark those tests where Mysymbol is unset. Space around the equals sign (=) is insignificant.
Users Guide
249
If you need to modify the symbol in the query, select it from the list box and click Edit. The testplan editor places it in the text field and changes the Add pushbutton to Replace. Edit the symbol or value and click Replace. If you need to exclude the symbol from the query, select it from the list box and click Remove. The testplan editor deletes it from the list box. 4 Once you are satisfied with the query, do one of the following, depending on which command you chose to create the query.
Then
If you chose
Mark by Query
Click Mark to run the query against the testplan. The testplan editor closes the dialog and marks the testplan, retaining the existing marks if requested. Click OK to create the query. The New Testplan Query dialog closes, and the Mark by Named Query dialog is once again visible. The new query appears in the Testplan Queries list box. If you want to: Run the query, select it from the list box and click Mark. Close the dialog without running the query, click Close.
Mark by Named Query
Creating a new query by combining existing queries
You can combine two or more existing queries into a new query through the Mark by Named Query dialog. The new query can represent the union of the constituent queries (logical OR) or the intersection of the constituent queries (logical AND).
250
Users Guide
Combining by union Combining two or more queries by union creates a new named query that marks all tests that would have been marked by running each query one after the other while retaining existing marks. Since Mark by Named Query clears existing marks before running a query, the only way to achieve this result is to create a new query that combines the constituent queries by union. For example, suppose you have two queries, Query1 and Query2, that you want to combine by union.
Query1 Query2
Developer: Emily Component: Searching
Developer: Craig TestLevel: 2
The new query created from the union of Query1 and Query2 will first mark those tests that match all the criteria in Query1 (Developer is Emily and Component is Searching) and then mark those tests that match all the criteria in Query2 (Developer is Craig and TestLevel is 2). Combining by intersection Combining two or more queries by intersection creates a new named query that marks every test that has the criteria specified in all constituent queries. For example, combining Query1 and Query2 by intersection would create a new query that comprised these criteria: Developer is Emily and Craig, Component is Searching, and TestLevel is 2. In this case, the new query would not mark any tests, since it is impossible for a test to have two different values for the attribute Developer (unless Developer were defined as type Set under Windows). Use care when combining queries by intersection.
How to combine queries
Procedure To combine queries: 1 2 Select Testplan/Mark by Named Query to display the Mark by Named Query dialog. Click the Combine pushbutton. The Combine Testplan Queries dialog appears, as shown in the following figure. All existing named queries are displayed in the Queries to Combine list box.
Users Guide
251
3 4 5 6
Specify a name for the new query in the Query Name field. Select two or more queries to combine from the Queries to Combine list box. Select a radio button representing the combination method to use: either Union of Queries or Intersection of Queries. Click OK to save the new query. The Combine Testplan Queries dialog closes and the Mark by Named Query dialog is once again visible.
The new query is displayed in the Testplan Queries list box.

If you want to Then
Run the query and close the dialog. Close the dialog without running the query.
How to edit queries
Select the query and click Mark. Click Close.
You can modify an existing query through the Mark by Named Query dialog. Procedure To edit a query: 1 2 Select Testplan/Mark by Named Query to display the Mark by Named Query dialog. Select a query from the Testplan Queries list box and click Edit. The Edit Testplan Query dialog appears. The name of the query to be edited appears in the Query Name field. The tabs show the current criteria for the query. 3 4 Edit the information on the appropriate tabs. When you are finished editing the query, click OK. The Mark by Named Query dialog once again becomes visible.
252
Users Guide
The query youve just edited appears in the Testplan Queries list box.
If you want to Then
Run the query youve just edited and close the dialog. Close the dialog without running the edited query.
How to delete queries
Select the query and click Mark. Click Close.
Procedure To delete a query: 1 2 Select Testplan/Mark by Named Query to open the Mark by Named Query dialog. Select a query from the Testplan Queries list box and click the Remove pushbutton. The testplan editor prompts you to confirm the deletion. 3 4 Click Yes to delete the query. Click the Close pushbutton to close the dialog.
Users Guide
253
254
Users Guide
2rh 1e C t p a
Working With Large Testplans

For large or complicated applications, the testplan can itself become quite large. This raises three issues: How to keep track of where you are in the plan and what is in scope at that level How to determine which portions of the plan have been implemented How to allow several staff members to work on the plan at the same time
To determine context and scope, you can use the Testplan Detail dialog. To determine which portions of the plan have been implemented, you can produce a Completion report. To allow multiple users to work on the same plan, you can structure your testplan as a master plan with one or more subplans. These topics are all discussed in this chapter.
What you will learn

Topic Page
Navigating through a large testplan Measuring testplan completion Dividing a testplan into a master plan and subplans Editing a master plan in a multi-user environment
256 258 259 262
Users Guide
255
12 WORKING WITH LARGE TESTPLANS Navigating through a large testplan
Navigating through a large testplan

When working with a large testplan that contains many nested levels, it can become hard to keep track of just where you are in the plan and where inherited statements and values were defined. For example, consider the following expanded testplan:
If this were all you could see on your display, you would have difficulty understanding the current context of symbols and statements. Procedure To understand where values are defined in a large testplan and to optionally redefine values: 1 Place the insertion point at the relevant point in the testplan and select Testplan/Detail to open the Testplan Detail dialog. For example:
256
Users Guide
12 WORKING WITH LARGE TESTPLANS Navigating through a large testplan
Locally assigned symbols appear in black; symbols that inherit their value appear in blue. 2 To see just the set of symbols, attributes, and statements that are defined on a particular level, click on the level in the list box at the top of the Testplan Detail dialog. For example, clicking on the Find dialog group description level in the list box shows just the symbols defined at that level:
Once you find the level at which a symbol, attribute, or statement was defined, you can change the value at that level, causing the inherited value at the lower levels to change also.
Users Guide
257
12 WORKING WITH LARGE TESTPLANS Measuring testplan completion
Measuring testplan completion

Definition of completion
To measure your QA departments progress in implementing a large testplan, you can generate a Completion report. The Completion report considers a test complete if the test description is linked to a testcase, with two exceptions: If the testcase statement invokes a data-driven testcase and a symbol being passed to the data-driven testcase is assigned the value ? (undefined), the test will be considered incomplete. If the testcase is manual and marked as Incomplete in the Update Manual Tests dialog (available under Windows only), the test will be considered incomplete. (A manual testcase is indicated with the testcase:manual syntax; for more information, see Documenting manual tests in the testplan on page 65.)
How to generate a testplan Completion report
Procedure To generate a testplan Completion report: 1 2 Open the testplan you want to report on. Select Testplan/Completion Report: The Testplan Completion Report dialog appears:
3 4
In the Report Scope group box, indicate whether the report is for the entire plan or only for those tests that are marked To subtotal the report by a given attribute, select an attribute from the Subtotal by Attribute field. For example, suppose you have an attribute named Developer and have specified the values that Developer can take. If each test has been assigned a Developer value, you can generate a Completion report that shows how many of the tests for each developer are complete.
258
Users Guide
12 WORKING WITH LARGE TESTPLANS Dividing a testplan into a master plan and subplans
Note For more information on how to define attributes and attribute values, see Chapter 11, Categorizing and Marking Testplans. 5 Click Generate. The testplan editor generates the report and displays it in the lower half of the dialog. For example:
Note If the testplan is structured as a master plan with associated subplans, the testplan editor opens any closed subplans prior to generating the report. 6 You can: Print the report. Export the report to a comma-delimited ASCII file. You can then bring the report into a spreadsheet application that accepts commadelimited data. Chart (graph) the report, just as you can chart a Pass/Fail report. For more information, see Generating a testplan Pass/Fail chart on page 164 (everything in that section also applies to charting Completion reports, except for the description of adding results from another execution of the testplan, which applies only to Pass/Fail reports).
Dividing a testplan into a master plan and subplans

Introduction
If several engineers in your QA department will be working on a testplan, it makes sense to break up the plan into a master plan and subplans. The master plan contains only the top few levels of group descriptions, and the subplans
Users Guide 259
contain the remaining levels of group descriptions and test descriptions. This approach allows multi-user access, while at the same time maintaining a single point of control for the entire project. For example, consider the following testplan:
If you wanted to break this testplan up into a master plan and subplans, one approach would be to place the case-sensitive tests in one subplan and the case-insensitive tests in another, as shown in the following figure:
Note that subplans are specified with an include statement. To expand the subplan files so that they are visible within the master plan, double-click in the left margin next to the include statement. For example, here is how the preceding master testplan looks when the casesens.pln and the caseinsv.pln testplan files are expanded inline:
260
Users Guide
Once a subplan is expanded inline, the subplan statement changes from red (the default color for statements) to magenta, indicating that the line is now read-only and that the subplan is expanded inline. At the end of the expanded subplan is the <eof> marker, which indicates the end of the subplan file.
Subplans inherit Information
Statements, attributes, symbols, and test data defined in the master plan are accessible within each of the subplans. For example, consider the following master plan:
The subplans contained in the casesens.pln file and the caseinsv.pln file each inherit, and can therefore use, the following from the master plan: The script statement The testcase statement The assignment statement for the $Text symbol
Users Guide
261
12 WORKING WITH LARGE TESTPLANS Editing a master plan in a multi-user environment Creating the subplan
You create a subplan in the same way you create any testplan: by opening a new testplan file and entering the group descriptions, test descriptions, and the testplan editor statements that comprise the subplan, either manually or using the Testplan Detail dialog. To connect the master plan to a subplan file, you enter an include statement in the master plan at the point where the subplan logically fits. The include statement cannot be entered through the Testplan Detail dialog, so it must be entered manually. The include statement uses this syntax:
include: myinclude.pln
Adding the include statement to the master plan
where myinclude is the name of the testplan file that contains the subplan. If you enter the include statement correctly, it appears in red, the default color used for the testplan editor statements. Otherwise, the statement appears in blue or black, indicating youve made a syntax error (the compiler is interpreting the line as a description, not a statement).
Sharing a testplan initialization file
All QA engineers working on a large testplan that is broken up into a master plan and subplans must use the same testplan initialization file. To do this, all engineers should specify the same file name in the Data File for Attributes and Queries field in the General Options dialog (invoked by selecting Options/General).
Editing a master plan in a multi-user environment

Opening the subplan
You should open the subplan from within the master plan. To do this, you can either double-click in the margin to the left of the include statement or highlight the include statement and select Include/Open. (Compiling a script also automatically opens all subplans.) Note If a subplan does not inherit anything (that is, statements, attributes, symbols, or data) from the master plan, you can open the subplan directly from the File Open dialog.
Acquiring a lock
When first opened, a master plan and its related subplans are read-only. This allows many users to open, read, run, and generate reports on the plan. When you need to actually edit the master plan or a subplan, you must first acquire a lock, which prevents others from making changes that conflict with your changes.
262
Users Guide
12 WORKING WITH LARGE TESTPLANS Editing a master plan in a multi-user environment
Procedure To acquire a lock, place the cursor in or highlight one or more subplans and then select Include/Acquire Lock. The bar in the left margin of the testplan changes from gray to yellow. For example, the following figure (even though gray scale) shows that after a lock is acquired for the casesens.pln subplan, the bar in the left margin changes color:
Releasing a lock
Procedure To release a lock, select Include/Release Lock. The margin bar changes from yellow to gray. For example, here is how the plan shown in the preceding figure changes when the lock is released:
Users Guide
263
12 WORKING WITH LARGE TESTPLANS Editing a master plan in a multi-user environment Copying a subplan
When you copy and paste the include statement and the contents of an open include file, note that only the include statement will be pasted. To view the contents of the subplan, open the pasted include file by selecting Include/ Open or by double-clicking in the margin to the left of the include statement. When finished editing, select Include/Save to save the changes to the subplan currently being edited. This is distinct from File/Save, which saves all open master plans and subplans. When another user modifies a subplan, those changes are not automatically reflected in your read-only copy of the subplan. Once the other user has released the lock on the subplan, there are two ways to refresh your copy: Close and then reopen the subplan Acquire a lock for the subplan
Saving changes
Refreshing a local copy
264
Users Guide
Cross-Platform Testing
I II tP r a
In this part

Chapter Page
Chapter 13, Porting Tests to Other GUIs Chapter 14, Supporting Internationalized Applications
267 283
Users Guide
265
266
Users Guide
3rh 1e C t p a
Porting Tests to Other GUIs

Using SilkTest, you can create portable testcases which will test your application on any of the supported GUIs. The reason for this is that your testcases use logical names, called identifiers, to refer to the GUI objects, and not actual names, called tags. Therefore, if there are differences in the ported applications appearance, you need only change the window declarations, not the testcases themselves. This chapter contains the following sections:
Topic Page
Introduction
What you will learn
Marking 4Test code as GUI-specific Conditional compilation Supporting GUI-specific executables Supporting GUI-specific captions Supporting GUI-specific menu hierarchies Supporting extra controls Supporting different implementations of the same control Supporting differences in application behavior
267 271 273 274 275 275 276 279
Marking 4Test code as GUI-specific

Introduction
The porting scenarios described in this chapter use 4Test keywords called GUI specifiers to indicate that portions of include files or script files are specific to a particular GUI. Before studying these scenarios, you should understand which GUI specifiers are available and how to use them in your include files and script files.
Users Guide
267
13 PORTING TESTS TO OTHER GUIS Marking 4Test code as GUI-specific List of available GUI specifiers Syntax of a GUI specifier
For a complete list of GUI specifiers, see GUITYPE data type in online Help. A GUI specifier has this syntax: [[gui-type [, gui-type]] | [! gui-type]] gui-type is the GUI. You can express this in one of two mutually exclusive ways. For example, you can specify one or more GUIs separated by commas, as in:
motif, msw
Or you can specify all but one GUI, as in the following, which indicates that what follows applies to all environments except Macintosh:
! mac
Where you use GUI specifiers

A GUI specifier can appear before any 4Test declaration or statement except the use statement, which must be evaluated at compile time, with the following exceptions.
switch statements
You can use GUI specifiers before an entire switch statement and before individual statements within a case clause, but you cannot use GUI specifiers before entire case clauses.
// legal: msw, motif switch (i) case 1 msw Print ("hello") motif Print ("goodbye") case 2 msw raise 1, "error" motif Print ("continue") default msw Print ("ok") // NOT legal: switch (i) msw case 1 Print ("hello") motif case 1 Print ("goodbye")
if statements
You can use GUI specifiers in if statements, as long as GUI specifiers used within the statement are subsets of any GUI specifiers that enclose the entire if statement:
268
Users Guide
13 PORTING TESTS TO OTHER GUIS Marking 4Test code as GUI-specific // legal because no GUI specifier // enclosing entire if statement: if (i==j) msw, motif Print ("hi") mac Print ("bye") // legal because msw is a subset of enclosing specifier: msw, mac if (i==j) msw Print("hi") // legal for the same reason as preceding example: msw, motif if (i==j) Print ("hi") msw else Print ("Not the same") // NOT legal because mac is not a subset // of the enclosing GUI specifier msw: msw if (i==j) mac Print ("bye") // Invalid GUI type type statements
You can use a GUI specifier before a type ... is enum or type ... is set statement, but not before an individual value within the declaration. You can use GUI specifiers to enclose an entire do...except statement and before individual statements, but you cannot use GUI specifiers before the except clause.
// legal: do msw Verify (expr1,expr2) motif Verify (expr3,expr4) except motif reraise msw if (ExceptNum () == 1) Print ("err, etc.") // NOT legal: msw do Verify (expr,expr) msw except reraise
do...except statements
Class declarations
Be careful using GUI specifiers before class declarations; they can be ambiguous. Any ambiguities must be resolvable at compile-time.
// bad style: msw winclass myclass mac winclass myclass window myclass inst // Ambiguous. Is it an instance of
Users Guide
269
13 PORTING TESTS TO OTHER GUIS Marking 4Test code as GUI-specific // the msw class or the mac class?
The preceding examples ambiguity can be resolved by specifying a GUI target with conditional compilation (so that, for example, only code for msw gets compiled, in which case inst would be an instance of the msw class myclassconditional compilation is described next) or by explicitly using a GUI specifier for the window, as follows:
// good style: msw winclass myclass mac winclass myclass msw window myclass inst With inheritance
When using GUI specifiers for parent classes, you must explicitly use the GUI specifiers with the descendants:
msw95 winclass newclass msw95 winclass subclass : newclass msw95 window subclass inst
With global variables
Be careful when using GUI specifiers with global variables: SilkTest initializes global variables before connecting to an Agent. This might not give you the results you want if you are doing distributed testing. Lets say that you are running tests on a remote machine that is listed in Runtime Options. Because SilkTest initializes all global variables before connecting to an Agent, any GUI specifier at the global level will initialize to the host machine, not the target machine you want to test against. For example, say the host machine is running Window 95 and the target machine is running Windows NT 4.0. Consider the following script:
mswnt40 STRING sVar1 = SYS_GetEnv("UserName") msw95 STRING sVar1 = SYS_GetRegistryValue (HKEY_LOCAL_MACHINE, "System\CurrentControlSet\Control", "Current User") main() print(sVar1)
This script fails, with the error message

*** Error: Registry entry 'Current User' not found
because sVar1 is initialized to the value for the host system (the GUI specifier msw95), not the target system (mswnt40). So, if you want GUI specifiers on variables at the global level, put the initialization of these variables in the main function or another function that is called after the Agent is contacted.
270
Users Guide
13 PORTING TESTS TO OTHER GUIS Conditional compilation
Conditional compilation
If you have GUI-specific code in your scripts and declarations, you can have SilkTest conditionally compile your code based on the values of the GUI specifiersonly code specific to a particular GUI is compiled (as well, of course, as all code that is not GUI-specific). This has two advantages: The compilation is faster The resulting code is smaller and require less memory to run
You can also cause conditional compilation by using constants, which are evaluated at compile time. Note Constants are not restricted to conditional compilation. You can use constants for any value that you want resolved at compile time. Procedure To conditionally compile code: 1 2 Prefix any 4Test statements that are GUI-specific with the appropriate GUI specifier, as described in the previous section. Specify the platforms that you want to compile for by entering the appropriate GUI specifiers in the GUI Targets field in the Runtime Options dialog. You can specify as many GUI targets as you want; separate each GUI specifier by a comma.
Users Guide
271
13 PORTING TESTS TO OTHER GUIS Conditional compilation
Note Setting a GUI target affects which classes are listed in the Library Browser. 3 To conditionalize code based on the value of constants you define, do the following: a Click the Compiler Constants pushbutton in the Runtime Options dialog. The Compiler Constants dialog is displayed.
b c 4
What happens
Define a constant and specify its value. Use the constant in your code anywhere you can specify an expression.
Click OK to close the Runtime Options dialog.
When the code is compiled: Only code relevant to the GUI environments specified in the GUI Targets field (plus all common code) will be compiled. If you dont list any GUI specifiers in the GUI Targets field, all code will be compiled; at runtime, code not relevant to the platform the application is running on will be skipped. The constants you have defined are evaluated and used to compile the code. You can use this feature to conditionally load include files, as described next.
Conditionally loading include files
If you are testing different versions of an application, such as versions that run on different platforms or versions in different languages, you probably have different include files for the different versions. For example, if your applications runs under different languages, you might have text strings that appear in windows defined in different include files, one per language. You want SilkTest to load the proper include file for the version of the application you are currently testing.
272
Users Guide
13 PORTING TESTS TO OTHER GUIS Supporting GUI-specific executables
Procedure To load different include files for different versions of the test application: 1 2 Define a compiler constant, as described above. For example, you could define a constant named MyIncludeFile. Insert the following statement into your 4Test file: use constant For example, if you defined a constant MyIncludeFile, insert the following statement:
use MyIncludeFile
Note In the above example, constant can also be an expression that evaluates to a constant at compile time. 3 When you are ready to compile your 4Test files, specify the file name of the include file you want loaded as the value of the constant in the Compiler Constants dialog (be sure to enclose the value in quotation marks if it is a string).
Compile your code. SilkTest evaluates all compiler constants and substitutes their values for the constants in your code. In this case, the constant MyIncludeFile will be evaluated to a file, which will be loaded through the use statement.
Supporting GUI-specific executables

The command to start the application will almost always be different on each GUI. SilkTests Invoke method expects to find the command in the constant sCmdLine, which is defined in your applications main window declaration.
Users Guide
273
13 PORTING TESTS TO OTHER GUIS Supporting GUI-specific captions
You should declare as many sCmdLine variables as there are GUIs on which your application runs, beginning each declaration with the appropriate GUI specifier. For example, the following constants specify how SilkTest should start the Text Editor application on Windows and Macintosh:
msw const sCmdLine = "c:\sample\texted2.exe" mac const sCmdLine = "{SYS_GetDrive ()}:Text Editor:TextEd2"
Supporting GUI-specific captions

Introduction
Recall that SilkTest, by default, bases the tag for an object on the objects actual caption or label. If the captions or labels change when the application is ported to a different GUI, you have two options: You can have multiple tags, each based on the platform-specific caption or label. You can have a single tag, using the index form of the tag, as long the relative position of the object is the same in the ported versions of the application.
Then, in your testcases, you can use the same identifier to refer to the object regardless of what the objects actual label or caption is.
Creating GUI-specific tags
To close a file on the Macintosh, you select File/Quit, whereas on all other platforms you select File/Exit. The following window declaration accounts for these differences with two tag statements:
MenuItem Exit tag "Exit" mac tag "Quit"
With this declaration, the Exit identifier can be used to refer to the menu item regardless of the actual label.
Using the index as the tag
If you are certain that an objects position in relation to its sibling objects of the same class will remain the same when the application is ported, you can use the index form for the tag. Repeating the example from the preceding section, because the Exit/Quit menu item is the fifth menu item on the File menu (on all platforms), you can use the index form for the tag (#5) as shown here:
MenuItem Exit tag "#5"
274
Users Guide
13 PORTING TESTS TO OTHER GUIS Supporting extra controls Deciding which form of tag to use
When an objects caption or label changes on a different GUI, it is usually preferable to use multiple tags, each based on the GUI-specific label or caption, instead of using the index. Not only does it make your declarations easier to understand, but it shields your testcases from changes to the sequence of child objects. For example, if the Exit item changes so that it is the fourth item and not the fifth, your testcases will still run.
Supporting GUI-specific menu hierarchies

Two common differences in menu hierarchy
When an application is ported, there are two common structural differences in the menu hierarchy: The menu bar contains a platform-specific menu. A menu contains different menu items.
Example: a platformspecific menu
To illustrate the first case, consider the Microsoft Windows system menu or the Macintosh Apple menu. SilkTest recognizes these kinds of standard GUIspecific menus and includes the appropriate GUI specifier for them when you record declarations, as shown in this declaration for the Macintosh Apple menu:
mac Menu AppleMenu
For menus that SilkTest does not recognize as platform-specific, you should preface the window declaration with the appropriate GUI specifier.
Example: different items on a menu
To illustrate the second case, suppose that the Edit menu for the Text Editor application has a menu item named Clear which appears on the Windows version only. The declaration for the Edit menu should look like this:
Menu Edit tag "Edit" msw MenuItem Clear tag "Clear" MenuItem Undo tag "Undo"
Supporting extra controls

You may find that the controls within a dialog are slightly different from platform to platform. For example, consider the Open dialog of the Text Editor application.
Users Guide
275
13 PORTING TESTS TO OTHER GUIS Supporting different implementations of the same control Dialog on Windows
The Windows GUI allows an end user to enter the file name to be opened as a path name in a text field; the Macintosh GUI does not. Therefore, the following two declarations are recorded under Windows, but not under Macintosh:
StaticText FileNameText tag "File Name:" TextField FileName1 tag "File Name:"
When merging the window declarations for the two GUIs, you need to preface these declarations with the msw GUI specifier:
msw StaticText FileNameText tag "File Name:" msw TextField FileName1 tag "File Name:" Dialog on Macintosh
Similarly, the Macintosh version of the dialog has two pushbuttons which do not appear on the Windows version. The first pushbutton allows the user to eject a floppy disk from the disk drive. The second pushbutton allows the user to navigate to the desktop directory. Therefore, the following two declarations are recorded under Macintosh, but not under Windows:
PushButton Eject tag "Eject" PushButton Desktop tag "Desktop"
So, when merging the window declarations for the two GUIs, you need to preface these declarations with the mac GUI specifier:
mac PushButton Eject tag "Eject" mac PushButton Desktop tag "Desktop"
Supporting different implementations of the same control

One logical control can have two implementations
Consider the case where the same logical control in your application is implemented using different classes on different GUIs: If the kinds of actions you can perform against the object classes are similar, and if SilkTest uses the same method names for the actions, then you do not have a portability problem to address.
276
Users Guide
13 PORTING TESTS TO OTHER GUIS Supporting different implementations of the same control
For example, the methods for the RadioList and PopupList classes have identical names, because the actions being performed by the methods are similar. Therefore, if a control in your application is a popup list on one GUI and a radio list on another, your scripts are already portable. If the two object classes do not have similar methods, or if the methods have different names, then you need to follow the steps outlined in this section to port your scripts.
Creating a class that maps to several SilkTest classes
Consider the Direction control in the Find dialog of the Text Editor application, which allows a user to specify the direction (up or down) of searches. Suppose that this control is implemented as a check box on the Macintosh, but as a radio list on all other GUIs. As a radio list, the user selects either the Up or the Down radio button. On the Macintosh, the user checks the check box to select Up, and leaves the check box unchecked to select Down. The first step in solving this portability scenario is to create a new window class that you will use for the object on all platforms. The class you need to define, in effect, generalizes several distinct 4Test classes into one logical class. To achieve this generalization, you: Derive the new class from the 4Test Control class, since both radio lists and check boxes derive from this class. Define the class with a GUI-specific tag statement for each platform. Each tag statement states the actual class of the control on the particular GUI. This allows SilkTest to know what the actual class on the control will be at runtime on each of the GUIs. Define generalized methods that use a switch statement to branch to the 4Test code that implements the method on the particular GUI.
Here is the class declaration, which is arbitrarily named DirButton:

// The class is derived from Control winclass DirButton : Control tag "[RadioList]" mac tag "[CheckBox]" void Select (LISTITEM Item optional) BOOLEAN bState switch (GetGUIType ()) case mac bState = (Item == "Up") CheckBox (WndTag).SetState (bState) Users Guide 277
13 PORTING TESTS TO OTHER GUIS Supporting different implementations of the same control default RadioList (WndTag).Select (Item)
Note the following: The Select method acts against the control, regardless of whether it is a RadioList or CheckBox. The method contains a switch statement which executes the SetState method if the control is a check box, and the Select method if the control is a radio list. Note The Select method also takes an optional parameter, as indicated by the keyword optional. For more information, see Functional declaration in online Help. Because the tag of the object will differ on each GUI, rather than specifying an identifier in the SetState and Select method calls, you use the WndTag property. By doing this, you force SilkTest to construct a dynamic identifier for the object at runtime which will uniquely identify the object as a check box in the Macintosh case and as a radio list in all other cases.
Changing the window declaration of the control
The next step is to change your window declarations so that the control has the new class. Continuing the example from the last section, you change the class of the control named Direction to DirButton.
window DialogBox Find tag "Find" parent TextEditor DirButton Direction tag "Direction"
Using cross-platform methods in your scripts
Finally, in your scripts, you can use your cross-platform method names. The window declarations map the cross-platform method names you use in your scripts to the actual methods required to carry out the actions you want on each of the GUIs. Continuing the example from the last section, you use the Select method in your code to select the control named Direction.
testcase SearchBackward () LISTITEM Item Item = "Up" Find.Invoke () Find.Direction.Select (Item) . .
278
Users Guide
13 PORTING TESTS TO OTHER GUIS Supporting differences in application behavior . Find.Dismiss ()
Note that the script does not indicate that anything unusual is happening; all of the steps necessary to make the Select method work properly, regardless of the class of the object, are encapsulated in the class and window declarations.
Supporting differences in application behavior

Introduction
Although you can account for differences in the appearance of your application in the window declarations, if the applications behavior is fundamentally different when ported, you need to modify your testcases themselves. To modify your testcases, you write sections of 4Test code that are platform-specific, and then branch to the correct section of code using the return value from the GetGUIType built-in function. This section first shows how to use the GetGUIType function in conjunction with the if and the switch statements. It then presents some examples of common implementation differences.
GetGUIType function
The GetGUIType function returns the current GUI. It is a matter of personal preference whether you test the return value from the function in an if statement or a switch statement. You typically use a switch statement when there are more than two logic branches. For example, suppose a feature is implemented one way on the Macintosh, but a second way on all other platforms:
if (GetGUIType () != mac) // test feature on all platforms but Macintosh else // test feature on Macintosh
If your application is deployed on multiple platforms, and each platform implements the same feature slightly differently, you would probably want to use the switch statement in a construction like the following:
switch (GetGUIType ()) case mac: // code to test feature on macintosh case mswdos: // code to test feature on windows case motif: // code to test feature on motif
Users Guide
279
13 PORTING TESTS TO OTHER GUIS Supporting differences in application behavior case pm: // code to test feature on Presentation Manager Text field requires Return keystroke
On some GUIs, the Enter/Return key must be pressed after data is entered into a text field. Suppose you want to create a testcase which enters invalid data into the field, and then checks if the application detects the error. After the testcase enters the invalid data, it needs to use the GetGUIType function to determine the GUI, and then press the Return key if the GUI requires it. For example:
// code to enter an invalid string into field if (GetGUIType () == motif) MyTextField.TypeKeys ("<Return>") // code to verify that application detected error
Error messages are different
The VerifyErrorBox function, shown below, illustrates how to solve the problem of different error messages on each GUI platforms. In this example, Motif always adds the prefix Error: to its message, while the other platforms do not.
VerifyErrorBox (STRING sMsg) // verifies that the error box has the correct error // message, then dismisses the error box const ERROR_PREFIX = "ERROR: " const ERROR_PREFIX_LEN = Len (ERROR_PREFIX) STRING sActMsg = MessageBox.Message.GetText () // strip prefix "ERROR: " from Motif error messages if (GetGUIType () == motif) sActMsg = SubStr (sActMsg, ERROR_PREFIX_LEN + 1) Verify (sActMsg, sMsg) MessageBox.Accept ()
Mechanism for navigating directories is different
In the Windows Open dialog, to navigate to a directory you can enter the path name of the directory into a text field. The Macintosh GUI, however, does not allow you to enter paths; you must instead navigate to a working directory by selecting successive directories from a list. The SetWorkingDirectory function, shown below, illustrates how to solve this problem. The function receives the sPath parameter, which is the path of the directory to change to. The function contains a case statement which For the Macintosh, parses sPath into its constituent directories and double-selects each directory name in succession until the working directory is reached.
280
Users Guide
13 PORTING TESTS TO OTHER GUIS Supporting differences in application behavior
For all other GUIs, enters the value of the sPath variable directly into the appropriate text field.
SetWorkingDirectory (STRING sPath) // this function sets the working directory switch (GetGUIType ()) case mac INTEGER iFolder = 1 STRING sFolder UserSetup.Invoke () // : is the directory separator on the mac sFolder = GetField (sPath, ':', iFolder) while (sFolder != '') SelectFolder.Folder.DoubleSelect (sFolder) iFolder += 1 sFolder = GetField (sPath, ':', iFolder) SelectFolder.CurrentFolder.Click () UserSetup.Dismiss () default UserSetup.Invoke () UserSetup.Directory.SetText (sPath) UserSetup.Accept ()
Users Guide
281
13 PORTING TESTS TO OTHER GUIS Supporting differences in application behavior
282
Users Guide
4rh 1e C t p a
Supporting Internationalized Applications

This chapter covers these topics:
Topic Page
Built-in support for international keyboards Internationalizing tags
283 284
Built-in support for international keyboards

Changing the keyboard layout
SilkTest provides built-in support for testing applications that use international keyboards. If your script is testing an application that uses only a single international keyboard, you do not need to modify your scripts. If your script needs to switch among keyboards, then you need to set the OPT_ KEYBOARD_LAYOUT option from within the script. The syntax is:
Agent.SetOption (OPT_KEYBOARD_LAYOUT,sLayoutName)
sLayoutName is a an operating system-specific name expressed as a string. On Windows NT the name represents the code page number. On Windows 3.1, sLayoutName is an internal name. On Windows 95 and Windows 98, sLayoutName is an English-like string, as in:
Agent.SetOption (OPT_KEYBOARD_LAYOUT,"English (Britain)")
Check your Windows documentation to determine what string is expected. You might also invoke the GetOption method on OPT_KEYBOARD_LAYOUT to determine your current layout. For example:
Print (Agent.GetOption (OPT_KEYBOARD_LAYOUT))
Users Guide
283
14 SUPPORTING INTERNATIONALIZED APPLICATIONS Testing applications with single-byte international characters
Testing applications with single-byte international characters

To test applications using single-byte international character sets you must manually add information to your partner.ini file. Add the following entry to partner.ini:
[RecordOptions] InternationalKeyboard=TRUE
With this change, the hotkey for all recording dialogs becomes Ctrl+Shift instead of Ctrl+Alt.
Internationalizing tags
If your application is localized into multiple languages, then the captions and labels upon which SilkTest bases the tags it records are valid only for the localized version you ran when recording your window declarations. One alternative is to use index numbers for the tags instead of the caption/ label. However, this approach has the drawback that you are not able to verify that the labels are correct when you are testing your applications GUI objects. To make your declarations valid for all localized versions, one alternative is: 1 2 3 Replace every tag in your declarations with a STRING variable, if the tag would otherwise be constructed from a caption or label. Create an enumerated type of the possible languages. You use a variable of this new enumerated type as an index into the STRING variable. Assign to each STRING variable the correct localized string, by indexing into a table of strings.
At runtime, SilkTest substitutes the appropriate localized string for each variable. The rest of this section explains each of these steps.
Replace tags with string variables
For example, the following portion of the window declarations for the Text Editor application shows the declarations for the File menu and for the Find dialog. Note that each tag has been replaced with a STRING variable.
window MainWin TextEditor msw tag sMainWin Menu File tag sFile
284
Users Guide
14 SUPPORTING INTERNATIONALIZED APPLICATIONS Internationalizing tags MenuItem New tag sNew MenuItem Open tag sOpen MenuItem Save tag sSave MenuItem SaveAs tag sSaveAs MenuItem Exit tag sExit // omitted menu declarations window DialogBox Find tag sFind parent TextEditor StaticText WhatText tag sFindWhatText TextField What tag sFindWhat CheckBox CaseSensitive tag sCaseSensitive RadioList Direction tag sDirection PushButton FindNext tag sFindNext PushButton Cancel tag sCancel
SilkTest retrieves the value for each STRING variable using the language index. You define the index and initialize the STRING variables as described in the next two sections.
Create an enumerated type for the languages
You should create an enumerated type like the one shown below which defines the set of possible languages into which your application can be localized. Although the values of the enumerated type can be any meaningful set of strings, you might want to use the two-letter International Standard IS 639 language codes, as shown in this example:
type LANGUAGETYPE is enum // uses the 2-letter IS 639 language code. LT_DE // German LT_EN // English LT_ES // Spanish
Use current language as index
Then, create and initialize a constant or a variable that specifies the language that you want to test when you run your scripts. For example, to test the English version of an application, you can create and initialize this constant:
Users Guide 285
14 SUPPORTING INTERNATIONALIZED APPLICATIONS Internationalizing tags const LANGUAGETYPE LANGUAGE = LT_EN Load string variables with the localized strings STRING STRING STRING STRING STRING STRING STRING STRING STRING STRING STRING STRING STRING STRING
The final step is to create and initialize a STRING variable for each object in the application, as shown in this example:
sFile = {"Datei", "File", "Archivo"}[LANGUAGE] sNew = {"Neu", "New", "Nuevo"}[LANGUAGE] sOpen = {"?ffnen", "Open", "Abrir"}[LANGUAGE] sSave = {"Speichern", "Save", "Guardar"}[LANGUAGE] sSaveAs = {"Speichern unter","Save As", "Guardar como"}[LANGUAGE] sExit = {"Beenden", "Exit", "Salir"}[LANGUAGE] sFind = {"Suchen", "Find", "Buscar"}[LANGUAGE] sWhatText = {"Suchen nach:", "Find What:", "Buscar"}[LANGUAGE] sWhat = {"Suchen nach:", "Find What:", "Buscar"}[LANGUAGE] sDirection = {"Suchrichtung", "Direction", "Direcci?n"}[LANGUAGE] sUp = {"Aufw?rts", "Up", "Arriba"}[LANGUAGE] sDown = {"Abw?rts", "Down", "Abajo"}[LANGUAGE] sFindNxt={"Weitersuchen","Find Next",Buscar siguiente"}[LANGUAGE] sCancel = {"Abbrechen", "Cancel", "Cancelar"}[LANGUAGE]
Each of those statements picks a value from a list based on the language, which serves as the index into the list. For example, if LANGUAGE is LT_EN, which is the second member of the enumerated data type LANGUAGETYPE, then the first statement would evaluate to:
STRING sFile = {"Datei", "File", "Archivo"}[2]
So sFile would be assigned the value File, which is the second element in the list. For more information about using lists, see LIST data type in online Help. Note If your application uses distinct captions/labels for each GUI, then you should create multiple versions of each STRING variable using GUI specifiers. See Chapter 13, Porting Tests to Other GUIs, for more information. An alternative approach is to create a two-dimensional list of strings, and use the GUI type and the language as the indices.
286
Users Guide
Customizing SilkTest
V tP I a r
In this part

Chapter Page
Chapter 15, Understanding the Recovery System Chapter 16, Extending the Class Hierarchy Chapter 17, Supporting Custom Objects Chapter 18, Modifying the Library Browser
289 303 317 327
Users Guide
287
288
Users Guide
5rh 1e C t p a
Understanding the Recovery System

Topic Page
What you will learn
Introduction to the recovery system Setting up the recovery system The recovery systems flow of control How the recovery system starts the application How the recovery system closes windows Specifying new window closing procedures Specifying windows to be left open Handling login windows Adding to the default base state Overriding the default recovery system Modifying the default recovery system
289 290 291 292 292 293 296 297 298 299 301
Introduction to the recovery system

The built-in recovery system is one of SilkTests most powerful features, because it allows you to run tests unattended. When your application fails, the recovery system restores the application to a stable state, known as the base state, so that the rest of your tests can continue to run unattended.
Users Guide
289
15 UNDERSTANDING THE RECOVERY SYSTEM Setting up the recovery system
Note This chapter focuses on the recovery system used in non-Web applications. For information about the recovery system that is used when you are testing Web applications, see The recovery system for Web applications in Getting Started: A Tutorial.
When the recovery system is used
SilkTest uses its recovery system for all testcases that are based on DefaultBaseState or based on a chain of application states that ultimately are based on DefaultBaseState. The recovery system is not used with testcases based on an application state of None or based on a chain of application states ultimately based on None. For example, if you define a testcase and specify (None) in the Application State list box, it will not use the recovery system. Such a testcase will be defined in the script file as:
testcase Name () appstate none
SilkTest records testcases based on DefaultBaseState as:

testcase Name ()
For more information about application states, see Using application states on page 185.
The base state
The base state is the state you expect the application to be in at the start of each testcase. Although this can differ from application to application, the default recovery system considers an application to be at the base state when The application is running The application is not minimized The application is active No other window besides the main window is open
If this is not sufficient or if the recovery system is not able to achieve one or more of these conditions, you need to modify the recovery system.
Setting up the recovery system

The recovery system works differently when you are testing Web applications than when you are testing non-Web applications. If you plan to test Web applications, make sure you enable the correct browser extension(s) on your target and host machine. For information on how to enable browser extensions, see Enabling browser extensions on page 72.
290
Users Guide
15 UNDERSTANDING THE RECOVERY SYSTEM The recovery systems flow of control
If you do not plan to test Web applications, you must disable all browser extensions on the host machine.
The recovery systems flow of control

Before you modify the recovery system, you need to understand the flow of control as each of your testcases executes. Here is the sequence of steps: 1 Execute the DefaultTestcaseEnter function. This function, in turn, calls the SetAppState function, which does the following: a Execute the DefaultBaseState application state, which makes sure that the application is running, is not minimized, has only its main window open, and is active (in that order). The DefaultBaseState application state also executes the BaseState method of the main window. This method is the place to add those additional steps that your application requires to be at the base state. b If there is a chain of application states associated with the testcase, execute each one in succession, starting at the lowest application state in the chain. An application state is chained with another by using the basedon keyword in its definition. Thus, there can be many application states, each of which is based on the previous in the chain.
2 3
Execute the testcase. Execute the DefaultTestcaseExit function, which calls the SetBaseState function, which calls the DefaultBaseState application state.
The following diagram shows the order of execution of the various statesetting routines:
Users Guide
291
15 UNDERSTANDING THE RECOVERY SYSTEM How the recovery system starts the application
DefaultBaseState application state
BaseState method of main window
Application states of testcase
Testcase
error?
DefaultBaseState application state
BaseState method
How the recovery system starts the application

To start the non-Web application, the recovery system executes the Invoke method for the main window of the application. The Invoke method relies on the sCmdLine constant as recorded for the main window when you create a test frame. For example, here is how a declaration for the sCmdLine constant might look for the sample Text Editor application running under Windows:
const sCmdLine = "d:\appdir\texted2.exe"
After it starts the application, the recovery system checks whether the main window is minimized, and, if it is, uses the Restore method to open the icon and restore the application to its proper size.
How the recovery system closes windows

As explained in the introduction to this chapter, the built-in recovery system restores the base state by making sure that the non-Web application is running, is not minimized, is active, and has no open windows except for the main window.
292
Users Guide
15 UNDERSTANDING THE RECOVERY SYSTEM Specifying new window closing procedures
To ensure that only the main window is open, the recovery system attempts to close all other open windows, using an internal procedure that you can customize when necessary. To make sure that there are no application windows open except the main window, the recovery system calls the built-in CloseWindows method. This method starts with the currently active window and attempts to close it using the sequence of steps below, stopping when the window closes. If any of the steps fails, none of the following steps is executed and the recovery system raises an exception. 1 2 3 4 If a Close method is defined for the window, call it. Click on the Close menu item on the system menu (on platforms and windows that have system menus). Click on the windows close box (if one exists). If the window is a dialog, type each of the keys specified by the OPT_ CLOSE_DIALOG_KEYS option and wait one second for the dialog box to close. By default, this option specifies the Esc key. If there is a single button in the window, click that button. Click each of the buttons specified by the OPT_CLOSE_WINDOW_ BUTTONS option. By default, this option specifies the Cancel, Close, Exit, and Done keys. Select each of the menu items specified by the OPT_CLOSE_WINDOW_ MENUS option. By default, this option specifies the File/Exit and the File/Quit menu items. 8 If the closing of a window causes a confirmation dialog to appear, CloseWindows attempts to close the dialog by clicking each of the buttons specified with the OPT_CLOSE_CONFIRM_BUTTONS option. By default, this option specifies the No button.
5 6
When the window (and any resulting confirmation dialog) closes, CloseWindows repeats the preceding sequence of steps with the next window, until all windows are closed.
Specifying new window closing procedures

When the recovery system cannot close a window using the procedures described in the preceding section, you can reconfigure it in one of two ways:
Users Guide
293
If the window can be closed by a button press, key press, or menu selection, specify the appropriate option either statically in the Close tab of the Agent Options dialog or dynamically at runtime. Otherwise, record a Close method for the window.
Specifying buttons, keys, and menus that close windows

Specify statically
To specify statically the keys, menu items, and buttons that the recovery system should use to close all windows, select Options/Agent and then select the Close tab.
The Close panel contains the following options, each of which takes a comma-delimited list of character string values: List of Buttons Used To Close a Window If a button press will close the dialog, add the button name to the default list. Keystrokes Used to Close a Dialog Box Window If a key sequence will close the dialog, add the key sequence to the default list. List of Menus Used To Close a Window If selecting a menu item will close the dialog, add the menu item to the default list. List of Buttons Used To Close a Confirmation Window If a button press will close the confirmation dialog, add the button name to the default list.
294
Users Guide
Name of Close Item On System Menu If selecting a menu item from the system menu will close the dialog, add the menu item to the default list. For a complete listing of the defaults for these options on all supported platforms, see Close options on page 468.
Specify dynamically
As you set Close options in the Agent Options dialog, the informational text at the bottom of the dialog shows the 4Test command you can use to specify the same option from within a script; add this 4Test command to a script if you need to change the option dynamically as a script is running. If you want to specify for an individual dialog the keys, menu items, and buttons that the recovery system should use to close that dialog, define the appropriate variable in the window declaration for the dialog: lsCloseWindowButtons lsCloseConfirmButtons lsCloseDialogKeys lsCloseWindowMenus
Specify for individual objects
Recording a Close method

Procedure To record a Close method, SilkTest provides the Record Method dialog: 1 2 3 4 Open your application. Open the applications test frame file. Place the insertion point on the window declaration for the dialog: Select Record/Method. SilkTest displays the Record Method dialog, which allows you to record a method for either a window declaration or a class. 5 Press the arrow button to the right of the Method Name field and select Close from the drop-down list.
Users Guide
295
15 UNDERSTANDING THE RECOVERY SYSTEM Specifying windows to be left open
Click the Start Recording pushbutton. SilkTest closes the Record Method dialog and displays the Record Status window, which indicates that you can begin recording the Close method. The Status field flashes the word Recording.
When you have finished recording the Close method, click the Done pushbutton on the Record Status window. SilkTest redisplays the Record Method dialog. The Method Code field contains the 4Test code youve just recorded.
Click OK to close the Record Method dialog and place the new Close method in the declaration for the dialog.
Specifying windows to be left open

By default, the recovery system closes all windows in your test application except the main window. To specify which windows, if any, need to be left opensuch as a child window that is always openuse the lwLeaveOpen constant.
The lwLeaveOpen constant
When you record and paste the declarations for your applications main window, the stub of a declaration for the lwLeaveOpen constant is automatically included, as shown in this example:
// The list of windows the recovery system is to leave open // const lwLeaveOpen = {?}
Procedure To complete the declaration for the lwLeaveOpen constant: 1 Replace the question mark in the comment with the 4Test identifiers of the windows you want to be left open. Separate each identifier with a comma.
296
Users Guide
15 UNDERSTANDING THE RECOVERY SYSTEM Handling login windows
Remove the comment characters (the two forward slash characters) at the beginning of the declaration.
For example, the following 4Test code shows how to set the lwLeaveOpen constant so that the recovery system leaves open the window with the 4Test identifier DocumentWindow when it restores the base state.
const lwLeaveOpen = {DocumentWindow}
Handling login windows

Although an applications main window is usually displayed first, it is also common for a login or security window to be displayed before the main window.
Use the wStartup constant and the Invoke method
To account for login windows, you need to record a declaration for the login window, set the value of the wStartup constant, and write a new Invoke method for the main window that enters the appropriate information into the login window and dismisses it. This enables the DefaultBaseState routine to perform the actions necessary to get past the Login window. Note You do not need to use the procedure outlined in this section for splash screens that disappear on their own. Procedure To account for login windows: 1 Activate the Login window and record a declaration for it using the Record/Declarations command. Paste the resulting declaration to the test frame file. Refer to About dialog declarations on page 91 for more information on using the Record/Declarations command. In your test frame file, find the stub of the declaration for the wStartup constant, located at the top of the declaration for the main window:
// First window to appear when application is invoked // const wStartup = ?
Complete the declaration for the wStartup constant by a b Removing the comment characters (the two forward slash characters) at the beginning of the declaration. Replacing the question mark with the identifier of the login window, as recorded in the window declaration for the login window.
Define an Invoke method in the main window declaration that calls the built-in Invoke method and additionally performs any actions required by the login window, such as entering a name and password.
Users Guide
297
15 UNDERSTANDING THE RECOVERY SYSTEM Adding to the default base state Example
After following this procedure, your test frame might look like this:
window MainWin MyApp tag "My App" const wStartup = Login // the declarations for the MainWin should go here Invoke () derived::Invoke () Login.Name.SetText ("Your name") Login.Password.SetText ("password") Login.OK.Click () window DialogBox Login tag "Login" // the declarations for the Login window go here PushButton OK tag "OK"
About the derived keyword and scope resolution operator Notice the statement derived::Invoke ( ). That statement uses the derived keyword followed by the scope resolution operator ( :: ) to call the built-in Invoke method, before performing the operations needed to fill in and dismiss the login window. For more information about the scope resolution operator, see online Help.
Adding to the default base state

As described in The recovery systems flow of control on page 291, SilkTest by default considers your application to be at its base state when: The application is running The application is not minimized The application is active No other window besides the main window is open
If you want the recovery system to perform additional steps after it restores the default base state, you need to record a new method named BaseState and paste it into the declaration for your applications main window.
How to record a BaseState method for the main window
Procedure To record a BaseState method, SilkTest provides the Record Method dialog. 1 2 Open your application. Open the applications test frame file.
298
Users Guide
15 UNDERSTANDING THE RECOVERY SYSTEM Overriding the default recovery system
3 4
Place the insertion point on the declaration for the applications main window. Select Record/Method. SilkTest displays the Record Method dialog, which allows you to record a method for a class or window declaration.
Press the arrow button (located to the right of the Method Name field) and select BaseState from the drop-down list.
Click the Start Recording pushbutton. SilkTest closes the Record Method dialog and displays the Record Status window, which indicates that you can begin recording the BaseState method. The Status field flashes the word Recording.
When you have finished recording the BaseState method, click the Done pushbutton on the Record Status window. SilkTest redisplays the Record Method dialog. The Method Code field contains the 4Test code youve just recorded.
Click OK to close the Record Method dialog and place the new BaseState method in the declaration for your main window.
Overriding the default recovery system

The default recovery system specifies what SilkTest does to restore your applications base state. It also specifies what SilkTest does whenever: A script file is first accessed A script file is exited A testcase is about to begin A testcase is about to exit
Users Guide
299
15 UNDERSTANDING THE RECOVERY SYSTEM Overriding the default recovery system How the default processing is implemented
The default recovery system is implemented through several functions, as described in the following table.
Function Purpose
DefaultBaseState DefaultScriptEnter DefaultScriptExit
Restore the default base state, then call the applications BaseState function, if defined. Executed when a script file is first accessed. Default action: none. Executed when a script file is exited. Default action: Call the ExceptLog function if the script had errors.
DefaultTestcaseEnter DefaultTestcaseExit
Executed when a testcase is about to start. Default action: Set the application state. Executed when a testcase has ended. Default action: Call the ExceptLog function if the script had errors, then set the base state.
Overriding the default processing
You can write functions that override some of the default behavior of the recovery system.
To override Define the following
DefaultScriptEnter DefaultScriptExit DefaultTestcaseEnter DefaultTestcaseExit
ScriptEnter ScriptExit TestcaseEnter TestcaseExit
If ScriptEnter, ScriptExit, TestcaseEnter, or TestcaseExit are defined, SilkTest will use them instead of the corresponding default function. Example You might want to specify that certain test files get copied from a server in preparation for running a script. You would specify such processing in a function called ScriptEnter in your test frame.
300
Users Guide
15 UNDERSTANDING THE RECOVERY SYSTEM Modifying the default recovery system
Modifying the default recovery system

The default recovery system is implemented in defaults.inc, which is in the directory where you installed SilkTest. If you want to modify the default recovery system, instead of overriding some of its features as described above, you can modify defaults.inc. We dont recommend that you modify defaults.inc, but if you decide you need to modify it, be sure that you: 1 2 Make a backup copy of the shipped defaults.inc. Tell Technical Support when reporting problems that you have modified the default recovery system.
Users Guide
301
15 UNDERSTANDING THE RECOVERY SYSTEM Modifying the default recovery system
302
Users Guide
6rh 1e C t p a
Extending the Class Hierarchy

The 4Test class hierarchy defines the methods and properties that enable you to query, manipulate, and verify the data or state of any GUI object in your application. You can define your own methods and properties, as well as define your own classes. You can also define your own attributes, which are used in the verification stage in testcases.
What you will learn

Topic Page
The class hierarchy Adding methods to a class Defining new class properties Defining new verification properties Defining new attributes Defining new classes
304 306 308 308 311 314
Users Guide
303
16 EXTENDING THE CLASS HIERARCHY The class hierarchy
The class hierarchy

The following illustration shows a partial listing of the class hierarchy:
304
Users Guide
16 EXTENDING THE CLASS HIERARCHY The class hierarchy The inheritance embodied in the class hierarchy
The inheritance depicted in the class hierarchy diagram can be summarized as follows:
This class Defines the methods common to
AnyWin BrowserChild Control
Any type of GUI object. The window that contains the contents of a Web page. Check boxes, combo boxes, list boxes, popup lists, pushbuttons, radio lists, scales, scroll bars, static text (labels), text fields, and so on. Does not apply. Assigned to objects that do not correspond to any class in the built-in class hierarchy. Because the CustomWin class derives from the AnyWin class, you can use the primitive methods of the AnyWin class, like PressKeys, to define higher level actions to perform on your custom GUI objects.
CustomWin
DesktopWin HtmlColumn HtmlTable Menu MoveableWin
The desktop. A column within an HtmlTable in a Web application. A series of two or more columns in a Web application. Apple menus, help menus, menu items, popup menus, and system menus. Child windows (free-standing windows, including MDI sheets) dialog boxes, message boxes, and the applications main window. The taskbar in Windows 95, Windows 98, or Windows NT 4.0.
TaskbarWin
Logical classes
The AnyWin, Control, and MoveableWin classes are logical (virtual) classes that do not correspond to any actual GUI objects, but instead define methods common to the classes that derive from them. This means that SilkTest never records a declaration that has one of these classes. There are three classes that are not part of the AnyWin class hierarchy, because they define methods for objects that are not windows: CursorClass, which defines the three methods you can use on the cursor: GetPosition, GetType, and Wait. ClipboardClass, which defines the two methods you can use on the system clipboard: GetText and SetText.
CursorClass, ClipboardClass, and AgentClass
Users Guide
305
16 EXTENDING THE CLASS HIERARCHY Adding methods to a class
AgentClass, which defines the methods you can use to set options in the 4Test Agent. (The 4Test Agent is the component of SilkTest that translates the method calls in your testcases into the appropriate GUIspecific event streams.)
The predefined identifiers for the Cursor, Clipboard, and Agent You do not record declarations for the cursor, the clipboard, or the Agent. Instead, you use predefined identifiers for each of these objects when you want to use a method to act against the object: The predefined identifier for the 4Test Agent is Agent The predefined identifier for the clipboard is Clipboard The predefined identifier for the mouse pointer is Cursor
For example, to set a 4Test Agent option, you use a call such as the following:
Agent.SetOption (OPT_VERIFY_COORD, TRUE) DesktopWin
Because the desktop is a GUI object, it derives from the AnyWin class. However, unlike other GUI objects, you do not have to record a declaration for the desktop. Instead, you use the predefined identifier Desktop when you want to use a method on the desktop. For example, to call the GetActive method on the desktop, you use a call like the following:
wActive = Desktop.GetActive ()
Adding methods to a class

Defining a new method
To add a method to an existing class, you use the following syntax to begin the method definition: winclass ExistingClass : ExistingClass The syntax ExistingClass : ExistingClass means that the declaration that follows extends the existing class definition, instead of replacing it. For more information, see winclass declaration in online Help.
Example 1
This example adds to the TextField class a method that selects all of the text in the text field.
winclass TextField : TextField SelectAll () STRING sKey1, sKey2
306
Users Guide
16 EXTENDING THE CLASS HIERARCHY Adding methods to a class switch (GetGUIType ()) case msw, motif sKey1 = "<Ctrl-Home>" sKey2 = "<Shift-Ctrl-End>" case mac sKey1 = "<Ctrl-Up>" sKey2 = "<Shift-Cmd-Down>" // return cursor to 1,1 this.TypeKeys (sKey1) // highlight all text this.TypeKeys (sKey2)
Note The keyword this refers to the object the method is being called on. The preceding method first decides which keys to press, based on the GUI. It then presses the key that brings the cursor to the beginning of the field. It next presses the key that highlights (selects) all the text in the field. For more information on the syntax used in declaring new methods, see Method declaration in online Help.
Example 2
To add a Tab method to the DialogBox class, you could add the following 4Test code to your frame.inc file (or other include file):
winclass DialogBox : DialogBox Tab (INTEGER iTimes optional) if (iTimes == NULL) iTimes = 1 this.TypeKeys ("<tab {iTimes}>")
Redefining a method
There may be some instances in which you want to redefine an existing method. For example, to redefine the GetCaption method of the Anywin class, you use this 4Test code:
winclass AnyWin : AnyWin GetCaption () // insert method definition here
Deriving a new method from an existing method
To derive a new method from an existing method, you can use the derived keyword followed by the scope resolution operator (::). The following example defines a GetCaption method for DialogBox that prints the string Dialog before calling the built-in GetCaption method (defined in the AnyWin class) and printing its return value:
winclass DialogBox : DialogBox GetCaption () Print ("Dialog") Print (derived::GetCaption ())
Users Guide
307
16 EXTENDING THE CLASS HIERARCHY Defining new class properties Polymorphism
If a class defines its own version of a method or property, that method or property overrides the one inherited from an ancestor. This is referred to as polymorphism. For example, the ListBox class has its own GetContents method, which overrides the GetContents method inherited from the AnyWin class. To define a new method to use on a single GUI object, not for an entire class of objects, you add the method definition to the window declaration for the individual object, not to the class. The syntax is exactly the same as when you define a method for a class. For examples, see Defining a method for a GUI object on page 96.
Defining a new method for a single GUI object
Defining new class properties

You can define new properties for existing classes using the property statement. You use these class properties to hold data about an object; you can use class properties anywhere in a script. For complete information, see property declaration in online Help.
Defining new verification properties

You can perform verifications in your testcases using properties. These verification properties are different than the class properties described in the previous section, which are defined using the property statement. Verification properties are used only when verifying the state of your application in a testcase. SilkTest comes with built-in verification properties for all classes of GUI objects. Note For more information about verification using properties, see Verifying using properties on page 115. You can define your own verification properties, which will be added to the built-in properties listed in the Verify Window dialog when you record a testcase. Procedure To define custom verification properties: 1 In a class declaration or in the declaration for an individual object, define the variable lsPropertyNames as follows:
LIST OF STRING lsPropertyNames
Specify each of your custom verification properties as elements of the list lsPropertyNames. Custom verification properties can be either:
308
Users Guide
16 EXTENDING THE CLASS HIERARCHY Defining new verification properties
Class properties, defined using the property statement Variables of the class or individual object
Any properties you define in lsPropertyNames will override built-in properties with the same name. With your custom verification properties listed as elements in lsPropertyNames, when you record and run a testcase, those additional properties will be available during verification. Tip You can add your custom properties to property sets. See Configuring your own property sets on page 121.
Confirming the property list
You can use the GetPropertyList method to confirm the list of verification properties for an object. For example, the following simple testcase prints the list of all the verification properties of the Find dialog to the results file:
testcase FindDialogPropertyConfirm () TextEditor.Search.Find.Pick () ListPrint (Find.GetPropertyList ()) Find.Cancel.Click ()
For more information about GetPropertyList, see online Help.

Example
Lets look at an example of defining a custom verification property. Say you want to test a dialog box. Dialog boxes come with built-in verification properties, as shown in the following illustration:
And lets say that you have defined a class property, NumChildren, that you want to make available to the verification system.
Users Guide
309
16 EXTENDING THE CLASS HIERARCHY Defining new verification properties
Here is the class property definition:

property NumChildren INTEGER Get () return ListCount (GetChildren ())
That property returns the number of children in the object, as follows: The built-in method GetChildren returns the children in the dialog box in a list. The built-in function ListCount returns the number of elements in the list returned by GetChildren.
To make the NumChildren class property available to the verification system (that is, to also make it a verification property) you list it as an element in the variable lsPropertyNames. So here is part of the extended DialogBox declaration that you would define in an include file:
winclass DialogBox : DialogBox // user-defined property property NumChildren INTEGER Get () return ListCount (GetChildren ()) // list of custom verification properties LIST OF STRING lsPropertyNames = {"NumChildren"}
Now when you verify a dialog box in a testcase, you can verify your custom property:
310
Users Guide
16 EXTENDING THE CLASS HIERARCHY Defining new attributes
An alternative Instead of defining NumChildren as a class property, you could also define it as a variable, then initialize the variable in a script. For example, in your include file, you would have:
winclass DialogBox : DialogBox INTEGER NumChild2 // list of custom verification properties LIST OF STRING lsPropertyNames = {"NumChild2"}
And in your scriptbefore you do the verificationyou would initialize the value for the dialog box under test, such as:
Find.NumChild2 = ListCount (Find.GetChildren ())
Defining new attributes

When you are recording a testcase, you can verify using attributes. Note You can choose to verify using either attributes or properties; generally you will verify using properties because property verification is more flexible. For more information on verifying using properties and attributes, see Chapter 5, Designing and Recording Testcases. For example, the following figure shows the attributes for the DialogBox class:
Users Guide
311
Here is the 4Test code in the winclass.inc file that implements the Default Button attribute:
attribute "Default button", VerifyDefaultButton, QueryDefaultButton
As this 4Test code examples shows, each attribute definition begins with the attribute statement, followed by three comma-delimited values: 1 2 3
Syntax
The text that you want to appear in the Attribute panel of the Verify Window dialog. This text must be a string. The method SilkTest should use to verify the value of the attribute at runtime. The method SilkTest should use to get the actual value of the attribute at runtime.
To add one or more attributes to an existing class, use the following syntax: winclass ExistingClass : ExistingClass... attribute_definitions Each attribute definition must begin and end on its own line. When you define a new attribute, you usually need to define two new methods (steps 2 and 3 above) if none of the built-in methods suffice. For example, to add a new attribute to the DialogBox class that verifies the number of children in the dialog, you add code like this to your test frame (or other include file):
winclass DialogBox:DialogBox attribute "Number of children", VerifyNumChild, GetNumChild integer GetNumChild() return ListCount (GetChildren ()) // return count of children of dialog hidecalls VerifyNumChild (integer iExpectedNum) Verify (GetNumChild (), iExpectedNum, "Child number test")
As this example shows, you use the hidecalls keyword when defining the verification method for the new attribute. About the hidecalls keyword The keyword hidecalls hides the method from the call stack listed in the results. Using hidecalls allows you to update the expected value of the verification method from the results. If you dont use hidecalls in a verification method, the results file will point to the frame file, where the method is defined, instead of to the script. We recommend that you use hidecalls in all verification methods so you can update the expected values.
312
Users Guide
The Number of Children attribute is listed in the Verify Window dialog when you do an attribute verification:
Users Guide
313
16 EXTENDING THE CLASS HIERARCHY Defining new classes
Defining new classes

Consider the declarations for the Open and the Save As dialogs of the Text Editor application, which each contain exactly the same child windows:
window DialogBox Open tag "Open" parent TextEditor StaticText FileNameText tag "File Name:" TextField FileName1 tag "File Name:" ListBox FileName2 tag "File Name:" StaticText DirectoriesText tag "Directories:" StaticText PathText tag "#3" ListBox Path tag "#2" StaticText ListFilesOfTypeText tag "List Files of Type:" PopupList ListFilesOfType tag "List Files of Type:" StaticText DrivesText tag "Drives:" PopupList Drives tag "Drives:" PushButton OK tag "OK" PushButton Cancel tag "Cancel" PushButton Network tag "Network" window DialogBox SaveAs tag "Save As" parent TextEditor StaticText FileNameText tag "File Name:" TextField FileName1 tag "File Name:" ListBox FileName2 tag "File Name:" StaticText DirectoriesText tag "Directories:" StaticText PathText tag "#3" ListBox Path tag "#2" StaticText ListFilesOfTypeText tag "List Files of Type:" PopupList ListFilesOfType tag "List Files of Type:" StaticText DrivesText tag "Drives:" PopupList Drives tag "Drives:" PushButton OK tag "OK" PushButton Cancel tag "Cancel" PushButton Network tag "Network"
It is not uncommon for an application to have multiple dialogs whose only difference is the caption: The child windows are all identical or nearly identical. Rather than recording declarations that repeat the same child objects, it is cleaner to create a new class that groups the common child objects. For example, here is the class declaration for a new class called FileDialog, which is derived from the DialogBox class and declares each of the children that will be inherited by the SaveAs and Open dialogs:
winclass FileDialog : DialogBox parent TextEditor
314
Users Guide
16 EXTENDING THE CLASS HIERARCHY Defining new classes StaticText FileNameText tag "File Name:" TextField FileName1 tag "File Name:" ListBox FileName2 tag "File Name:" StaticText DirectoriesText tag "Directories:" StaticText PathText tag "#3" ListBox Path tag "#2" StaticText ListFilesOfTypeText tag "List Files of Type:" PopupList ListFilesOfType tag "List Files of Type:" StaticText DrivesText tag "Drives:" PopupList Drives tag "Drives:" PushButton OK tag "OK" PushButton Cancel tag "Cancel" PushButton Network tag "Network"
To make use of this new class, you 1 2 Rewrite the declarations for the Open and Save As dialogs, changing the class to FileDialog. Remove the declarations for the child objects inherited from the new class.
Here are the rewritten declarations for the Open and Save As dialogs:
window tag window tag FileDialog SaveAs "Save As" FileDialog Open "Open"
For more information on the syntax used in declaring new classes, see winclass declaration in online Help.
Users Guide
315
16 EXTENDING THE CLASS HIERARCHY Defining new classes
316
Users Guide
7rh 1e C t p a
Supporting Custom Objects

While you record window declarations, SilkTest attempts to identify the class of each object in your GUI and assign the appropriate class from the built-in class hierarchy. If an object does not correspond to one of the built-in classes, SilkTest designates the object as a custom window and assigns it to the class CustomWin. This chapter shows you how to support certain kinds of custom objects by modifying your window declarations and by creating new classes and methods.
Introduction
What you will learn

Topic Page
Why SilkTest sees objects as custom windows Mapping custom classes to standard classes Supporting graphical controls Supporting nongraphical controls
317 318 320 323
Why SilkTest sees objects as custom windows

There are two things that define an object: The objects actual class name The underlying software code that creates and manipulates the object
An object (window or control) is defined as a CustomWin whenever either or both of these definitions vary from what is considered to be standard.
Users Guide
317
17 SUPPORTING CUSTOM OBJECTS Mapping custom classes to standard classes
For example, on Windows, SilkTest supports the standard MFC library, which is a library of functions that allow for the creation of objects and the mechanism of interaction with them. In supporting these libraries, SilkTest contains algorithms to interrogate the objects based upon the standard libraries. When these algorithms do not work, SilkTest reports the object as a CustomWin. Lets say that you see a text box in a window in your application under test. It looks like a normal text field, but SilkTest calls it an object of the class CustomWin.
The reasons
There are two reasons why SilkTest sees the object as a CustomWin. Upon its definition in the application under test, the control was simply named differently than the standard name. For example, instead of it being named a TextField, it was named EnterTextRegion. If this is the only reason, then you can class map the control to the standard name, as described in the next section, Mapping custom classes to standard classes. You never know whether class mapping will work until you try it. It will work if the object isnt really a custom object, but rather a standard control with a non-standard name. Try this as your first attempt at dealing with a CustomWin. If the class mapping does not work, it is because of the following reason. The object truly is a custom object, that is, the software in the application under test that creates and manipulates the object is not from the standard library. That means that the SilkTest algorithms written to interrogate this kind of object will not work, and other approaches will have to be used to manipulate the object. If the object is a graphical control, such as a tool bar, see Supporting graphical controls on page 320. If the object is not a graphical control, such as a text box, see Supporting nongraphical controls on page 323.
Mapping custom classes to standard classes

When an object shows up in the Record Window Declarations dialog as a CustomWin, but is actually a standard GUI object which the application developers have renamed, you should map the custom class name to the built-in class name while in the dialog. For a list of built-in classes, see SilkTest online help.
318
Users Guide
17 SUPPORTING CUSTOM OBJECTS Mapping custom classes to standard classes
Among the standard classes the SilkTest displays in the Class Map dialog, there are three classes that can be described as meta classes: BUTTON causes the Agent to treat the object as a kind of button, whether it be an instance of PushButton, CheckBox, or RadioButton. The kind of button depends on the objects style bits. STATIC causes the Agent to treat the object as Static Text if the appropriate style bits (for example, SS_LEFT and SS_CENTER) are set. Otherwise, only the methods of the AnyWin class apply. MDI client windows are containers that sit between application frame windows and MDI document windows. Mapping a custom object to MIDCLIENT means you do not need a tag for it in order to refer to one of its children. You cannot perform operations on them.
Procedure To perform a class mapping when a declaration for a CustomWin appears in the Record Window Declarations dialog: 1 Press Ctrl+Alt. The declarations are frozen in the Window Declaration list box in the lower half of the Record Window Declarations dialog. 2 In the Window Declarations list box, click on the line containing the declaration for the custom object. The line is highlighted and the declaration for the CustomWin appears in the Window Detail group box. 3 In the Window Detail group box, press the Class Map pushbutton. The Class Map dialog appears. The name of the custom class is displayed in the Custom Class text field.
4 5
In the Standard Class field, enter the name of the built-in 4Test class to which you are mapping the custom class. Click Add.
Users Guide
319
17 SUPPORTING CUSTOM OBJECTS Supporting graphical controls
Click OK.
When you resume recording, the object has the standard 4Test class. Any subsequent similar objects encountered will automatically be mapped to the correct 4Test class. You must modify any prerecorded declarations containing these objects to use the standard class.
Supporting graphical controls

If an application contains a graphical area, for example a tool bar, which is actually composed of a discrete number of graphical objects, SilkTest records a single declaration for the entire graphical area; it does not understand that the area contains individual objects.
Add a location suffix to the declarations tag
You can, however, create declarations for each discrete object. To do this, make as many copies of the original recorded declaration as there are discrete objects. Then add a location suffix to the tag in each declaration, which is the location of the object within the graphical area. SilkTest provides two ways to specify the location suffix of contained graphical objects, depending on the size and spacing of the control: Controls that are sized and spaced evenly in a grid Controls that are irregularly sized and spaced
Evenly sized and spaced controls
If a group of graphical controls are equal in size and evenly spaced in a grid, you can specify the location of each control as column y of the total number of columns and row x of the total number of rows. This syntax is both cross-platform and resolution independent. Example Consider the following declaration for a tool bar that contains a single row of 26 icons:
CustomWin PaintWindow1 mswdos tag "[PaintWindow]#1"
Note the following: Because there is no label or caption for the tool bar, SilkTest uses an index for the tag. SilkTest qualifies the tag with the name of the class, PaintWindow, enclosed within square brackets ([]). SilkTest bases the identifier PaintWindow1 on the internal class, since the tool bar has no label or caption.
320
Users Guide
17 SUPPORTING CUSTOM OBJECTS Supporting graphical controls
Procedure To modify declarations for each of the icons contained in an evenly sized and spaced tool bar: 1 2 In the window declarations file make as many copies of this recorded declaration as there are discrete objects. You can retain the original class (CustomWin) if the functionality inherited from the AnyWin class is sufficient. Or you can specify the name of a class you create that contains the added functionality you need. Change the identifier to some string that represents the icons action. Append the tag with the icons location suffix in the tool bar. You express the location using this syntax:
(column:total-columns, row:total-rows)
3 4
For example, you specify the icon in the third column, first row, like this:
(3:26, 1:1)
You append this location to the tag with the forward slash (/) character. Example To alter the previous declaration for the tool bar so that it becomes the declaration for the Printer icon, which is the third icon, you write this declaration:
CustomWin Printer mswdos tag "[PaintWindow]#1/(3:26, 1:1)"
The declarations for the first seven icons in the tool bar are shown here:
CustomWin mswdos CustomWin mswdos CustomWin mswdos CustomWin mswdos CustomWin mswdos CustomWin mswdos CustomWin mswdos Using the modified declaration FileOpen tag "[PaintWindow]#1/(1:26, Save tag "[PaintWindow]#1/(2:26, Printer tag "[PaintWindow]#1/(3:26, Preview tag "[PaintWindow]#1/(4:26, Cut tag "[PaintWindow]#1/(5:26, Copy tag "[PaintWindow]#1/(6:26, Paste tag "[PaintWindow]#1/(7:26, 1:1)" 1:1)" 1:1)" 1:1)" 1:1)" 1:1)" 1:1)"
Once you create window declarations like these for the graphical objects in your application, you can manipulate them as you would any other object. For example, if the tool bar was contained in an application named MyApp, to click on the FileOpen icon in the tool bar, you use this command:
Users Guide
321
17 SUPPORTING CUSTOM OBJECTS Supporting graphical controls MyApp.FileOpen.Click ()
Note You need to write this statementand others that access the objects declared above, such as Save and Printerby hand. Record/Testcase and Record/Actions will not use these identifiers.
Irregularly sized or spaced controls
If the graphical controls in a group are not the same size or are not evenly spaced in a grid, you need to specify in the declaration the location suffix of each control as an exact x,y point. This x,y point typically corresponds to the center of the object. This syntax is not necessarily cross-platform or resolution independent. You specify a location in its declaration as an x,y coordinate using the following syntax:
(x, y)
You append this location to the tag with the forward slash (/) character. Example The following declaration specifies that the FileOpen icon on the tool bar is located at position 74, 22.
CustomWin FileOpen mswdos tag "[PaintWindow]#1/(74,22)"
Procedure To add the x,y coordinates to a declaration: 1 2 3 4 Position the cursor in the declaration at the end of the tag to which you want to add coordinates. Type a slash character (/). Select Record/Window Locations to open the Record Window Locations dialog. Track the cursor over the object. The dialog displays the name of the object and its x,y coordinates relative to the screen, the frame (the main window and its window decoration), and the client (the main window minus its window decoration). 5 6 7 8
Using the modified declaration
Press Ctrl+Alt to freeze the declaration. Select the Client radio button, if not already selected. Press the Paste to Editor pushbutton. Press the Close pushbutton.
In a script, to use the Click method on this icon, you use this command:
app-name.FileOpen.Click ()
322
Users Guide
17 SUPPORTING CUSTOM OBJECTS Supporting nongraphical controls
Supporting nongraphical controls

Options for nongraphical, custom controls
If your application uses a nongraphical control that does not map to any of those supported by SilkTest, you have these options: If you have the SilkTest Extension Kit, you can add full support for the custom object to SilkTest. Refer to the extension kit guide for more information. If the applications developer created DLLs to interact with the custom object, you can call the DLL functions from a script. For more information, see Chapter 24, Calling Windows DLLs From 4Test Scripts. Otherwise, you can add partial support for the custom object by creating a new class derived from AnyWin and writing methods that use the primitive methods of the AnyWin class, like TypeKeys and MoveMouse, to implement, as much as is possible, the methods appropriate for the non-graphical control.
Supporting custom text fields
Suppose your application has an object that acts like a text field, but which is not implemented using your GUIs standard text field object. The following example illustrates how you can derive a new class from AnyWin and define methods for the custom object. The example defines the ClearText, GetMultiText, SetMultiText, and GetMultiSelText methods.
// // // // // This new class defines methods that re-implement the methods of the TextField class so that they will work on custom text fields. To be able to use these methods, you must change the class of object in the declarations from CustomWin to CustomTextField.
winclass CustomTextField : AnyWin // This method clears the text field by moving the // cursor to the start of field, selecting the text // to the end of the file, and deleting the selected // text void ClearText () TypeKeys ("<Ctrl-Home>") TypeKeys ("<Ctrl-Shift-End>") TypeKeys ("<Backspace>") // This method writes text to the text field. // It first calls ClearText and then uses TypeKeys to // input the text passed in. void SetMultiText (STRING sText)
Users Guide
323
17 SUPPORTING CUSTOM OBJECTS Supporting nongraphical controls ClearText () TypeKeys (sText) // This copies the currently selected text to the // clipboard and returns the clipboard contents. LIST OF STRING GetMultiSelText () Clipboard.SetText () // Clear the clipboard TypeKeys ("<Ctrl-Insert>") return (Clipboard.GetText ()) // This method highlights all of the text in the // text field, copies the highlighted text to the // clipboard, and returns the clipboard contents. LIST OF STRING GetMultiText () Clipboard.SetText () // Clear the clipboard TypeKeys ("<Ctrl-Home>") TypeKeys ("<Ctrl-Shift-End>") TypeKeys ("<Ctrl-Insert>") TypeKeys ("<Left>") return (Clipboard.GetText ()) Using Clipboard methods
If you are having trouble getting or setting information with a custom object that contains text, you might want to try the 4Test Clipboard methods. For example, assume you have a class, CustomTextBuffer, which is similar to a TextField, but using the TextFields GetText and SetText methods do not work with the CustomTextBuffer. You can use the ClipboardClass GetText and SetText methods as follows. Getting text The following sample code retrieves the contents of the CustomTextBuffer by placing it on the Clipboard, then printing the Clipboard contents:
// Go to beginning of text field CustomTextBuffer.TypeKeys ("<Ctrl-Home>") // Highlight it CustomTextBuffer.TypeKeys ("<Ctrl-Shift-End>") // Copy it to the Clipboard CustomTextBuffer.TypeKeys ("<Ctrl-Insert>") // Print the contents of the Clipboard Print (Clipboard.GetText())
Setting text Similarly, the following sample code inserts text into the custom object by pasting it from the Clipboard:
// Go to beginning of text field CustomTextBuffer.TypeKeys ("<Ctrl-Home>") // Highlight it
324
Users Guide
17 SUPPORTING CUSTOM OBJECTS Supporting nongraphical controls CustomTextBuffer.TypeKeys ("<Ctrl-Shift-End>") // Paste the Clipboard contents into the text field CustomTextBuffer.TypeKeys ("<Shift-Insert>")
Tip You can wrap this functionality in GetText and SetText methods you define for your custom class, similar to what was shown in Supporting custom text fields on page 323.
Supporting custom list boxes
To support custom list boxes, you can write low-level methods as demonstrated in the previous section. A second approach is to implement the necessary Microsoft Windows messages and set the necessary Windows style bits so that you can use the standard list box methods on your custom list box. The Microsoft Windows messages which you must implement are:
LB_GETCOUNT LB_GETTEXT LB_GETITEMRECT LB_GETTOPINDEX LB_GETSEL LB_GETTEXTLEN
And the Windows style bits that you must set are:
WS_VSCROLL LBS_EXTENDEDSEL LBS_MULTIPLESEL
Users Guide
325
17 SUPPORTING CUSTOM OBJECTS Supporting nongraphical controls
326
Users Guide
8rh 1e C t p a
Modifying the Library Browser

By default, the Library Browser provides online documentation for: Built-in 4Test methods, properties, and functions User-defined methods and properties
It does not by default provide documentation for your user-defined functions. This chapter describes how to enhance the contents of the Library Browser so you can provide:
What you will learn
Descriptive text for your user-defined methods and properties Documentation for your user-defined functions

Topic Page
The standard Library Browser contents Modifying the Library Browser contents
327 329
The standard Library Browser contents

Built-in methods
The Library Browser shows the following for built-in methods: Name and class of the method One line of descriptive text Syntax List of parameters, including a description
Users Guide
327
18 MODIFYING THE LIBRARY BROWSER The standard Library Browser contents
Here is the display for the built-in method VerifyDefaultButton:
User-defined methods
The Library Browser displays the same information for user-defined methods that it does for built-in methods, except that it does not include descriptive text for the method or its parameters. It displays User defined as the method description and displays the data type for each parameter. Note If the Library Browser isnt displaying your user-defined objects, close the Library Browser, recompile the include files that contain your user-defined objects, then reopen the Library Browser. Here is the default display for the user-defined method VerifyNumChild:
328
Users Guide
18 MODIFYING THE LIBRARY BROWSER Modifying the Library Browser contents Properties
The Library Browser also shows all built-in and user-defined properties. As with user-defined methods, the description for user-defined properties by default is User defined. By default, the Library Browser shows all built-in functions but does not display your user-defined functions.
Functions
Modifying the Library Browser contents

4test.txt
The core contents of the Library Browser are based on a standard SilkTest text file, 4test.txt, which contains information for the built-in methods, properties, and functions. You can edit 4test.txt to include your user-defined information, then have SilkTest compile the file (creating 4test.hlp) to make it available to the Library Browser. (Information about methods in 4test.hlp is also used in the Verify Window dialog for methods.) Note SilkTest does not update 4test.txt with user-defined information; instead it populates the Library Browser from information it receives when include files are compiled in memory. You modify 4test.txt to override the default information displayed for user-defined objects. Procedure To enhance the Library Browsers contents: 1 Make a backup copy of the default 4test.txt, which is in the directory where you installed SilkTest. Store your backup copy in a different directory. 2 3 In an ASCII text editor, edit the 4test.txt file in your SilkTest installation directory. Quit, then restart SilkTest, which will automatically compile the modified 4test.txt.
Steps 2 and 3 are described next. Note There is another approach to updating the Library Browser you can usemaintain information in different source files. See Using multiple source files on page 334.
Modifying the source file

Simply looking through 4test.txt should give you all the help you need about how to structure the information in the file. The following table lists all the keywords and describes how they are used. Following the table are examples.
Users Guide 329
18 MODIFYING THE LIBRARY BROWSER Modifying the Library Browser contents
Keywords are followed by a colon and one or more spaces.

Keyword What follows the keyword
class function
Name of the class. Name of the function. Specify the full syntax. If the function returns a value, specify: return_value = function_name (parameters) Otherwise, specify: function_name (parameters)
group method
Name of the function category. Description of the method. Specify the full syntax. If the method returns a value, specify: return_value = method_name (parameters) Otherwise, specify: method_name (parameters)
notes parameter
Description of the method, property, or function, up to about 60 characters. Name and description of a method or function parameter; each parameter is listed on its own line. Specify the name, followed by a colon, followed by the description of the parameter.
property returns
Name of the property. Type and description of the return value of the method or function. Specify the name, followed by a colon, followed by the description of the return value.
Comment.
Edit your copy of 4test.txt to add the information you want, as shown in the following examples.
330
Users Guide
18 MODIFYING THE LIBRARY BROWSER Modifying the Library Browser contents User-defined methods
To fully document your methods, add the method descriptions to the appropriate class section in 4test.txt, such as:
#****************************************************** class: DialogBox ... #** custom method method: VerifyNumChild (iExpectedNum) parameter: iExpectedNum: The expected number of child objects (INTEGER). notes: Verifies the number of child objects in a dialog box.
Here is the resulting display:
Compare this with the figure under User-defined methods on page 328, which shows the default display for VerifyNumChild.
User-defined properties
To include full documentation for your custom properties in the Library Browser, add the property descriptions to the appropriate class section in 4test.txt, such as:
#*********************************************** class: DialogBox ... #** custom property property: iNumChild notes: The number of child objects in the dialog box.
Users Guide
331
Here is the resulting display:
User-defined functions
If you want your user-defined functions to display in the Library Browser, create a group called User-defined functions and document your functions, such as:
group: User-defined functions
function: FileOpen (sFileName) parameter: sFileName = "myFile": The name of the file to open. notes: Opens a file from the application. function: FileSave (sFileName) parameter: sFileName = "myFile": The name of the file to save. notes: Saves a file from the application.
332
Users Guide
Here is the resulting display for FileOpen:
Compiling and using the modified source file

Once you have updated your copy of 4test.txt, you simply make it available to SilkTest. Procedure To compile and use a modified 4test.txt: 1 2 3 Quit SilkTest. Place your modified 4test.txt file in the SilkTest installation directory. Restart SilkTest. SilkTest sees that your source file is more recent than 4test.hlp and automatically compiles 4test.txt, creating an updated 4test.hlp. If there are errors, SilkTest opens a window listing them and continues to use the previous 4test.hlp file for the Library Browser. 4 If there were errors, fix them in 4test.txt and restart SilkTest. Your new definitions are displayed in the Library Browser (assuming that the files containing the declarations for your custom classes, methods, properties, and functions are loaded in memory).
Users Guide
333
Using multiple source files

Instead of modifying 4test.txt to include all your site-specific information, you can define your information in one or more separate files, then tell SilkTest to include them in 4test.hlp, as follows. Procedure To have SilkTest add user-defined files to the Library Browser: 1 Create a text file that includes information for all your custom classes and functions, using the formats described in Modifying the source file on page 329. Note If you have added methods or properties to built-in classes, you should add that information in the appropriate places in 4test.txt, as described above. Only document your custom classes and functions in your own help file. 2 3 4 Select Options/General to display the General Options dialog. Add your help file to the list in the Help Files For Library Browser field. Separate the files in this list with commas. Click OK. SilkTest recompiles 4test.hlp to include the information in all the files listed in the Help Files For Library Browser field. If there are errors, SilkTest opens a window listing them and continues to use the previous 4test.hlp file for the Library Browser. 5 If you had errors, fix them in your source file, then quit and restart SilkTest. SilkTest recompiles 4test.hlp using your modified source file.
334
Users Guide
Testing Client/Server Applications

VtP r a
In this part

Chapter Page
Chapter 19, Introduction to Client/Server Testing Chapter 20, Configuring SilkTest for Client/Server Testing Chapter 21, Implementing Client/Server Testing Chapter 22, Multi-Application Testing
337 351 361 379
Users Guide
335
336
Users Guide
9rh 1e C t p a
Introduction to Client/Server Testing

SilkTest provides powerful support for testing client/server applications and/ or databases in a networked environment. Testing multiple remote applications raises the level of complexity of QA engineering above that required for stand-alone application testing. This chapter introduces certain principles of concurrent programming that you need to understand when developing client/server testcases or any testcases that must operate in a networked environment.
What you will learn

Topic Page
Client/server testing challenges Client/server testing with SilkTest The kinds of network testing you can perform Client/server testing configurations Features of the database tester Concurrent programming issues
337 338 339 343 346 347
Client/server testing challenges

The challenges of Quality Assurance in todays application development environment are many. Where once a computer program ran on one computer, today it must run on many different computers. Where once an application communicated only with local input/output devices, today it must
Users Guide
337
19 INTRODUCTION TO CLIENT/SERVER TESTING Client/server testing with SilkTest
also communicate across a network with remote devices and with a variety of other applications. The step from testing a stand-alone application running on a single PC or workstation to testing an application that runs on multiple different platforms and communicates across a network with other applications is a big one. Here are just a few of the testing methodology problems raised by such a step: Managing simultaneous automatic regression tests on different configurations and/or platforms. Ensuring the reproducibility of client/server tests that modify a server database. Verifying a client applications server operations independently without relying on the application under test. Testing the concurrency features of a client/server application. Testing the intercommunication capabilities of networked applications. Closing down multiple failed applications and bringing them back to a particular base state (recovery control). Testing the functioning of the server application when driven at peak request rates and at maximum data rates (peak load and volume testing). Automated regression testing of multi-tier client/server architectures.
Client/server testing with SilkTest

With SilkTest: You can test an application that runs on a different machine than SilkTest. You can test applications running on two or more machines at once. You can test multiple different applications simultaneously.
The database tester provides an additional feature: You can access a server database directly, thereby circumventing the untested database operations of the application under test. This section introduces the factors that make these features possible.
SilkTest architecture
Normal use of an application consists of a person manipulating a keyboard and mouse to initiate application operations. The person is said to be interacting with the GUI (Graphical User Interface). During SilkTest testing, SilkTest interacts with the GUI to submit operations to the application automatically. Thus SilkTest can simulate the actions of a person who is exercising all the capabilities of an application and verifying the results of
338
Users Guide
19 INTRODUCTION TO CLIENT/SERVER TESTING The kinds of network testing you can perform
each operation. The simulated user (SilkTest) is said to be driving the application. The application under test reacts to the simulated user exactly as it would react to a human user. SilkTest consists of two distinct software components that execute in separate processes: the SilkTest host software and the 4Test Agent software: The SilkTest host software is the program you use to develop, edit, compile, run, and debug your 4Test scripts and testplans. This manual refers to the system that runs this program as the host machine or the SilkTest machine. The 4Test Agent is the software process that translates the commands in your 4Test scripts into GUI-specific commands. In other words, it is the Agent that actually drives and monitors the application you are testing. One Agent can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. This manual refers to the systems that run remote Agents as target machines. In a client/server environment, SilkTest drives the client application by means of an Agent process running on each applications machine. The application then drives the server just as it always does. SilkTest is also capable of driving the GUI belonging to a server or of directly driving a server database by running scripts that submit SQL statements to the database. These methods of directly manipulating the server application are intended to support testing in which the client application drives the server. See the section Features of the database tester on page 346 for more information on this capability.
The kinds of network testing you can perform

Software testing can be categorized according to the various broad testing goals that are the focus of the individual tests. This section describes at a conceptual level the kinds of automated application testing you can perform using SilkTest in a networked environment: Functional Configuration Concurrency Stress Load Performance
Users Guide
339
The ordering of this list conforms to the incremental functional testing methodology supported by SilkTest. Each stage of testing depends for its effectiveness on the successful completion of the previous stage. Functional, configuration, and concurrency testing are variations of regression testing, which is a prerequisite for any type of load testing. These three plus stress testing verify that the application functions properly apart from any stresses exerted by a heavy multi-user load. Thus the testing stages progress in terms of complexity of testing and numbers of testbed machines. You can perform functional testing with a single client machine. You can perform the first four types of test with a testbed containing only two clients. The last two testing types require a heavy multi-user load and so need a larger testbed.
Functional testing
Before you test the multi-user aspects of a client/server application, you should verify the functional operation of a single instance of the application. This is the same kind of testing that you would do for a non-distributed application. Once you have written scripts to test all the operations of the application as it runs on one platform, you can modify the scripts as needed for all other platforms on which the application runs. Testing multiple platforms thus becomes almost trivial. Moreover, many of the tests you script for functional testing can become the basis of your other types of testing. For example, you can easily modify the functional tests (or a subset of them) to use in load testing.
Configuration testing
A client/server application typically runs on multiple different platforms and utilizes a server that runs on one or more different platforms. A complete testing program needs to verify that every possible client platform can operate with every possible server platform. This implies the following combinations of tests: Test the client application and the server application when they are running on the same machineif that is a valid operational mode for the application. This testing must be repeated for each platform that can execute in that mode. Test with the client and server on separate machines. This testing should be repeated for all different platform combinations of server and client.
Concurrency testing
Test two clients using the same server. This is a variation of functional testing that verifies that the server can properly handle simultaneous requests from two clients. The simplest form of concurrency testing just verifies that two clients can make multiple non-conflicting server requests during the same period of time. This is a very basic sanity test for a client/server application.
340
Users Guide
To test for problems with concurrent access to the same database record, you need to write specific scripts that synchronize two clients to make requests of the same records in your servers database(s) at the same time. Your goal is to encounter faulty read/write locks, software deadlocks, or other concurrency problems. See Code for concurrency test example on page 387 for an example of a test that demonstrates the necessary 4Test coding techniques. Once the application passes the functional tests, you can test the boundary conditions that might be reached by large numbers of transactions. The next sections describe this kind of testing.
Stress testing
Stress testing verifies that running a particular command or set of commands a large number of times does not break the software. For example, stress testing discovers a failure to release resources. This is no different than stress testing for a non-distributed application, but as with functional testing, you need to discover whether the application functions properly with a remote server under stress conditions. You can do this type of stress testing with a single client. You can create stress tests by repeating the same commands a large number of times. Usually, the 4Test code looks like this:
for i = 1 to 50 DoSomeCommands ()
For example, you can stress-test remote logins or you can stress-test all application operations that allocate significant amounts of memory (to see if they release that memory). One simple form of stress testing is to set up your functional tests to repeat indefinitely and then run them night and day over a two- to four-day period.
Load testing
Load testing verifies that running a large number of concurrent clients does not break the server or client software. For example, load testing discovers deadlocks and problems with queues. SilkTests unique architecture provides built-in load testing. You can implement your functional tests using multiapplication techniques and then, after completing functional testing on a single machine, you can run them on multiple targets. Unlike simulated load testing tools, SilkTest allows you to create a real-life scenario that can determine the actual impact of multi-machine operation on your application, the server, the network, and all related elements. There are two variations of load testing: peak load (which is sometimes called stress testing in the client/server environment) and volume. Peak load testing is placing a load on the server for a short time to emulate the heaviest demand that would be generated at peak user timesfor example, credit card verification between noon and 1 PM on Christmas Eve.
Users Guide
341
This type of test requires a significant number of client systems. If you submit complex transactions to the server from each client in your test network, using minimal user setup, you can emulate the typical load of a much larger number of clients. Your testbed may not have sufficient machines to place a heavy load on your server systemeven if your clients are submitting requests at top speed. In this case it may be worthwhile to reconfigure your equipment so that your server is less powerful. An inadequate server configuration should enable you to test the servers management of peak server conditions. Volume testing is placing a heavy load on the server, with a high volume of data transfers, for 24 to 48 hours. One way to implement this is to use one set of clients to generate large amounts of new data and another set to verify the data (and delete data to keep the size of the database at an appropriate level). You would need to synchronize the verification scripts to wait for the generation scripts. The 4Test script language makes this easy. You would not normally need a very large testbed to drive this type of server load, but again, if you under-configure your server you will probably be able to exercise the sections of the software that handle the outer limits of data capacity.
Performance testing
Segue believes that the most accurate determination of a systems performance is achieved by actually testing the system in a real-world environment, rather than through simulation. However, if it is impractical to test the full scope of the system (500 users, for instance), you will typically have a better understanding of the load on your system at this usage level by projecting the actual system performance based on the real-world testing of a subset of systems. Typically, the performance trend is reached long before all users have been added, and the mathematical projection is a simple one: Increase the rate of server requests submitted until the trend is isolated and then project the remainder. From the response times at different request rates, you can calculate the expected performance for different numbers of users submitting requests at a normal rate. The result of this method is frequently more accurate than simulation can provide, because a simulation cannot produce the same complex interactions as these scenarios. Although you probably cannot establish a 500-user testbed that runs a realistic workload (which might average two server requests per client per hour), you can design an artificial workload that submits sequential requests as fast as the response to the previous request is received. If you cannot generate a sufficient load using SilkTests GUI-driving scripts, use the Extension Kit (EK) to drive the applications API directly. Using either method, you can drive the server with different rates of received requests and
342
Users Guide
19 INTRODUCTION TO CLIENT/SERVER TESTING Client/server testing configurations
chart the response times for different numbers of representative users. You can then repeat the test with a more (or less) powerful server. See the extension kit guide for more information. While SilkTest does not provide support for sophisticated performance analysis, this is an easy method for characterizing user-perceived performance. It provides a method for gathering data for configuration guidelines that requires little more effort than the repackaging of tests that were already written for functional testing.
Client/server testing configurations

The processes that participate in a client/server testing scenario are logically associated with three different computers: 1 2 3 System A runs SilkTest, which processes test scripts and sends application commands to the Agent. System B runs the client application and the SilkTest Agent, which submits the application commands to the client application. System C runs the server software, which reacts to requests submitted by the client application.
During your initial test development phase, you can reduce your hardware needs by making two (and possibly all) of these systems the same. If you write tests for an application running on the same system as SilkTest, you can implement the tests without consideration of any of the issues of remote testing. You can then expand your testing program incrementally to take your testing into each new phase. The figures that follow show different hardware/software configurations that can support SilkTest testing. In the first figure, Machine 1 shows the software configuration you would have when testing a stand-alone application. Machine 2 shows SilkTest and a client/server application with all of your software running in one machine. This configuration allows you to do all types of functional testing other than testing the behavior of the connection between a client and a remote server.
Users Guide
343
Machine 1 SilkTest
Machine 2 SilkTest Agent
Agent
Application Testing a stand-alone application with SilkTest
Client Server Server Testing a client/server application with software all in one machine
The next figure shows a testing configuration in which the client application runs on the same machine as SilkTest and the server application runs on a separate machine. Note that in this configuration and with Machine 2 in the previous figure, there is no communication between SilkTest and the server. This means that you must manage the work of starting and initializing the server database manually. For some kinds of testing this is appropriate. This configuration lets you test the remote client-to-server connection and is appropriate for many stress tests. It allows you to do volume load testing from the point of view of the client application, but not the server. Machine 1 SilkTest Server Agent Client Machine 2
The next figure shows multiple copies of the client application running on separate machines, with SilkTest driving the client application by means of the Agent process on each client machine, and the client application driving
344 Users Guide
the server application. This is just the multi-client version of the previous configuration. You could run a fourth instance of the client application on the SilkTest machine. The actual number of client machines used is your choice. This configuration is appropriate for load testing and configuration testing if you have no need to automatically manipulate the server. You must have at least two clients to test concurrency and mutual-exclusion functionality. Target Machines Agent
Client Host Machine Agent SilkTest Client Server Server Machine
Agent
Client
Once you are running SilkTest, it makes sense to have your script initialize your server automatically. The last figure uses the same hardware configuration as the previous figure, but SilkTest is also driving the server directly. This figure shows SilkTest using an Agent on the server machine to drive the servers GUI (the lower connecting arrow); this approach can be used to start the servers database and sometimes can be used to initialize it to a base state. The upper arrow shows SilkTest using SQL commands to directly manipulate the server database; use this approach when using the Agent is not sufficient.
Users Guide
345
19 INTRODUCTION TO CLIENT/SERVER TESTING Features of the database tester
After starting the database with the Agent, use SQL commands to initialize it to a base state. The SQL commands are submitted by means of SilkTests database functions, which do not require the services of the Agent. This is the most complete testing configuration. It requires the database tester. You can use it for all types of SilkTest testing, including volume load testing of the server, peak load testing, and performance testing. database tester Host Machine Client Targets Agent SilkTest Client Server Agent Server Target
The special features that allow SilkTest to provide rigorous testing for client/ server applications are the following: Automatic control of multiple applications Multithreading for automatic control of concurrent applications Reporting results by thread ID Testing across networks using a variety of protocols
The added value that the database tester provides for the client/server environment is: Direct database access from the test script. These different features are discussed in the sections that follow.
Features of the database tester

You may be testing a distributed application that accesses a database or you may be directly testing database software. In either of these cases, you might want to manipulate the database directly from SilkTest for several purposes:
346
Users Guide
19 INTRODUCTION TO CLIENT/SERVER TESTING Concurrent programming issues
To exercise certain database functions that are present in a GUI that runs directly on the server machine and is not a client application. For example, administrative functions used for setting up the database. To set the server database to a known state. To verify an applications database results without using the application. To read information from the database to use as input to a testcase.
SilkTest can drive a server applications GUI by means of the SilkTest Agent exactly as it drives a client application. In addition, DBTester provides direct access, using SQL, from a test script to any database supported by ODBC drivers. These database functions enable you to read and write database records without using the client application. Thus you can verify client test results without assuming the ability of the client to do that verification. In addition to using the SQL functions in your tests, you can also use these functions to help manage your testing process as follows: Maintain a bug database, updating it with the results of your testing. Manage your test data in a database instead of in a text file.
The database functions, among other things, allow you to connect to a database, submit an SQL statement, read data from the selected record(s) if the SQL statement was SELECT, and subsequently disconnect from the database. About a dozen of these functions allow you to access your databases catalog tables. The DBTester functions that support these operations are documented in online Help. They begin with the letters DB_.
Concurrent programming issues

The features provided by SilkTest enable you to manage the complexity raised when you test applications on multiple machines and when you run those tests in parallel. This section provides a conceptual understanding of the issues and how to manage them. Chapter 21, Implementing Client/ Server Testing and Chapter 22, Multi-Application Testing explain specific techniques and give coding examples. Note When you create tests for a stand-alone (local) application, you can do many of them using SilkTests recording capabilities. When you generate testcases for a multi-application or client/server environment, you will have to do some 4Test coding to specify the
Users Guide
347
multi-machine test configuration. Be sure you are familiar with the principles in the chapter on structuring and generalizing recordings in this manual.
Driving multiple machines
When you want to run tests on multiple machines simultaneously, you connect to all the machines and then you direct specific test operations to particular machines. This enables you to drive different applications concurrently. For example, you can test the intercommunication capabilities of two different applications or you can drive both a client application and its server. To do this, at the beginning of a test script you issue for each machine an explicit connection command. This can be either Connect (agent_name) or SetUpMachine (agent_name). This connection lasts for the duration of the script unless you issue a Disconnect (agent_name) command. In the body of the script you can specify that a particular portion of code is to be executed on a particular machine. The SetMachine (agent_name) command specifies that the following statements are directed to that Agent. You can specify that just one statement is directed to a particular Agent by using the bracket form of the machine handle operator. For example:
["Client_A"]SYS_SetDir ("c:\mydir")
For a detailed description of the machine handle operator, refer to SilkTest online help. Since 4Test allows you to pass variables to these functions, you can write a block of code that sends the same operations to a particular set of target machines and you can pass the SetMachine function in that block of code a variable initialized from a list that specifies the machines in that set. Thus specifying which machines receive which operations is very simple. Chapter 22, Multi-Application Testing gives examples and detailed explanations of these techniques.
Recovering multiple tests
There are three major categories of operations that an Agent executes on a target machine: Setup operations that bring the application to the state from which the next test will start. Testing operations that exercise a portion of the application and verify that it executed correctly. Cleanup operations that handle the normal completion of a test plus the case where the test failed and the application is left in an indeterminate state. In either case, the cleanup operations return the application to a known base state.
348
Users Guide
When there are multiple machines being tested and more than one application, the Agent on each machine must execute the correct operations to establish the appropriate state, regardless of the current state of the application. Chapter 22, Multi-Application Testing explains how to use the 4Test support functions with which a testcase accomplishes this and gives code examples.
Concurrency
For SilkTest, concurrent processing means that Agents on a specified set of machines drive the associated applications simultaneously. To accomplish this, the host machine interleaves execution of the sets of code assigned to each machine. This means that when you are executing identical tests on several machines, each machine can be in the process of executing the same operation (for example, select the Edit.FindChange menu item). At the end of a set of concurrent operations, you will frequently want to synchronize the machines so that you know that all are ready and waiting before you submit the next operation. You can do this easily with 4Test. There are several reasons for executing testcases concurrently: You want to save testing time by running your functional tests for all the different platforms at the same time and by logging the results centrally (on the host machine). You are testing cross-network operations. You need to place a multi-user load on the server. You are testing the applications handling of concurrent access to the same database record on the server.
To accomplish the last example, testing concurrent database accesses, you simply set all the machines to be ready to make the access and then you synchronize. When all the machines are ready, you execute the operation that commits the access operationfor example, clicking the OK button. Consider the following example:
// [A] Execute 6 operations on all machines concurrently for each sMachine in lsMachine spawn SixOpsFunction (sMachine) rendezvous // Synchronize // [B] Do one operation on each machine for each sMachine in lsMachine spawn [sMachine]MessageBox.OK.Click () // One operation rendezvous // Synchronize
Users Guide
349
In code fragment [A], the six operations defined by the function SixOpsFunction are executed simultaneously on all machines in a previously defined list of Agent names. After the parallel operation, the script waits for all the machines to complete; on completion, they will present a message box (unless the application fails). In code fragment [B], the message box is dismissed. By putting the message dismissal operation into its own parallel statement block instead of adding it to the SixOpsFunction, you are able to synchronize and all machines click at almost the same instant. In order to specify that a set of machines should execute concurrently, you use a 4Test command that starts concurrent threads. In the fragments above, the spawn statement starts a thread for each machine. Threads and the 4Test constructs that generate them are discussed in detail in section Using 4Tests parallel processing features on page 363.
Global variables
Suppose the code for each machine is counting instances of some event. You want a single count for the whole test and so each machine adds its count to a global variable. When you are executing the code for all your machines in parallel, two instances of the statement iGlobal = iGlobal + iCount could be executing in parallel. Since the instructions that implement this statement would then be interleaved, you could get erroneous results. To prevent this problem you can declare a variable shareable. When you do so, you can use the access statement to gain exclusive access to the shared variable for the duration of the block of code following the access statement. Make variables shareable whenever the potential for conflict exists.
350
Users Guide
0rh 2e C t p a
Configuring SilkTest for Client/Server Testing

SilkTest provides powerful support for testing machine configurations and client/server applications in a networked environment.
What you will learn

Topic Page
Understanding host and target machines Networking protocols used by SilkTest The configuration process Selecting the SilkTest protocol Configuration tasks
351 352 353 354 354
Understanding host and target machines

SilkTest consists of two distinct software components that execute in separate processes: the SilkTest host software and the 4Test Agent software: The SilkTest host software is the program you use to develop, edit, compile, run, and debug your 4Test scripts and testplans. This manual refers to the system that runs this program as the host machine or the SilkTest machine. The Agent is the software that translates the commands in your 4Test scripts into GUI-specific commands. In other words, it is the Agent that actually drives and monitors the application you are testing. One Agent
Users Guide
351
20 CONFIGURING SILKTEST FOR CLIENT/SERVER TESTING Networking protocols used by SilkTest
can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. This manual refers to the systems that run remote Agents as target machines. Note Target machines running an Agent must be on the same subnet as the host machine. In a client/server environment, SilkTest typically drives the client application by means of an Agent process running on each applications machine. The application then drives the server. Alternatively, SilkTest is capable of driving the GUI for a server (if it has one) or of driving a server database directly by running scripts that submit SQL statements to the database. See Features of the database tester on page 346 for more information on this capability.
Networking protocols used by SilkTest

Three protocols
SilkTest runs on many platforms, but only three different protocols are used. This means that a SilkTest script on one platform can drive the Agent on a target platform, as long as both the host and Agent platforms are running an appropriate protocol for the platform and both are running the same protocol (regardless of the protocols used by the applications under test). The table on page 353 lists the protocols available for each platform. For example, suppose you are running SilkTest under Windows 95 and testing an application that requires TCP/IP communications in order to communicate with a server on a Sun Sparc station. SilkTests Windows machine can run NetBIOS for the host and the applications Windows machine must then run NetBIOS for the Agent as well as TCP/IP for the application under test. Running NetBIOS will have no impact on your TCP/ IP connections but will allow SilkTest to communicate with the Agent. Alternatively, since the application is already running TCP/IP, you can choose to use TCP/IP for SilkTest and its Agents as well. In another example, if the network under test combines Windows platforms and Motif platforms, SilkTests machine and all the Agent machines will run TCP/IP because that is the only protocol that SilkTest supports for Motifbased platforms.
352
Users Guide
20 CONFIGURING SILKTEST FOR CLIENT/SERVER TESTING The configuration process
The following table lists the protocols that SilkTest can use for each of the platform types.
Platform TCP/IP NetBIOS/ NetBEUI IPX/SPX AppleTalk
Windows 95 Windows NT OS/2 Solaris SunOS HP-UX AIX IRIX
There is no limit on the protocol or API an application under test may use. Just make sure that the protocol required by SilkTest and the protocol required by your application are running at the same time.
The configuration process

To configure a network of computers so that they can run SilkTest and its Agents, you must do the following: Install (or have already running) networking protocols supported by SilkTest. Install SilkTest on the host machine and the Agent software on all target machines. Establish connectability between host and Agents. This may be automatic or may require some setup, depending on circumstances. Enable networking on any PC-based target machines; use the Agent window as described in One or more remote applications on page 355. Enable networking on the host machine. Use the Runtime Options dialog; details vary, depending on your configuration. Gather the information that your test scripts will need when making explicit connections. For example, you can edit the Agent names into a list definition and have your testplan pass the list variable name as an
Users Guide
353
20 CONFIGURING SILKTEST FOR CLIENT/SERVER TESTING Selecting the SilkTest protocol
argument for testcases controlled by that plan. The testcases then pass each Agent name to a Connect or SetUpMachine function and that function makes the explicit host-to-Agent connection. The rest of this chapter provides configuration details specific to the different protocols and operating systems you might be using. This detailed information makes it possible for you to implement the configuration process outlined above. Look for headings that mention topics applicable to your configuration.
Selecting the SilkTest protocol

On a PC-Class platform, SilkTest and its Agents can use one of three possible protocols, depending on circumstances. On a Motif-based platforms, TCP/IP is the only available protocol.
Use TCP/IP...
If the host or any Agent is Motif-based, you must use TCP/IP. Or you can use TCP/IP just because you prefer to.
Use NetBIOS/NetBEUI...
If this is the protocol you normally use and you will not be including any Motif-based workstations in your testing configuration, you can use NetBIOS.
Configuration tasks
This section presents some configuration requirements that must be met to use SilkTest to test applications over a network. In general you should set up your Agents and make all adjustments to the partner.ini file or environment variables before enabling networking on the host machine. As you look through the rest of this chapter, please be sure to read all sections that apply to your configuration. You can ignore topics that do not apply to your configuration.
354
Users Guide
20 CONFIGURING SILKTEST FOR CLIENT/SERVER TESTING Configuration tasks
If you use any PC-class platforms...

You have to explicitly assign a unique network name to remote Agents so that SilkTest can identify the Agent when your testcase connects to that machine. This step is not necessary for Motif-based Agents because SilkTest automatically uses the workstations TCP/IP name.
Single, local application
In a single-application test environment, if the application is local, you do not have to determine an Agent name or issue a connection command because if you start an Agent on the local machine, SilkTest automatically connects to it and directs all Agent commands to it. In a single-application test environment, if the application is remote, specify the Agent name in the Runtime Options dialog (select the Runtime choice from SilkTests Options menu). This causes SilkTest to automatically connect to that machine and to direct all Agent commands to that machine. This contrasts with the multi-application case, in which you explicitly connect to the target machines and explicitly specify which machines are to receive which sections of code. When you have one or more remote PC-based Agents in your testing network, you have to enable networking by specifying the network type. If you are not using TCP/IP, you have to assign to each Agent the unique name that your scripts will use to direct test operations to the associated application. Use the following procedure to enable networking and assign the Agent its network name: 1 2 3 4 5 Start the Agent software. Left-click the Agent icon to display the icon menu. Select the Network item from that menu. In the dialog that appears, select the network type from the Network pulldown list. For NetBIOS, enter the Agent name into the Agent name field. For TCP/ IP, the default port number appears in the Port number field. You normally want to accept the default, but if you have port number conflicts, read the section And you have port number conflicts... on page 356. Click OK.
Single, remote application
One or more remote applications
You only need to execute this procedure the first time you run that Agent.
Multiple remote applications
When you enable networking by selecting the networking type in the Runtime Options dialog on the host, do not set the Agent Name field to an Agent name if you have multiple remote Agents. This field only accepts a
Users Guide 355
single Agent name and using it prevents you from handling all your client machines the same way. If you specify one Agent name from your set of Agents, then you cannot issue a Connect call to that Agent and thus would not receive the machine handle that the Connect function returns. Since you have to issue some Connect calls, be consistent and avoid writing exception code to handle a machine that is automatically connected.
If you use TCP/IP...

TCP/IP is an intrinsic part of the UNIX operating system and is the only protocol that SilkTest supports on UNIX-family platforms. Therefore, in a mixed PC and UNIX network, SilkTest and all the Agents must use TCP/IP. For Agents running under TCP/IP, the Agent name is the TCP/IP name of the Agents machine. You do not have to set the Agent name on the target machine as you do for NetBIOS.
And you have port number conflicts...
SilkTest connects to each Agent through a TCP/IP port that has a 4-digit ID. Normally all the machines in your testing network automatically use the same default port number. This allows the 4Test Connect function to automatically specify the port number for all connections. If some other application on one of your machines has already used the default port number, you have a port number conflict. If you start an Agent on a Motif workstation and there is a port conflict, UNIX assigns a new port and prints its number to standard output. If you start an Agent on a PC and the default port is already in use, you get an error message. In either case, you can use a different port number just for this machine (using the default number for the rest) or you can have all your machines use the same available port number. When you have an Agent that uses other than the default port number, you have to specify the port number in every reference to that Agent. The syntax is:
"AgentName:nnnn"
where AgentName is the target machine name and nnnn is the port number. Since you will normally use a file or a list variable to hold your Agent names, you can just add the ":nnnn" where needed. In most cases, the default TCP/IP port specified by UNIX will be available for SilkTest. If there are no port conflicts, there is nothing you have to do to specify ports. If you have a conflict, the port number used for that machine must change and you can choose to change the port numbers used by all your PCs and workstations so that all use the same number. See the instructions below for PCs and for workstations.
356
Users Guide
20 CONFIGURING SILKTEST FOR CLIENT/SERVER TESTING Configuration tasks Enable networking on TCP/IP host
Once the protocol has been picked for any PC Agents and the port settings are consistent, you can enable networking on the host. Do this by selecting Options/Runtime and setting the port number and/or Agent name. You can skip this step if you do not have to change the default port number and you are not specifying an Agent name for a single-remote-application configuration.
If you use TCP/IP on PCs...

Windows 95, Windows NT, and OS/2 come with TCP/IP. SilkTest on Microsoft Windows can use any TCP/IP software package that supports the Windows Sockets Interface Standard, Version 1.1, (WINSOCK) and supplies WINSOCK.DLL.
Discover the Agent name
For each Agent, the Agent name that you specify in the Connect function is that Agents host name, stored in the network host database. You can find the host name in the name given to the Agents icon. It takes the form:
Agent [TCP/IP <Host Name> <Port Number>]
For Windows 95/NT, move the mouse pointer over the Agent icon and wait two seconds; the icon name will appear.
Enable networking on Agent
You must enable networking for each Agent PC by selecting the protocol type to be used in the Agent window (this operation is described in One or more remote applications on page 355). When you select TCP/IP as the protocol, you will see the port number field displayed with the default TCP/ IP port number. When you click OK, the selection is accepted if the default port is available. If when you click OK in the Agent window on a PC, the system responds with an error message stating that the number is in use, then you have a port number conflict. You can find an available port number for that machine and leave the rest as the default. Or you can find a port number that all your machines can use. Invoke the Agent window again and specify an alternative port number. The value of the port number must be in the range 1000 to 9999. If this number is accepted, you can try setting all of your PCs to this port again using the Agent window. When you find an available port for all your PCs, check its availability with any workstations in your testing network (as described below). Port number conflicts are rare. Note You only have to set the port number once.
Handle port number conflict
Users Guide
357
If you use TCP/IP on workstations...

For Motif Agents, the Agent name is the TCP/IP name of the Agents machine. If you do not know a machine name, type hostname at the UNIX prompt.
Handle port number conflict
If when you start the Agent on a workstation, the system responds with the number of a port used instead of the default port, then you have a port number conflict. You can use the number provided by UNIX for that machine and leave the rest using the default. Or you can find a port number that all your machines can use. To do this, start the Agent on each workstation using a possible alternative port number as an argument to the port number switch (-p). The syntax is:
> agent -p nnnn
The value of nnnn must be in the range 1000 to 9999. If any workstation cannot use the proposed value (an unlikely possibility), try another. Then set this number on any PCs you have in the testing network (as described above). Note When you start an Agent that does not use the default port number, you must always specify the number you want used; otherwise UNIX selects a number for you.
Handle limited licenses
If your testbed seems to be short one SilkTest Agent license, this may be because SilkTest has started up an unplanned Agent on the local workstation. If you do not want to use the local workstation as a test machine, set the Runtime Options dialogs Agent Name field to (none) instead of (local). This will free up one license for a remote Agent.
If you run LAN Manager or Windows for Workgroups...

For LAN Manager networks or Windows for Workgroups, you might have to increase the SESSIONS value (the default is 6) to a higher value. This variable is defined in the protocol.ini file, which is normally located in your Windows directory. You should also increase the NCBS value in protocol.ini to twice the SESSIONS value. The LAN Manager network environment and Windows for Workgroups have the ability to use more than one protocol driver at a time. NetBEUI is the protocol driver frequently used by LAN Manager. In order for SilkTest and the Agent to run, the NetBEUI protocol must be the first protocol loaded. The LANABASE option under the [NETBEUI_XIF] section of protocol.ini must
358
Users Guide
be set to 0 (zero). If additional protocols are loaded, they must have a sequentially higher LANABASE setting. For example, if you are running both NetBEUI and TCP/IP, the LANABASE setting for NetBEUI would (as always) be 0 (zero), and that for TCP/IP would be 1 (one).
If you use NetBIOS on PCs...

Under Windows, you must install NetBEUI with NetBIOS. In the Network control panel, you must have NetBEUI set as the default protocol. In Windows NT, Windows 95, and Windows for WorkGroups, NetBIOS is started automatically. You have to explicitly assign a unique network name to remote Agents so that SilkTest can identify the Agent when your testcase issues a Connect function for that machine. This step is not necessary for Agents using TCP/IP because SilkTest automatically uses the workstations TCP/IP name. The name must be from 1 to 16 alphanumeric characters long and must not be the standard name you use for your machine itself or the name of any other distributed Agent. (On some systems, using the same name can cause a system crash. A safe alternative is to derive the Agent name from the machine name. For example, if a machine is called Rome, you could call the Agent Rome_QAP.)
Enable networking on Agent
You assign the selected name to the Agent when you enable networking for each Agent PC (this operation was described in One or more remote applications on page 355). Once the protocol has been picked for all the Agents and they are named, you can enable networking on the host. Do this by selecting Options/Runtime and selecting the NetBIOS network type. Then fill in the Agent name if you have a single-remote-application configuration.
Enable networking on NetBIOS host
Users Guide
359
360
Users Guide
1rh 2e C t p a
Implementing Client/Server Testing

Topic Page
What you will learn
Evolving a testing strategy Connecting to a target machine Using 4Tests parallel processing features Reporting distributed results Incremental functional test design Running tests on one remote target Running tests serially on multiple targets Testing clients plus server serially Testing clients concurrently Testing in parallel, but not synchronously Application states and the recovery system
362 362 363 370 371 372 372 373 374 376 378
Users Guide
361
21 IMPLEMENTING CLIENT/SERVER TESTING Evolving a testing strategy
Evolving a testing strategy

There are several reasons for moving your QA program from local to remote testing: You may have a stand-alone application that runs on many different platforms and now you want to simultaneously drive testing on all the platforms from one SilkTest host system. You may have been testing a client/server application as a single local application and now you want to drive multiple instances of the application so as to apply a heavier load to the server. You may want to upgrade your client/server testing so that your testcases can automatically initialize the server and recover from server failures in addition to driving multiple application instances. You may need to test applications that have different user interfaces and that communicate as peers.
If you are already a SilkTest user, you will find that your testing program can evolve in any of these directions while preserving large portions of your existing tests. This chapter helps you to evolve your testing strategy by showing the incremental steps you can take to move into remote testing. Think of this chapter as a tutorial that introduces the details of multiapplication, remote testing in small pieceswhile preparing you for the fully developed testing strategy described in Chapter 22, Multi-Application Testing. Note that while the testing techniques introduced in this chapter are all valid and might find use in your testing programs, the complete power of SilkTest can only be gained by using the techniques presented in Chapter 22, MultiApplication Testing.
Connecting to a target machine

Two ways to establish a connection
There are two ways to establish a connection to a target Agent from SilkTest running in the host: as a runtime option or in a script. As a runtime option: You can specify the targets name in the hosts Runtime Options dialog before you run a script or suite. This is sufficient when you are connecting to only one Agent. If you have been testing a script by running SilkTest and the Agent on the same system, you can then test the script on a remote system without editing your script by using this method.
362
Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Using 4Tests parallel processing features
In a script: SilkTest allows you to specify the targets name in an explicit connection command within the script. If you wish to connect to multiple Agents, you must do it this way. There are two connection commands; this chapter discusses the use of the Connect command and Chapter 22 describes the SetUpMachine command. The method you use depends on the type of network testing you are doing. The examples in this chapter show each method, as appropriate.
Remote recording
Once you establish a connection to a target machine, any action you initiate on the host machine (the one running SilkTest) is executed on the target machine. For example, if you initiate a Record/Testcase command on the host machine, you will record the interactions of the user manipulating the application under test on the target machine. In order to use the Record menus remote recording operations, you have to place the target machines name into the Runtime Options dialog (select Options/Runtime).
Using 4Tests parallel processing features

Developing concurrent scripts
A concurrent (or multithreaded) script is one in which multiple statements can execute in parallel. Concurrency allows you to more effectively test distributed systems by allowing multiple client applications to submit requests to a server simultaneously. The 4Test language fully supports the development of concurrent scripts, enabling a script to: Create and coordinate multiple concurrent threads Protect access to variables that are global to all threads Synchronize threads with semaphores Protect critical sections of code (for atomic operations) Recover from errors in the event of deadlock
This section explains each of these features.

Using 4Test to initiate parallel testing
SilkTest can run testcases in parallel on more than one machine. To run testcases in parallel, you can use parallel threads within main() or in a function called by main(). A more elegant alternative is to use a multitestcase function, which provides a robust multi-machine recovery system. For more information, see Chapter 22, Multi-Application Testing and Appendix D, Useful Multitestcase Code Templates. Note that, if you attempt to run testcases in parallel on the same machine, you will generate a runtime error.
Users Guide
363
21 IMPLEMENTING CLIENT/SERVER TESTING Using 4Tests parallel processing features Threads and concurrent programming
In the 4Test environment, a thread is a mechanism for interleaving the execution of blocks of client code assigned to different Agents so that one script can drive multiple client applications simultaneously. A thread is part of the script that starts it; a thread is not a separate script. Each thread has its own call stack and data stack. However, all the threads that a script spawns share access to the same global variables, function arguments, and data types. A file that one thread opens is accessible to any thread in that script. While the creation of a thread carries no requirement that you use it to submit operations to a client application, the normal reason for creating a multithread script is so that each thread can drive test functions for one client, which allows multiple client application operations to execute in parallel. When a script connects to a machine, any thread in that script is also connected to the machine. Therefore you must direct the testing operations in a thread to a particular Agent machine. The following sections, in particular Specifying the target provide more details. Note Threads interleave at the machine instruction level, therefore no single 4Test statement is atomic with respect to a statement in another thread.
Parallel processing statements
You create and manage multiple threads using combinations of the 4Test statements parallel, spawn, rendezvous, and critical: A spawn statement begins execution of the specified statement or block of statements in a new thread. Since the purpose of spawn is to initiate concurrent test operations on multiple machines, the structure of a block of spawned code is typically: a b c A SetMachine command, which directs subsequent machine operations to the specified Agent. A set of machine operations to drive the application. A verification of the results of the machine operations.
A rendezvous statement blocks execution of the calling thread until all threads that were spawned by the calling thread have completed. When the last child thread exits, it unblocks the parent thread. The rendezvous statement is unnecessary if it is the last statement at the end of the testcase. If the calling thread has no child threads, rendezvous does nothing. A parallel statement spawns a statement for each machine specified and blocks the calling thread until the threads it spawns have all completed. It condenses the actions of spawn and rendezvous and can make code more readable.
364
Users Guide
A critical statement guarantees that no other thread will execute until the specified statement or indented block of code has executed and passed control to the next statement at the level of the critical statement. Only one thread can have critical status at any time.
In 4Test, all running threads (those not blocked) have the same priority with respect to one another. 4Test executes one instruction for a thread, then passes control to the next thread. The first thread called is the first run, and so on. All threads run to completion unless they are deadlocked. 4Test will detect a script deadlock and raise an exception (see How 4Test handles script deadlock on page 370). Note that the 4Test exit statement terminates all threads immediately when it is executed by one thread.
The parallel statement
The parallel statement executes a single statement for each thread. Thus if you wish to run complete tests in parallel threads, use the invocation of a test function (which may execute many statements) with the parallel statement, or use a block of statements with spawn and rendezvous. To use the parallel statement, you have to specify the machines for which threads are to be started. You can follow the parallel keyword with a list of statements, each of which specifies a different Agent name. For example:
parallel DoSomething ("Client1") DoSomething ("Client2")
The DoSomething function then typically issues a SetMachine(sMachine) call to direct its machine operations to the proper Agent.
The spawn statement
You can use spawn to start a single thread for one machine, and then use successive spawn statements to start threads for other machines being tested. SilkTest scans for all spawn statements preceding a rendezvous statement and starts all the threads at the same time. However, the typical use of spawn is in a loop, as follows:
for each sMachine in lsMachine spawn // start thread for each sMachine SetMachine (sMachine) DoSomething () rendezvous
The above example could achieve the same result when written as follows:
for each sMachine in lsMachine spawn [sMachine]DoSomething () rendezvous
Users Guide
365
21 IMPLEMENTING CLIENT/SERVER TESTING Using 4Tests parallel processing features Specifying the target
While the typical purpose for a thread is to direct test operations to a particular test machine, you have total flexibility as to which machine is being driven by a particular thread at any point in time. For example, in the code below, the spawn statement starts a thread for each machine in a predefined list of test machines. The SetMachine command directs the code in that thread to the Agent on the specified machine. But the [server] machine handle operator directs the code in the doThis function to the machine named server. The code following the doThis invocation continues to be sent to the sMachine specified in the SetMachine command.
for each smachine in lsMachine spawn // start thread for each sMachine SetMachine (sMachine) // ... code executed on sMachine ["server"]doThis() // code executed on "server" // ...continue with code for sMachine rendezvous
Note that while the machine handle operator takes only a machine handle, 4Test implicitly casts the string form of the Agent machines name as a machine handle and so the machine name is effectively the same as a machine handle.
More about machine handle operator
You can use the SetMachine function to change target machines for an entire block of code. To specify the target machine for a single command, use the machine handle operator on the command. There are two forms of this operator: the arrow form (sMachine->) and the bracket form ([sMachine]). For example, to execute the SYS_SetDir function on the target machine specified by the sMachine1 variable, you do this:
sMachine1->SYS_SetDir (sDir)
Note To allow you to conveniently perform system related functions (SYS_) on the host, you can preface the function call with the machine handle operator, specifying the globally defined constant hHost as the argument to the operator. For example, to set the working directory on the host machine to c:\mydir, you do this:
hHost->SYS_SetDir ("c:\mydir")
You can use this syntax with a method call, for example:
sMachine-> TextEditor.Search.Find.Pick
But when invoking a method, this form of the machine handle must be the first token in the statement. The following examples show valid and invalid syntax:
366
Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Using 4Tests parallel processing features // Valid machine handle operator use for each sMachine in lsMachine sMachine-> TextEditor.Search.Find.Pick // Invalid machine handle operator use with method if (sMachine->ProjX.DuplicateAlert.Exists()) Print ("Duplicate warning on {sMachine} recipient.")
If you need to use this kind of statement, use the alternative form of the machine handle operator described below.
Alternative machine handle operator
An alternative syntax for the machine handle operator is the bracket form:
[hMachine] Any4TestFunctionCall ()
where [hMachine] is a machine handle (or use [sMachine.]) For example, to execute the SYS_SetDir function on the target machine specified by the string sMachineA, you do this:
[sMachineA] SYS_SetDir (sDir)
The correct form of the invalid syntax shown above is:

// Invalid machine handle operator use if ([sMachine]ProjX.DuplicateAlert.Exists()) Print ("Duplicate warning on {sMachine} recipient.")
To execute the SYS_SetDir function on the host machine, you can do the following:
[hHost] SYS_SetDir (sDir)
You can also use this form of the machine handle operator with a function that is not being used to return a value or with a method, as in the following examples.
for each sMachine in lsMachine [sMachine] FormatTest7 () for each sMachine in lsMachine [sMachine] TextEditor.Search.Find.Pick Protecting access to global variables
When a new thread is spawned, 4Test creates a new copy of all local variables and function arguments for it to use. However, all threads have equal access to global variables. To avoid a situation in which multiple threads modify a variable simultaneously, you need to declare the variable as shareable. A shareable variable is available to only one thread at a time. Note Instances where threads modify variables simultaneously will generate unpredictable results. Errors of this kind are very hard to detect. Make variables shareable wherever the potential for conflict exists.
Users Guide 367
A declaration for a shareable variable has the form: [scope] share data-type name [= expr] {, name [= expr]} scope can be either public or private. If omitted, the default is public. data-type is a standard or user-defined data type. name is the identifier that refers to the shareable variable. expr is an expression that evaluates to the initial value you want to give the variable. The value must have the same type you gave the variable. If you try to use a variable before its value is set, 4Test raises an exception.
At any point in the execution of a script, a shared variable can only be accessed from within the block of code that has explicitly been granted access to it. You request access to shareable variables by using the access statement. An access statement has the form: access name1, name2, ...
statement
where name1, name2, ... is a list of identifiers of optional length, each of which refers to a shareable variable and statement is the statement(s) to be executed when access to the variables can be granted. If no other thread currently has access to any of the shareable variables listed, 4Test executes the specified statement(s). Otherwise, 4Test blocks the thread where the access statement occurs until access can be granted to all the shareable variables listed. At that point, 4Test blocks competing threads and executes the blocked thread. For example:
share INTEGER iTestNum = 0 public share STRING asWeekDay [7] share ANYTYPE aWhoKnows void IncrementTestNum () access iTestNum iTestNum = iTestNum + 1 Synchronizing threads with semaphores
Use semaphores to mutually exclude competing threads or control access to a resource. A semaphore is a built-in 4Test data type that can only be assigned a value once. The value must be an integer greater than zero. Once it is set, your code can get the semaphore's value, but cannot set it.
368
Users Guide
The following code examples show legal and illegal manipulations of a variable of type SEMAPHORE:
SEMAPHORE semA = 10 semA = 20 // Legal // Illegal - existing semaphore // cannot be reinitialized if (semA == [SEMAPHORE]2)... // Legal - note the typecast Print ("SemA has {semA} resources left.") // Legal SEMAPHORE semB = 0 // Illegal - must be greater than 0
Note To compare an integer to a semaphore variable, you must typecast from integer to semaphore using [SEMAPHORE]. But note that you cannot cast a semaphore to an integer. To use a semaphore, you first declare and initialize a variable of type
SEMAPHORE. Thereafter, 4Test controls the value of the semaphore variable.
You can acquire the semaphore if it has a value greater than zero. When you have completed your semaphore-protected work, you release the semaphore. The Acquire function decrements the value of the semaphore by one and the Release function increments it by one. Thus, if you initialize the semaphore to 5, five threads can simultaneously execute semaphore-protected operations while a sixth thread has to wait until one of the five invokes the Release function for that semaphore. The Acquire function either blocks the calling thread because the specified semaphore is zero, or acquires the semaphore by decrementing its value by one. Release checks for any threads blocked by the specified semaphore and unblocks the first blocked thread in the list. If no thread is blocked, Release releases the semaphore by incrementing its value by one so that the next invocation of Acquire succeeds (does not block). A call to Acquire has the form:
void Acquire (SEMAPHORE semA)
where semA is the semaphore variable to acquire. A call to Release has the form:
void Release (SEMAPHORE semA)
where semA is the semaphore variable to release. If more than one thread was suspended by a call to Acquire, the threads are released in the order in which they were suspended. A semaphore that is assigned an initial value of 1 is called a binary semaphore, because it can only take on the values 0 or 1. A semaphore that is assigned an initial value of greater than one is called a counting semaphore because it is used to count a number of protected resources.
Users Guide
369
21 IMPLEMENTING CLIENT/SERVER TESTING Reporting distributed results
Example Suppose you are running a distributed test on eight machines using eight 4Test threads. Assume that the application you are testing accesses a database, but can support only three simultaneous users. The following code uses a semaphore to handle this situation:
SEMAPHORE DBUsers = 3 ... Acquire (DBUsers) access database Release (DBUsers)
The declaration of the semaphore is global; each thread contains the code to acquire and release the semaphore. The initial value of three ensures that no more than three threads will ever be executing the database access code simultaneously.
How 4Test handles script deadlock
It is possible for a multi-threaded 4Test script to reach a state in which competing threads block one another, so that the script cannot continue. This is called script deadlock. When the 4Test runtime environment detects deadlock, it raises an exception and halts the deadlocked script. For example, the following script will never exit successfully:
share INTEGER iIndex1 = 0 share INTEGER iIndex2 = 0 main () parallel access iIndex1 Sleep (1) access iIndex2 Print ("Accessed iIndex1 and iIndex2") access iIndex2 Sleep (1) access iIndex1 Print ("Accessed iIndex2 and iIndex1")
Reporting distributed results

You can view test results in each of several formats, depending on the kind of information you need from the report. Each format sorts the results data differently, as follows:
370
Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Incremental functional test design
Elapsed time sorts results for all threads and all machines in event order. This enables you to see the complete set of results for a time period and may give you a sense of the load on the server during that time period or may indicate a performance problem. Machine sorts results for all threads running on one machine and presents the results in time-sorted order for that machine before reporting on the next machine. Thread sorts results for all tests run under one thread and presents the results in time-sorted order for that thread before reporting on the next thread.
Incremental functional test design

SilkTest simplifies and automates the classic QA testing methodology in which testing proceeds from the simplest cases to the most complex. This incremental functional testing methodology applies equally well in the client/ server environment, where testing scenarios should proceed from the simplest functional testing of one instance of a client application, through functional and performance testing of a heavily loaded, multi-client configuration. Therefore, we recommend the following incremental progression for client/server testing: Perform functional testing on a single client application that is running on the same system as SilkTest, with the server application on the same system (if possible). Perform functional testing on a single remote client application, with the server application on a separate system. Perform functional and concurrency testing on two remote client applications. Perform stress testing on a single client application running locally or remotely. Perform volume load testing on a configuration large enough to stress the server application. Perform peak load testing on a large configuration (to the limits of the server, if possible). Perform performance testing on several sets of loads until you can predict performance.
Users Guide
371
21 IMPLEMENTING CLIENT/SERVER TESTING Running tests on one remote target
The rest of this chapter describes and gives examples of the techniques you use to direct test operations to one or more remote target applications and to control whether those tests run serially or concurrently.
Running tests on one remote target

Ways to specify a single remote target
There are three ways in SilkTest to specify that you want a script, suite, or testplan to run on a remote target instead of the host: Enter the target Agents name in the hosts Runtime Options dialog. You also need to select a network protocol in the dialog. In a suite, specify the target Agents name by enclosing it within brackets before the script or suite name:
[Ohio]myscript.t
You can select (none) in the hosts Runtime Options dialog and then specify the target Agents name in a call to the Connect function in your script. For example, to connect to a machine named Ontario:
testcase MyTestcase () Connect ("Ontario") // Call first testcase DoTest1 () // Call second testcase DoTest2 () Disconnect ("Ontario")
When you are only driving one remote target, there is no need to specify the current machine; all testcase code will automatically be directed to the only connected machine. When you use the multi-application support functions, you will not have to make explicit calls to Connect; the support functions will issue these calls for you.
Running tests serially on multiple targets

To run your scripts or suites serially on multiple target machines, specify the target Agents name within the suite file. For example, this code runs a suite of three scripts serially on two target machines named Ohio and Montana:
[Ohio] script1.t [Ohio] script2.t [Ohio] script3.t [Montana] script1.t 372 Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Testing clients plus server serially [Montana] script2.t [Montana] script3.t
Note that any spaces between the target Agents name and the script name are not significant. Alternatively, to run testcases serially on multiple target machines, switch among the target machines from within the script using 4Tests built-in Connect and Disconnect functions. For example, the following script contains a function named DoSomeTesting that is called once for each machine in a list of target machines, with the target Agents name as an argument:
testcase TestSerially () STRING sMachine // Define list of agent names LIST OF STRING lsMachines = {...} "Ohio" "Montana" // Invoke test function for each name in list for each sMachine in lsMachines DoSomeTesting (sMachine) // Define the test function DoSomeTesting (STRING sMachine) Connect (sMachine) Print ("Target machine: {sMachine}") // do some testing... Disconnect (sMachine)
You will rarely need to run one test serially on multiple machines. Please consider this example a step on the way to understanding parallel testing.
Testing clients plus server serially

In a client/server application, the server and its clients typically each run on different target machines. This section explains how to build tests that test the server and its clients in a serial fashion. In this scenario, the SetMachine function switches among the target machines on which the server and its clients are running. The following script fragment tests a client/server database application in the following steps: 1 2 Connect to three target machines: server, client1, and client2. Call the DoSomeSetup function, which calls SetMachine to make server the current target machine, and then perform some setup.
Users Guide
373
21 IMPLEMENTING CLIENT/SERVER TESTING Testing clients concurrently
Call the UpdateDatabase function once for each client machine. The function sets the target machine to the specified client, then does a database update. It creates a timer to time the operation on this client. Disconnect from all target machines.
testcase TestClient_Server () Connect ("server") Connect ("client1") Connect ("client2") DoSomeSetup ("server") UpdateDatabase ("client1") UpdateDatabase ("client2") DisconnectAll () DoSomeSetup (STRING sMachine) HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to do server setup goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer) UpdateDatabase (STRING sMachine) HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to update database goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer)
The preceding example shows how you direct sets of testcase statements to particular machines. If you were doing functional testing of one application, you might want to drive the server first and then the application. However, this example is not realistic because it does not show the support necessary to bring the different machines to their different application states and to recover from a failure on either machine.
Testing clients concurrently

This section demonstrates one way to perform the same tests concurrently on multiple clients. In concurrent testing, SilkTest executes one function on two or more clients at the same time.
374 Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Testing clients concurrently
For example, suppose you want to initiate two concurrent database transactions on the same record, and then test how well the server performs. To accomplish this, you need to change the script shown in the previous section to look like this:
testcase TestConcurrently () Connect ("server") Connect ("client1") Connect ("client2") DoSomeSetup ("server") // initialize server first Disconnect ("server") // testcase is thru with server spawn // start thread for client1 UpdateDatabase ("client1") spawn // start thread for client2 UpdateDatabase ("client2") rendezvous Disconnect ("client1") Disconnect ("client2") // synchronize
DoSomeSetup (STRING sMachine) // define server setup HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to do server setup goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer) UpdateDatabase (STRING sMachine) // define update test HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to update database goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer)
An alternative but equivalent approach is to use the parallel statement in place of the spawn and rendezvous:
testcase TestConcurrently2 () Connect ("server") Connect ("client1") Connect ("client2")
Users Guide
375
21 IMPLEMENTING CLIENT/SERVER TESTING Testing in parallel, but not synchronously DoSomeSetup ("server") Disconnect ("server") parallel UpdateDatabase ("client1") // thread for client1 UpdateDatabase ("client2") // thread for client2 // automatic synchronization Disconnect ("client1") Disconnect ("client2") DoSomeSetup (STRING sMachine) HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to do server setup goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer) UpdateDatabase (STRING sMachine) HTIMER hTimer hTimer = TimerCreate () TimerStart (hTimer) SetMachine (sMachine) // code to update database goes here TimerStop (hTimer) Print ("Time on {sMachine} is: {TimerStr (hTimer)}") TimerDestroy (hTimer)
Note If you use variables to specify different database records for each clients database transactions, you can use the above techniques to guarantee parallel execution without concurrent database accesses.
Testing in parallel, but not synchronously

This section illustrates a method for running test functions in parallel on multiple clients, but with different tests running on each client. This provides a realistic multi-user loadas opposed to a load in which all clients perform the same operations at roughly the same time.
376
Users Guide
21 IMPLEMENTING CLIENT/SERVER TESTING Testing in parallel, but not synchronously
The following example suggests a method by which each client, operating in a separate thread, executes a test that is assigned by a random number. The RandSeed function is called first so that the random number sequence will be the same for each iteration of this multi-user test scenario. This enables you to subsequently repeat the test with the same conditions. The example reads a list of client machines from a file, clients.txt, and receives the test count as in input argument. These external variables make the example scalable as to the number of machines being tested and the number of tests to be run on each. The number of different testcases is twelve in this example, but could be changed by modifying the SelectTest function and adding further test functions. For each machine in the client machine list, the example spawns a thread in which the specified client will execute a randomly selected test, repeating for the specified number of tests. Note that you could execute this test as it is written because it sets its own application states. However, when you use multi-application support (as described in Chapter 22) this is automatic. And if you wanted to use this approach to drive different applications or to initialize a server before starting the testing, you would have to add the multi-application support described in the next chapter.
testcase ParallelRandomLoadTest (INTEGER iTestCount) LIST OF STRING lsClients RandSeed (3) // list of client names ListRead (lsClients, "clients.txt") STRING sClientName for each sClientName in lsClients spawn // Connect to client, which becomes current machine Connect (sClientName) SetAppState (MyAppState) // Initialize application TestClient (iTestCount) Disconnect (sClientName) rendezvous TestClient (INTEGER iTestCount) for i = 1 to iTestCount SelectTest () SelectTest () INTEGER i = RandInt (1, 12)
Users Guide
377
21 IMPLEMENTING CLIENT/SERVER TESTING Application states and the recovery system // This syntax invokes Test1 to Test12, based on i @("Test{i}") () // Define the actual test functions Test1 () // Do the test . . . Test2 () // Do the test . . . . . . Test12 () // Do the test . . .
Application states and the recovery system

The examples given in this chapter illustrate the progression from testing on one remote machine to testing multiple machines and testing concurrently. However, they do not properly cover the complexity of multi-application testing because they do not consider the recovery system. When you are driving two applicationsfor example, setting up a server and then driving the client applicationsyou have to configure the recovery system so that it knows what application states and base states to apply to which machines. Chapter 22 covers these subjects and gives some classic client/server testing scenarios.
378
Users Guide
2e rh C t p a
Multi-Application Testing
SilkTest can easily drive multiple different applications simultaneously. Thus you can bring a servers database to a known state at the same time you are bringing multiple instances of the client application to their base state window. Likewise, you can drive a server database with several different client applications at the same time. This chapter shows you how to drive multiple applications and gives examples of testing methods you can use for several classic client/server testing scenarios.
What you will learn

Topic Page
Multi-application startup and recovery Testcase structure in a single-application environment Testcase structure in a multi-application environment Recording window declarations for remote machines Code for template.t template.t explained Code for concurrency test example Concurrency test explained Code for notification example 1 Notification example 1 explained Code for notification example 2 Notification example 2 explained
380 380 381 384 384 385 387 389 393 394 397 398
Users Guide
379
22 MULTI-APPLICATION TESTING Multi-application startup and recovery
Multi-application startup and recovery

The essential difference between single-application and multi-application testing is clearly the difference between one and many. When the following entities in a testcase are greater than one, they need special consideration and support functions found in SilkTest: Agent names Application main window names Sets of application states associated with each main window name
Multi-machine testing requires that you map both the name of an application and all application states for that application to the machine on which it will be tested. This makes it possible for you to direct test operations to the right machines, and it enables SilkTest to automatically set the machines to the proper application state before a test is run and to clean up after a test has failed.
Testcase structure in a single-application environment

The code that implements a testcase for a single application is similar to that of a testcase for applications on multiple separate machines in a client/server environment. This section summarizes the structure of the single-application version and some SilkTest components used to implement it. You can compare the structure with the support code needed for running multiple applications, which is described later in this chapter.
defaults.inc
The defaults.inc file is provided by SilkTest and implements the recovery system for a single application test. That is, it contains the DefaultBaseState function that performs any cleanup needed after an operation under test fails and returns the application to its base state. You can define a base state function to replace the DefaultBaseState function by defining an application state without using the basedon keyword. This creates an application state that 4Test executes instead of the DefaultBaseState function. The defaults.inc file contains four other functions that 4Test automatically executes unless you define functions that replace them: DefaultScriptEnter a null function, allows you to define a ScriptEnter function, as discussed below.
380
Users Guide
22 MULTI-APPLICATION TESTING Testcase structure in a multi-application environment
DefaultScriptExit logs an exception to the results file when a script exits because of an exception. DefaultTestcaseEnter executes the SetAppState function. If you have specified an application state for this testcase, the SetAppState function brings your test application to that state. If you have no application state defined, SetAppState brings the application to the base state (if necessary). DefaultTestcaseExit logs an exception to the results file when a testcase exits because of an exception. The function then executes the SetBaseState function, which calls a base state function that you have defined or the DefaultBaseState function.
The word Default in each of the above function names signifies that you can define alternative functions to replace these. If, for example, you define a function called TestcaseEnter, 4Test will invoke your function before executing any of the code in your testcase and will not invoke DefaultTestcaseEnter.
Other include files
Your testcase needs certain definitions that other testcases in your testing program will also need. These include: Window declarations Application states Utility functions
Placing these general purpose definitions in an include file (or several smaller files) saves repetitive coding. When you use SilkTest to record window declarations and application states, SilkTest names the generated file frame.inc.
Testcase structure in a multi-application environment

This section describes some SilkTest components that enable concurrent testing of more than one application. For example, there are functions that make it possible to drive both the client application and the clients server from SilkTest, to set each to its base state, and to recover each if it fails.
Invoking a testcase
The keyword for a testcase declaration is different when you are performing distributed testing. Instead of preceding the testcase function declaration with the keyword testcase, you must use the keyword multitestcase to give your testcase the multi-application recovery system. Declaring a function as a multitestcase gives that function the ability to invoke functions declared with the keyword testcase. A multitestcase thus can be viewed as a wrapper
Users Guide 381
for stand-alone testcases; it provides a means of assigning tests to particular machines and lets you invoke previously written testcases from the multitestcase file by simply adding a use statement to the file to include the testcase definitions. In the single-application environment, you invoke a testcase with no arguments or you may specify an application state function. When you are using multi-application environment support, you can pass the testcase the names of the machines to be tested during that execution of the testcase, but not the application state function. In a multi-application environment, one testcase can use multiple application states; you specify these in the required code at the beginning of the testcase (see Required testcase code next).
Required testcase code
There are certain functions that you must invoke in every testcase being used for multi-application testing. These functions enable you to drive several different applications at the same time and to map the different applications and associated application states to particular machines. After you have specified the application-to-machine mapping, you invoke another function that initializes all your test machines in preparation for your tests. The sections Code for template.t on page 384 and template.t explained on page 385 provide a detailed explanation of the required code. The multi-application environment uses the same defaults.inc file as does the single-application environment. However, when you define a function as a multitestcase, 4Test uses functions defined in the cs.inc file to invoke functions in defaults.inc. Thus it can pass the appropriate application states or base states to these functions, depending on the requirements of a particular test machine. cs.inc is an automatically included file that contains functions used only in the multi-application environment. The following four functions provide a recovery system for managing automated testing of client/server applications. SetMultiAppStates sets an application state for each connected machine, if the AppState machine data lists one; if not, it calls the DefaultBaseState function, which sets the application to its main window. SetMultiBaseStates sets the application to the lowest state in the application state hierarchy for each connected machine, if the AppState machine data lists an application state. The lowest application state is one in which the appstate declaration did not use the basedon keyword. If there is no AppState information associated with
defaults.inc
cs.inc
382
Users Guide
this machine, SetMultiBaseStates calls the DefaultBaseState function, which sets the application to its main window, invoking it beforehand if necessary. SetUpMachine connects SilkTest to an Agent on the specified machine. It provides a way to associate a main window declaration and an application state function with a machine name. These parameters are stored as data accessible by means of the GetMachineData function. Both of these names (the second and third arguments to the function) are optional; however, if you omit both arguments, you will have no recovery system. DefaultMultiTestCaseEnter executes at the beginning of a multitestcase. It invokes a DisconnectAll function. The invocation of the SetAppState function is performed by the SetMultiAppStates function because the DefaultTestCaseEnter function is not executed for a multitestcase. DefaultMultiTestCaseExit executes just before a multitestcase terminates. It logs any pending exception, then invokes SetMultiBaseStates and DisconnectAll.
Other include files
In the client/server environment, as opposed to the stand-alone environment, you can test two or more different applications at the same time. For example, you could run the functional tests for application A on a Microsoft Windows machine and a Motif-based machine at the same time that you are running the functional tests for application B on both Windows and Motif-based machines. The include file(s) that you must generate may therefore have to take into consideration different platforms and/or different applications. When you are driving two or more applications from SilkTest, you need separate window declarations for each different application. You must be certain that your main window declaration for each separate application is unique. If the same application is running on different platforms concurrently, you may need to use GUI specifiers to specialize the window declarations. 4Test will identify a window declaration statement that is preceded by a GUI specifier (for example, motif) as being true only on the specified GUI. Please see Chapter 13, Porting Tests to Other GUIs, for more details on GUI specifiers. In addition, you may find that the operations needed to establish a particular application state are slightly different between platforms. In this case, you just record application states for each platform and give them names that
Users Guide
383
22 MULTI-APPLICATION TESTING Recording window declarations for remote machines
identify the state and the GUI (for your convenience). This chapter explains how you map the application state to the machine that requires it, using the SetUpMachine function.
Recording window declarations for remote machines

Recording window declarations on a client machine that is not the host machine requires that you operate both SilkTest on the host machine and the application on its machine at the same time. You record window declarations and application states in much the same way for a remote machine as for an application running in the SilkTest host machine. The primary difference is that you start the recording operation by selecting Test Frame in SilkTest on the host system and you do the actual recording of application operations on the remote system. If you have two or more applications being tested in parallel, you need to have two or more sets of window declarations. You must have window declarations (and application states, if needed) for each different application. When recording window declarations and application states on a remote machine, you will find it convenient to have the machine physically near to your host system.
Code for template.t

This fragment of an example testcase shows the required code with which you start a multi-application testcase. It connects SilkTest to all the machines being tested and brings each to its first screen. This is just a template; you must tailor your code to fit your actual needs. The next section explains the significance of each of these lines of code.
multitestcase MyTest (STRING sMach1, STRING sMach2) SetUpMachine (sMach1, MyFirstApp, "MyFirstAppState") SetUpMachine (sMach2, MySecondApp, "MySecondAppState") SetMultiAppStates () spawn SetMachine (sMach1) // Here is placed code that drives test operations spawn SetMachine (sMach2)
384
Users Guide
22 MULTI-APPLICATION TESTING template.t explained // Here is placed code that drives test operations rendezvous // . . .
template.t explained
The following line of code is the first required line in a multi-application testcase file. It is the testcase declaration. Note that it does not pass an application state as in the stand-alone environment.
multitestcase MyTest (STRING sMach1, STRING sMach2)
In the multi-application environment the arguments to your testcase are names of the machines to be tested; you specify application states inside the testcase, as will be explained shortly. You can code the machine names argument(s) as you wish. For example, you can pass a file name as the only argument and then in the testcase you can read the names of the machines from that file. Or you can define a LIST OF HMACHINE data structure in your testplan (if you are using the testplan editor) to specify the required machines and pass the name of the list when you invoke the testcase from the testplan. This template assumes that you are using a testplan and that it passes the Agent names when it invokes the testcase. For this example, the testplan might specify the following:
Mytest ("Client1", "Client2")
The next two code lines are the first required lines in the testcase:
SetUpMachine (sMach1, MyFirstApp, "MyFirstAppState") SetUpMachine (sMach2, My2ndApp, "My2ndAppState")
You must execute the SetUpMachine function for every client machine that will be tested. For each SetUpMachine call, you specify the application to be tested (by passing the name of the main window) and the state to which you want the application to be set (by passing the name of the application state if you defined one). The SetUpMachine function issues a Connect call for a machine you want to test and then configures either the base state or a specified application state. It does this as follows: It associates the client applications main window name with the specified machine so that the DefaultBaseState function can subsequently retrieve it to set the base state.
Users Guide
385
22 MULTI-APPLICATION TESTING template.t explained
It associates the name of the applications base state (if one is specified) with the specified machine so that the SetMultiAppStates function can subsequently retrieve it and set the application to that state at the start of the testcase.
The first argument for SetUpMachine is the machine name of one of your client machines. The second argument is the name you supply in your main window declaration in your test frame (frame.inc) file. For this example, the frame.inc file specifies the following:
window MainWin MyFirstApp
Because this template specifies two different applications, it requires two different test frame files. The third argument is the name you provide for your application state function in your appstate declaration for this test. For this example, the appstate declaration is the following:
appstate MyFirstAppState () based on MyFirstBaseState
The appstate declaration could also be of the form:

appstate MyFirstBaseState ()
Although the DefaultBaseState function is designed to handle most types of GUI-based applications, you may find that you need to define your own base state. It would be the application state that all your tests for this application use. In this case, you would still pass this application state to SetUpMachine so that your application would always be brought to this state at the start of each testcase. This template specifies two application states for generality. You would not use an application state if you wanted to start from the main window each time. If you have a number of tests that require you to bring the application to the same state, it saves testcase code to record the application state once and pass its name to SetUpMachine. You will probably place your application state declarations in your test frame file.
SetMultiAppStates ()
The SetMultiAppStates function must always be called, even if the testcase specifies no application state, because SetMultiAppStates calls the DefaultBaseState function in the absence of an appstate declaration. SetMultiAppStates uses the information that SetUpMachine associated with each connected machine to set potentially different application states or base states for each machine.
spawn SetMachine (sMach1)
386
Users Guide
22 MULTI-APPLICATION TESTING Code for concurrency test example // Here is placed code that drives test operations
The spawn statement starts an execution thread in which each statement in the indented code block below it runs in parallel with all currently running threads. There is no requirement that your testcase should drive all your test machines at the same timehowever, this is usually the case. The SetMachine function directs 4Test to execute this threads code by means of the Agent on the specified machine. This thread can then go on to drive a portion, or all, of the test operations for this machine.
spawn SetMachine (sMach2) // Here is placed code that drives test operations rendezvous // . . .
The second spawn statement starts the thread for the second machine in this template. The rendezvous statement blocks the execution of the calling thread until all threads spawned have completed. You can use the rendezvous statement to synchronize machines as necessary before continuing with the testcase.
Code for concurrency test example

The concurrency test example shown below is designed to allow any number of test machines to attempt to access a server database at the same time. This tests for problems with concurrency, such as deadlock or out-of-sequence writes. This example uses only one application. However, it is coded in the style required by the multi-application environment because you will probably want to use an Agent to start and initialize the server during this type of test. There is no requirement in the client/server environment that you use the single-application style of testcase just because you are driving only one application. For consistency of coding style, you will probably find it convenient to always use the multi-application files and functions. The section after this one explains this code example in detail.
const ACCEPT_TIMEOUT = 15 multitestcase MyTest (LIST OF STRING lsMachine) STRING sMachine
Users Guide
387
22 MULTI-APPLICATION TESTING Code for concurrency test example INTEGER iSucceed STRING sError for each sMachine in lsMachine SetUpMachine (sMachine, Personnel) SetMultiAppStates () /*** HAVE EACH MACHINE EDIT THE SAME EMPLOYEE ***/ for each sMachine in lsMachine spawn /*** SET THE CURRENT MACHINE FOR THIS THREAD ***/ SetMachine (sMachine) /*** EDIT THE EMPLOYEE RECORD "John Doe" ***/ Personnel.EmployeeList.Select ("John Doe") Personnel.Employee.Edit.Pick () /*** CHANGE THE SALARY TO A RANDOM NUMBER BETWEEN 50000 AND 70000 ***/ Employee.Salary.SetText ([STRING] RandInt (50000, 70000)) rendezvous /*** ATTEMPT TO HAVE EACH MACHINE SAVE THE EMPLOYEE RECORD ***/ for each sMachine in lsMachine spawn /*** SET THE CURRENT MACHINE FOR THIS THREAD ***/ SetMachine (sMachine) /*** SELECT THE OK BUTTON ***/ Employee.OK.Click () /*** CHECK IF THERE IS A MESSAGE BOX ***/ if (MessageBox.Exists (ACCEPT_TIMEOUT)) SetMachineData (NULL, "sMessage", MessageBox.Message.GetText ()) MessageBox.OK.Click () Employee.Cancel.Click () else if (Employee.Exists ()) AppError ("Employee dialog not dismissed after {ACCEPT_TIMEOUT} seconds")
388
Users Guide
22 MULTI-APPLICATION TESTING Concurrency test explained rendezvous /*** VERIFY THE OF NUMBER OF MACHINES WHICH SUCCEEDED ***/ iSucceed = 0 for each sMachine in lsMachine sError = GetMachineData (sMachine, "sMessage") if (sMessage == NULL) iSucceed += 1 else Print ("Machine {sMachine} got message '{sMessage}'") Verify (iSucceed, 1, "number of machines that succeeded")
Concurrency test explained

Before you record and/or code your concurrency test, you record window declarations that describe the elements of the applications GUI. These are placed in a file named frame.inc, which is automatically included with your testcase when you compile. Use SilkTest to generate this file because SilkTest does most of the work. The following code sample gives just those window declarations that are used in the concurrency test example:
window MainWin Personnel tag "Personnel" PopupList EmployeeList Menu Employee tag "Employee" MenuItem Edit tag "Edit" // ... window DialogBox Employee tag "Employee" parent Personnel TextField Salary tag "Salary" PushButton OK tag "OK" // ...
The following explanation of the concurrency test example gives the testing paradigm for a simple concurrency test and provides explanations of many of the code constructs. This should enable you to read the example without
Users Guide 389
22 MULTI-APPLICATION TESTING Concurrency test explained
referring to the online Help. There you will find more detailed explanations of these language constructs, plus explanations of the constructs not explained here. The explanation of each piece of code follows that code.
const ACCEPT_TIMEOUT = 15
The first line of the testcase file defines the timeout value (in seconds) to be used while waiting for a window to appear.
multitestcase MyTest (LIST OF STRING lsMachine)
The testcase function declaration starts with the multitestcase keyword. It specifies a LIST OF STRING argument that contains the machine names for the set of client machines to be tested. You can implement and maintain this list in your testplan (using the testplan editor). The machine names you use in this list are your target machines Agent names. See Chapter 20, Configuring SilkTest for Client/Server Testing which provides platformspecific instructions for determining Agent names.
for each sMachine in lsMachine SetUpMachine (sMachine, Personnel)
To prepare your client machines for testing, you must connect SilkTest to each Agent and, by means of the Agent, bring up the application on each machine. In this example, all Agents are running the same software and so all have the same MainWin declaration and therefore just one test frame file. This means you can initialize all your machines the same way; for each machine being tested, you pass SetUpMachine the main window name you specified in your test frame file. The SetUpMachine function issues a Connect call for each machine. It associates the main window name you specified (Personnel) with each machine so that the DefaultBaseState function can subsequently retrieve it.
SetMultiAppStates ()
The SetMultiAppStates function reads the information associated with each machine to determine whether the machine needs to be set to an application state. In this case no application state was specified (it would have been a third argument for SetUpMachine). Therefore, SetMultiAppStates calls the DefaultBaseState function for each machine. In this example, DefaultBaseState drives the Agent for each machine to open the main window of the Personnel application. This application is then active on the client machine and 4Test can send testcase statements to the Agent to drive application operations.
for each sMachine in lsMachine spawn // The code to be executed in parallel by // all machines... (described below)
390
Users Guide
22 MULTI-APPLICATION TESTING Concurrency test explained rendezvous
Because this is a concurrency test, you want all client applications to execute the test at exactly the same time. The spawn statement starts an execution thread in which each statement in the indented code block runs in parallel with all currently running threads. In this example, a thread is started for each machine in the list of machines being tested. 4Test sends the statements in the indented code block to the Agents on each machine and then waits at the rendezvous statement until all Agents report that all the code statements have been executed. The following is the code defined for the spawn statement:
// The code to be executed in parallel by // all machines: SetMachine (sMachine) Personnel.EmployeeList.Select ("John Doe") Personnel.Employee.Edit.Pick () Employee.Salary.SetText [STRING] RandInt (50000, 70000))
Each thread executes operations that prepare for an attempt to perform concurrent writes to the same database record. The SetMachine function establishes the Agent that is to execute the code in this thread. The next two statements drive the applications user interface to select John Does record from the employee list (a PopupList) and then to pick the Edit option from the Employee menu. This opens the Employee dialog box and displays John Does employee record. The last thread operation sets the salary field in this dialog box to a random number. At this point the client is prepared to attempt a write to John Does employee record. When this point has been reached by all clients, the rendezvous statement is satisfied, as described above, and 4Test can continue with the next script statement.
for each sMachine in lsMachine spawn SetMachine (sMachine) Employee.OK.Click () if (MessageBox.Exists (ACCEPT_TIMEOUT)) SetMachineData (NULL, "sMessage", MessageBox.Message.GetText ()) MessageBox.OK.Click () Employee.Cancel.Click ()
Users Guide
391
22 MULTI-APPLICATION TESTING Concurrency test explained else if (Employee.Exists ()) AppError ("Employee dialog not dismissed after {ACCEPT_TIMEOUT} seconds") rendezvous
Now that all the clients are ready to write to the database, the script creates a thread for each client, in which each attempts to save the same employee record at the same time. There is only one operation for each Agent to execute: Employee.OK.Click, which clicks the OK button to commit the edit performed in the previous thread. The test expects the application to report the concurrency conflict with message boxes for all but one client and for that client to close its dialog box within 15 seconds. The if-else construct saves the text of the message in the error message box by means of the SetMachineData function. It then closes the message box and the Employee window so that the recovery system will not report that it had to close windows. This is good practice because it means fewer messages to interpret in the results file. The else if section of the if-else checks to see whether the Employee window remains open (presumably because it is held by a deadlock condition); this is a testcase failure. In this case, the AppError function places the string ***ERROR: in front of the descriptive error message and raises an exception; all Agents terminate their threads and the testcase exits.
iSucceed = 0 for each sMachine in lsMachine sMessage = GetMachineData (sMachine, "sMessage") if (sMessage == NULL) iSucceed += 1 else Print ("Machine {sMachine} got message '{sMessage}'") Verify (iSucceed, 1, "number of machines that succeeded")
The last section of code evaluates the results of the concurrency test in the event that all threads completed. If more than one client successfully wrote to the database, the test actually failed. GetMachineData retrieves the message box message (if any) associated with each machine. If there was no message, iSucceed is incremented; it holds the count of successes. The Print function writes the text of the message box to the results file for each machine that had a message box. You can read the results file to verify that the correct message was reported. Alternatively, you could modify the test to automatically verify the message text.
392 Users Guide
22 MULTI-APPLICATION TESTING Code for notification example 1
The Verify function verifies that one and only one machine succeeded. If the comparison in the Verify function fails, Verify raises an exception. All exceptions are listed in the results file.
Code for notification example 1

This section contains the complete testcase file for a single-user notification test. It shows a testing technique for a type of communication frequently used in client/server applications. The example after this one shows a notification test between two users running their own copies of the client application. This illustrates doing the simplest case first and then adding the next level of complexity when you go from one user to two users. The section after this test code explains the testing technique.
// ccmail.t use "ccmail.inc" LogMeIn() LogInUser(GetMachineData( NULL, "Username" ), GetMachineData( NULL, "Password" ) ) //-----------------------------------------------------------multitestcase SingleUserNotification ( STRING sMachine1 optional ) if( sMachine1 == NULL ) sMachine1 = "(local)" //=== MULTI-APPLICATION SETUP SECTION ===================// SetUpMachine( sMachine1, CcMail, "EnsureInBoxIsEmpty" ) SetMachineData( sMachine1, "Username", "QAtest1" ) SetMachineData( sMachine1, "Password", "QAtest1" ) SetMultiAppStates() //=== TEST BEGINS HERE ==================================// SetMachine( sMachine1 ) SimpleMessage( "QAtest1", "Message to myself", "A message to myself" ) Verify( CcMailNewMailAlert.Exists( NOTIFICATION_TIMEOUT ), TRUE ) Verify( CcMailNewMailAlert.IsActive(), TRUE, "ALERT" ) CcMailNewMailAlert.OK.Click() CcMail.xWindow.GoToInbox.Pick ()
Users Guide
393
22 MULTI-APPLICATION TESTING Notification example 1 explained Verify( CcMail.Message.DeleteMessage.IsEnabled(), TRUE, "MESSAGE WAITING" )
Note This script continues in Code for notification example 2 on page 397.
Notification example 1 explained

The first line in the testcase file is a comment that lists the name of the file holding this code.
// csmail.t
The next line is an include statement. The explanations for each fragment of code follow that code.
use "ccmail.inc"
The ccmail.inc file is defined for this testcase. It contains the window declarations for the application plus application state definitions and definitions for general purpose utility functions also needed by other testcases designed for this application. You can find the ccmail.inc file in the SilkTest Examples directory. Code fragments from that file are included as needed in this discussion.
LogMeIn() LogInUser(GetMachineData(NULL, "Username"), GetMachineData(NULL, "Password") )
The utility function LogMeIn is called by the Invoke method for the CC Mail main window, called CcMail. The LogInUser function is defined in ccmail.inc. The machine data that LogInUser retrieves for its arguments gets established by each test before the application state function for each machine is invoked.
multitestcase SingleUserNotification (STRING sMachine1)
The function declaration for the testcase passes in the name of the SilkTest Agent for the machine on which the application is running.
if(sMachine1 == NULL) sMachine1 = "(local)"
This if statement allows you to invoke the testcase without specifying a machine name when you want to run on the local machine.
SetUpMachine(sMachine1, CcMail, "EnsureInBoxIsEmpty")
394
Users Guide
22 MULTI-APPLICATION TESTING Notification example 1 explained
The SetUpMachine function provides the name of the main application window (CcMail) and the application state (EnsureInBoxIsEmpty) to be established by SilkTest. EnsureInBoxIsEmpty is defined in ccmail.inc. This statement is part of the standard setup code for multi-application tests. The standard multi-application setup code is documented in template.t explained on page 385 and Concurrency test explained on page 389. The setup code in this testcase is essentially the same. This is a single-user testcase and therefore does not actually need the setup methodology required by a multi-application test. However, this chapter describes client/server testing, which is frequently multi-application testing, and therefore all the example testcases use the multi-application coding methods. We recommend that you also follow this practice, since consistency of testing styles reduces coding errors in your testcases. One difference for this testcase is that this is an application that requires the user to log in. Therefore the following code fragment provides the user name and password for the application under test:
SetMachineData (sMachine1, "Username", "QAtest1") SetMachineData (sMachine1, "Password", "QAtest1")
These statements associate two pieces of information, named Username and Password, with the specified machine. In both cases the value of the associated information is the same, QAtest1. Now that this information is available to the application state function, that function can log the user in. This will happen as a result of the next statement.
SetMultiAppStates()
In this test, SetMultiAppStates will actually only set the application state for the one machine.
SimpleMessage ("QAtest1", "Message to myself", "A message to myself")
The above line invokes the following utility function from ccmail.inc, which sends the short message to the local machine:
Utility function void SimpleMessage (STRING sRecipient, STRING sSubject, STRING sBody) CcMail.Message.NewMessage.Pick() NewMessage.MailingLabel.Recipient.SetText (sRecipient) NewMessage.MailingLabel.Recipient.TypeKeys ("<Enter>") NewMessage.MailingLabel.Recipient.TypeKeys ("<Enter>") NewMessage.MailingLabel.SubjectField.SetText (sSubject) NewMessage.MailingLabel.SubjectField.TypeKeys ("<Enter>")
Users Guide
395
NewMessage.EditBody.Body.TypeKeys NewMessage.EditBody.Body.TypeKeys
(sBody) ("<Ctrl-s>")
This function uses standard methods on Ccmail window components (defined in ccmail.inc) to do the following: 1 2 3 4 Pick the NewMessage item from the Message menu. Enter the string in argument one into the Recipient field and press the Enter key twice to move to the Subject field. Enter the string in argument two into the Subject field and press Enter to move to the message body portion of the window (EditBody.Body). Type the string in argument three into the Body field and type <Ctrl-s> to send the message.
The next block of code verifies the results of the test.

Verify(CcMailNewMailAlert.Exists(NOTIFICATION_TIMEOUT), TRUE ) Verify(CcMailNewMailAlert.IsActive(), TRUE, "ALERT") CcMailNewMailAlert.OK.Click() CcMail.xWindow.GoToInbox.Pick () Verify(CcMail.Message.DeleteMessage.IsEnabled(), TRUE, "MESSAGE WAITING")
The above code does the following: 1 Verifies that the sent message was received (as indicated by the NewMailAlert message box). The NOTIFICATION_TIMEOUT value causes the Verify function to wait for that period of time for the window to exist. If the timeout value is reached, the Verify raises an exception. Verify that the dialog box CcMailNewMailAlert is active. If the Verify executes without an exception, click on the OK button in the CcMailNewMailAlert dialog box. Pick the GoToInbox menu item from the Window menu. Verify that a message exists in the Inbox by checking to see that the Message menu has its DeleteMessage menu item enabled. If the menu item is not enabled, there is no message in the Inbox and the Verify function raises an exception.
2 3 4 5
396
Users Guide
22 MULTI-APPLICATION TESTING Code for notification example 2
Code for notification example 2

This is the complete testcase file for a two-user notification test. It shows the next level of complexity in testing client/server notification operations. The section after this test code explains the testing technique.
//-----------------------------------------------------------// This testcase logs in as user QAtest1 on the first machine, // and logs in as user QAtest2 on the second machine; then // sends a message from the user on the first machine to the // user on the second machine; it then switches to the second // machine and waits to be notified that new mail has arrived. // multitestcase TwoUserNotification ( STRING sMachine1, STRING sMachine2 ) //=== MULTI-APPLICATION SETUP SECTION ===================// SetUpMachine( sMachine1, CcMail ) SetUpMachine( sMachine2, CcMail, "EnsureInBoxIsEmpty" ) SetMachineData( SetMachineData( SetMachineData( SetMachineData( sMachine1, sMachine1, sMachine2, sMachine2, "Username", "Password", "Username", "Password", "QAtest1" "QAtest1" "QAtest2" "QAtest2" ) ) ) )
SetMultiAppStates() //=== TEST BEGINS HERE ==================================// //--------------------------------------------------------// Switch to the first machine: SetMachine( sMachine1 ) // Send mail from user 1 to user 2 SimpleMessage("QAtest2", "Message to user 2", "Message from me to you.") //--------------------------------------------------------// Switch to the second machine: SetMachine( sMachine2 ) // Wait for notification to occur, then acknowledge it: Verify( CcMailNewMailAlert.Exists( NOTIFICATION_TIMEOUT ), TRUE ) Verify( CcMailNewMailAlert.IsActive(), TRUE, "ALERT" ) CcMailNewMailAlert.OK.Click() // Refresh the In box and check that a message is waiting there: CcMail.xWindow.GoToInbox.Pick () Verify( CcMail.Message.DeleteMessage.IsEnabled(), TRUE, "MESSAGE WAITING" )
Users Guide
397
Notification example 2 explained

The code in this two-user notification test is much the same as the code in the single-user example, except that the test is distributed across two CcMail applications. Thus the primary differences in this example are in program flow. The following is what happensnote that it happens sequentially rather than concurrently: Before the test starts: 1 Do SetUpMachine for two machines; the first machine defaults to the base state, but the second machine specifies an application state that ensures that its InBox is empty. Set the Username and Password values for both machines. Invoke SetMultiAppStates for both machines. Note that this function will set different application states for the two machines.
2 3
On Machine 1: 1 2 Do SetMachine to specify that Machine 1 should receive the next operation. Send a simple message to Machine 2.
On Machine 2: 1 2 3 4 5 Verify that the Alert dialog box exists. Verify that the Alert dialog box is active. If it is not, the exceptions error message will be Verify ALERT failed... If the Alert dialog box has appeared, dismiss it by clicking the OK button. Refresh the InBox by picking the Inbox choice from CcMails Window menu. Verify that the Message menus DeleteMessage menu item is enabled (proving that the message is in the Inbox). If Verify fails (the menu item is not enabled), the exceptions error message will read, Verify MESSAGE WAITING failed...
398
Users Guide
GUI-Specific Tips and Tools

VtP I a r
In this part

Chapter Page
Chapter 23, Using the Windows Bitmap Tool Chapter 24, Calling Windows DLLs From 4Test Scripts Chapter 25, Using PVCS with SilkTest
401 423 429
Users Guide
399
400
Users Guide
3rh 2e C t p a
Using the Windows Bitmap Tool

Topic Page
What you will learn
Introduction to the Windows Bitmap Tool Windows Bitmap Tool window Starting the Windows Bitmap Tool Capturing bitmaps Comparing bitmaps Creating and applying masks
401 403 406 408 412 417
Introduction to the Windows Bitmap Tool

The Windows Bitmap Tool is an application that allows you to test and correct your Windows applications appearance by comparing two or more bitmaps and identifying the differences between them. The bitmap tool is especially useful for testing inherently graphical applications, like drawing programs, but you can also check the graphical elements of other applications. For example, you might want to compare the fonts you expect to see in a dialog with the fonts actually displayed, or you might want to verify that the pictures in toolbar buttons have not changed.
Capturing bitmaps
The bitmap tool can be used as a stand-alone product, in which you create and compare bitmaps of entire windows, client areas, the desktop, or selected areas of the screen. More commonly, however, you use the tool in conjunction with SilkTest. Bitmaps captured within SilkTest can be opened in
Users Guide
401
23 USING THE WINDOWS BITMAP TOOL Introduction to the Windows Bitmap Tool
the Windows Bitmap Tool, where you can compare them using the tools special comparison features. Conversely, bitmaps captured by the bitmap tool can be compared by SilkTests bitmap functions.
When to use the Windows Bitmap Tool
You might want to use the Windows Bitmap Tool in these particular situations: To compare a baseline bitmap against a bitmap generated during testing To compare two bitmaps from a failed test
For example, suppose during your first round of testing you create a bitmap using one of SilkTests built-in bitmap functions, CaptureBitmap. Assume that a second round of testing generates another bitmap, which your test script compares to the first. If the testcase fails, SilkTest raises an exception but cannot specifically identify for you the ways in which the two images differ. At this point you can open the Windows Bitmap Tool from the results file to inspect both bitmaps.
Features for comparing bitmaps
The Windows Bitmap Tool can create and graphically locate the differences between two bitmaps. You can use all Windows functionality to resize, save, and otherwise manipulate bitmaps, in addition to the special comparison features included in the tool. Using Windows Bitmap Tool, you can Show the areas of difference Zoom in on the differences Jump from one zoomed difference to the next View on-line statistics about the bitmaps Edit (copy and paste), print, and save bitmaps Create masks
Masks are screen images that are used to exclude bitmap differences or any part of a bitmap from comparison by the bitmap tool. You might consider masking any differences that you decide are insignificant or that you know will vary in an effort to avoid testcase failure. For example, suppose a testcase fails because one bitmap includes a flashing area of a dialog. In the bitmap tool you can block the flashing area from the two bitmaps by creating and applying a mask to them. Once a mask is applied and the masked bitmaps are saved, the mask becomes a permanent part of the baseline bitmaps you are comparing. Masks can also be saved in separate files and used in SilkTest testcases.
402
Users Guide
23 USING THE WINDOWS BITMAP TOOL Windows Bitmap Tool window
Windows Bitmap Tool window

The following figure shows the Windows Bitmap Tool window.
Bitmap Tool menus
This section documents the menus that are unique to the Windows Bitmap Tool Edit, Bitmap, Differences, and Capture.
Edit menu
Copy Paste New Mask Apply Mask
Copies the selected region to the clipboard. Inserts the contents of the clipboard into the bitmap tool. Creates a child window into which you can place a monochrome (black-and-white) mask bitmap. Applies the current mask bitmap to the baseline and result bitmaps in memory (not on disk).
Bitmap menu
Set Baseline Set Result
Toggles the bitmap in the active window as the baseline bitmap. Toggles the bitmap in the active window as the result bitmap.
Users Guide 403
Menu item
Description
Set Mask Fit to Window
Toggles the bitmap in the active window as the mask bitmap. Zooms or stretches the bitmap in the active child window to the exact size of the child window.
Differences menu
Show
Creates a Differences window that contains a monochrome bitmap of the differences between the current baseline bitmap and the current result bitmap. Black represents areas with no differences; white represents areas with differences. Available only when the Differences window is open. Fits and stacks the Baseline, Differences, and Result windows, and creates a special (not sizable) Zoom window with three panes. When you place the mouse cursor at a given location in the Baseline Bitmap, Differences, or Result Bitmap window The top pane of the Zoom window shows the same portion of the Baseline Bitmap window The middle pane shows the same zoomed portion of the Differences window The bottom pane shows the same zoomed portion of the Result Bitmap window. Moving the mouse within one of the three windows generates a simultaneous and synchronized real-time display in all three panes of the Zoom window. The zoom ratio is 4 to 1.
Zoom
Scan
Available only when the Zoom window is open. Starts automated zoom mode, in which the bitmap tool scans for differences from left to right and top to bottom. When the tool finds the first difference, it displays a square in the same relative location of the Baseline Bitmap, Differences, and Result Bitmap windows. In addition, that location is shown in all three panes of the Zoom window. Selecting Scan while the tool is in scan mode exits Scan and returns to zoom mode.
404
Users Guide
Menu item
Description
Next
Jumps to the location of the next difference in scan mode, denoted as a square in Baseline Bitmap, Differences, and Result Bitmap windows. That location is also shown in all panes of the Zoom window. In zoom mode, Next starts scan mode. Jumps to the location of the previous difference in scan mode, denoted as a square in the Baseline Bitmap, Differences, and Result Bitmap windows. That location is also shown in all panes of the Zoom window. Displays a Bitmap Comparison Statistics window, which compares baseline and result bitmaps with respect to width, height, color format, number of pixels, and the number and percentage of differences. Comparison takes masks into consideration. Creates a mask bitmap from the Differences window, swapping black and white, and applies the mask to the baseline and result bitmaps (in memory only).
Previous
Comparison Statistics
Convert to Mask
Capture menu
Window Client Area Rectangle
Captures window that you select. Captures client area of window you select. Captures rectangular area of choice. Move the mouse cursor to desired location to begin capture. As you press and hold the left button, drag the mouse to the screen location to end capture and then release the button. Captures entire desktop.
Desktop
Hide Window on Hides bitmap tool window temporarily when you select a Capture window or area to be captured. By default, menu item is checked (selected). Select the item again to deselect it. Use before beginning capture procedure.
Users Guide
405
23 USING THE WINDOWS BITMAP TOOL Starting the Windows Bitmap Tool
Starting the Windows Bitmap Tool

Three ways to start the bitmap tool
You can start Windows Bitmap Tool from The tools icon The Run dialog SilkTest results file
From the tools icon
Procedure To start the Windows Bitmap Tool from its icon and open bitmap files: 1 2 Select the Bitmap Tool icon in the program group containing SilkTest. The Windows Bitmap Tool window appears. Do one of the following:
To Action
Open an existing bitmap file Capture a new bitmap

From the Run dialog
Select File/Open and specify a file in the Open dialog. See Comparing bitmaps on page 412. See Capturing bitmaps on page 408.
To start the Windows Bitmap Tool from the Run dialog, specify the full path of the tools executable file, bitview.exe, which is in the SilkTest installation directory. Optionally, you can also specify parameters for opening bitmaps files. For example, if you want to open two bitmap files to be compared as well as an associated mask bitmap file, you might type:
install-dir\bitview.exe bitmap1.bmp bitmap2.bmp mask.msk
Procedure To start the Windows Bitmap Tool from the Run dialog: 1 2 Select Run from the Start menu. The Run dialog appears. Type the pathname of the tools executable file and any parameters in the Command Line text field and click OK. The Windows Bitmap Tool starts. Any bitmaps you specified on the command line are opened. Go to Comparing bitmaps on page 412. If you didnt specify any files in the command line, go to step 3.
406
Users Guide
23 USING THE WINDOWS BITMAP TOOL Starting the Windows Bitmap Tool
You can now open existing bitmaps created in SilkTest or in the tool, or you can capture new bitmaps:
To Action
Open an existing bitmap file Capture new bitmaps

From SilkTest results file
Select File/Open and specify a file in the Open dialog. Go to Comparing bitmaps on page 412. Go to Capturing bitmaps on page 408.
When the verification of a bitmap fails in a testcase, SilkTest saves the actual result in a bitmap file with the same name as the baseline bitmap but with the extension .rmp. So, if the bitmap file testbase.bmp fails the comparison, SilkTest names the result bitmap file testbase.rmp. It also logs an error message in the results file, as shown in the following figure.
Note In some cases this error message does not reflect an actual error. In particular, when SilkTest compares a bitmap it captured with one captured in the bitmap tool, the comparison fails because SilkTest stores footer information in its bitmap. The bitmaps might in fact be identical in all ways except for this information.
Users Guide
407
23 USING THE WINDOWS BITMAP TOOL Capturing bitmaps
Procedure To compare the actual bitmap generated by the testcase against the baseline bitmap generated by the bitmap tool or one of SilkTests built-in functions, click on the box icon preceding the error message.
As shown in the preceding figure, SilkTest opens the bitmap tool, opens both baseline (expected, .bmp file) and result (actual, .rmp file) bitmaps, creates a Differences window and places it in between the baseline and result bitmaps. To the right, the tool displays a three-paned Zoom window. (The Differences and the Zoom windows are described in Zooming in on the differences on page 414.)
Capturing bitmaps
Two ways to capture bitmaps
You can capture bitmaps in two ways: Windows Bitmap Tool SilkTests bitmap functions and methods embedded in testcases
This section explains how to capture bitmaps in the Windows Bitmap Tool and how to save them.
Capturing bitmaps in the Windows Bitmap Tool
You use the Capture menu in the Windows Bitmap Tool to capture a bitmap for any of the following in your application: A window
408
Users Guide
The client area of a window (working area, without borders or controls) A selected rectangular area of the screen (this is especially useful for capturing controls within a window) The desktop
Procedure To capture a bitmap in the Windows Bitmap Tool: 1 2 3 Start the application in which you want to capture bitmaps and set up the window or area to capture. Start the Windows Bitmap Tool, as described in Starting the Windows Bitmap Tool on page 406. If you want to change the current behavior of the tool window, select Capture/Hide Window on Capture. By default, the tool window is hidden during capture. Choose a window or screen area to capture:
To capture Action
Window Client area Selected rectangular area
Select Capture/Window. Click on the window you want to capture. Select Capture/Client Area. Click on the client area you want to capture. Select Capture/Rectangle. Move the mouse cursor to desired location to begin capture. While pressing and holding the left mouse button, drag the mouse to outline a rectangle, and then release the mouse button to capture it. During outlining, the size of the rectangle is shown in pixels. Select Capture/Desktop.
Desktop
The bitmap tool creates a new MDI child window containing the newly captured bitmap. The following figure shows a bitmap of the Font dialog. Note that the title bar reads Bitmap - (Untitled) and the status line at the bottom right of the window gives the dimensions of the bitmap (height by width), and the number of colors.
Users Guide
409
5 6
Repeat steps 3 and 4 to capture another bitmap. Alternatively, open an existing bitmap file. Save the bitmap, as described below.
Now you are ready to compare the two bitmaps (see Comparing bitmaps on page 412) or create a mask for the baseline bitmap (see Creating and applying masks on page 417).
Saving captured bitmaps
You can, if you want, save the bitmaps youve captured in the tool. You should adopt a naming convention that helps you distinguish between the first bitmap in the comparison, called the baseline bitmap, and the second bitmap, called the result bitmap. You can make the distinction in the file name itself, for example, by appending or prefixing a b or r to the name and using the same file extension for all bitmap files. Or you might use the same file name for both baseline and result bitmaps and add a unique file extension. Example You save baseline and result bitmaps of the Open dialog as open.bmp and open.rmp. Alternatively, you might name them openbase.bmp and openres.bmp, respectively. The following table lists the file extensions supported by the Windows Bitmap Tool. We recommend that you use .bmp for baseline bitmaps and .rmp for result bitmaps.
If you are saving And you want the file name to be Then use this file extension
Baseline bitmap Identical to the result bitmaps
.bmp
410
Users Guide
If you are saving
And you want the file name to be
Then use this file extension
Result bitmap
Identical to the baseline bitmaps
.rmp .bmp or .dib (Device Independent Bitmap)
Either baseline Unique or result bitmap
Note SilkTest uses .rmp for bitmaps that are captured within a testcase and fail verification.
Capturing bitmaps in SilkTest
You can compare a baseline bitmap captured in the bitmap tool with one captured in a SilkTest testcase of your application. If you write testcases by hand, you can use SilkTests built-in bitmap functions. If you prefer to record testcases via Record/Testcase, the Verify Window dialog allows you to record a bitmap-related verification statement.
Using bitmap functions
CaptureBitmap, SYS_CompareBitmap, WaitBitmap, and VerifyBitmap are built-in bitmap-related 4Test functions. In particular, VerifyBitmap is useful for comparing a screen image during the execution of a testcase to a baseline bitmap created in the bitmap tool. If the comparison fails, SilkTest saves the actual bitmap in a file. In the following example, the code compares the testcase bitmap (the baseline) against the bitmap of TestApp captured by VerifyBitmap:
TestApp.VerifyBitmap ("c:\sample\testbase.bmp")
For a complete description of these functions, see the online Help.

Using Record/ Testcase
Procedure To capture a bitmap during recording: 1 2 Invoke the dialog by pointing at the object you want to capture and pressing Ctrl+Alt. Then select the Bitmap tab. The Bitmap panel shown below is for the document window of the Text Editor application.
Users Guide
411
23 USING THE WINDOWS BITMAP TOOL Comparing bitmaps
3 4
Enter a file name in the Bitmap File Name field. Use the Browse pushbutton to help select a directory name. Select a radio button: Entire Window, Client Area of Window, or Portion of Window, and click OK. Note The Portion of Window radio button is identical to Capture/ Rectangle in the bitmap tool. To capture a portion of the window, move the mouse cursor to the desired location to begin capture. While pressing and holding the left mouse button, drag the mouse to outline a rectangle, and then release the mouse button to capture the bitmap. Note Keep in mind that during the capture of a bitmap, SilkTest always adds a bitmap footer to the bitmap file on the disk. This means that the physical size of the bitmap will be slightly bigger than if you capture the bitmap in the bitmap tool. The bitmap footer always contains the window tag for a given bitmap.
Comparing bitmaps
The Windows Bitmap Tool has four major comparison commands Show, Zoom, Scan, and Comparison Statistics. You can also use masks to alter the tools comparison capabilities. Masks are described in Creating and applying masks on page 417.
412
Users Guide
23 USING THE WINDOWS BITMAP TOOL Comparing bitmaps Baseline and Result bitmaps
To compare two bitmaps, you must designate one bitmap in the comparison as the baseline and the second bitmap as the result. While you may have many bitmap files open in the tool, at any one time only one bitmap can be set as the baseline and one as the result. If you want to set new baseline and result bitmaps, you must first unset the current assignments. These designations are temporary; you can at any point set and reset a bitmap as a baseline, result, or neither. Procedure To designate a bitmap as a baseline, select Bitmap/Set Baseline. The Set Baseline menu item is checked. The title bar of the child window changes to Baseline Bitmap -- filename.bmp. Procedure To designate a bitmap as a result, select Bitmap/Set Result. The Set Result menu item is checked. The title bar of the child window changes to Result Bitmap -- filename.rmp. Procedure To unset a bitmap designation, deselect the menu item. For example, to unset a baseline bitmap, deselect Bitmap/Set Baseline. The check mark is removed.
Comparison commands
The comparison commands of the Windows Bitmap Tool perform a pixel-bypixel comparison between the baseline bitmap and the result bitmap. All comparison commands Show, Zoom, Scan, and Comparison Statistics are provided on the Differences menu: Show indicates general areas of difference in a Differences window. Zoom simultaneously homes in on the same zoomed areas of the Baseline, Result, and Differences windows. Scan automates the zoom and allows you to jump from one difference to the next. Comparison Statistics displays statistics on the differences between the baseline and result bitmaps with respect to certain characteristics.
Rules for using the comparison commands
You should be familiar with these rules before using the commands: If you are comparing two new bitmaps captured in the tool, designate one bitmap as the baseline, the other as the result bitmap. If you are comparing two existing, saved bitmaps, open first the bitmap that you consider the baseline. The tool automatically designates the first bitmap you open as the baseline, and the second as the result. The commands must be used in this order: Show, Zoom, and Scan.
Users Guide
413
23 USING THE WINDOWS BITMAP TOOL Comparing bitmaps Showing areas of difference
The Show command creates a Differences window, which is a child window containing a black-and-white bitmap. Black represents areas with no differences and white represents areas with differences. Procedure To graphically show areas of difference between a baseline and a result bitmap, click Differences/Show. The bitmap tool displays a Differences window along with the source baseline and result bitmaps from which it was derived.
Zooming in on the differences
The Zoom command creates a special (not sizable) Zoom window with three panes and resizes and stacks the Baseline, Differences, and Result window. The top pane of the Zoom window contains a zoomed portion of the Baseline Bitmap window.The middle pane shows a zoomed portion of the Differences window. The bottom pane shows a zoomed portion of the Result Bitmap window. All three zoomed portions show the same part of the bitmap. When you move the mouse within any of the three windows, the bitmap tool generates a simultaneous and synchronized real-time display in all three panes of the Zoom window.
414
Users Guide
Procedure To zoom the Baseline Bitmap, Result Bitmap, and Differences window, select Differences/Show and then Differences/Zoom. The tool arranges the Baseline Bitmap on top, the Result Bitmap on the bottom, and the Differences window in the middle. To the right of these, the tool creates a Zoom window with three panes, arranged like the bitmap windows. See the following figure
.
Jumping from one difference to the next
The Scan command on the Differences menu automates zoom mode and causes the bitmap tool to scan for differences from left to right and top to bottom. When the first difference is found, a small square (32 x 32 pixels) is shown in the Baseline Bitmap, Result Bitmap, and Differences bitmap windows in the same relative location. In addition, that location is shown in all three panes in the Zoom window. Use Differences/Next to jump forward and Differences/Previous to jump back. Note You must first create a Differences window and a Zoom window using Differences/Show and Differences/Zoom. Procedure To scan bitmap differences, select Differences/Scan or Differences/Next. The tool indicates the location of the first difference it finds by placing a square in the same relative location of the Baseline, Result, and Differences windows. The three panes of the Zoom window also show the difference.
Users Guide 415
Procedure To move to the next or previous difference, use Differences/Next or Differences/Previous. Procedure To exit from scan mode, select Differences/Scan. Exiting Scan leaves the tool in zoom mode. Capturing the Zoom window in scan mode While in scan mode, you can capture the Zoom window to examine a specific bitmap difference. To capture all or part of the Zoom window while in scan mode: 1 2 3 Make sure the Capture/Hide Window is unchecked. If necessary, select the item to remove the check mark. Select Next or Previous until the Zoom window contains the difference you want to capture. Press one of the following:
To capture Action
Entire Zoom window Client area of Zoom window Selected area of Zoom window
Press Ctrl+W and select the Zoom window. Press Ctrl+A and select the Zoom window. Press Ctrl+R. Move the mouse cursor to desired location to begin capture. While pressing and holding the left mouse button, drag the mouse to the screen location to end capture, and release the mouse button.
4
Looking at statistics
Optionally, you can fit the bitmap in its window, resize it, and save it.
The Comparison Statistics command displays information about the baseline and result bitmaps, with respect to width, height, colors, bits per pixel, number of pixels, and the number and percentage of differences (in pixels). Procedure To see statistics comparing the baseline bitmap and result bitmaps, select Differences/Comparison Statistics. The Bitmap Comparison Statistics window appears, as shown below.
416
Users Guide
23 USING THE WINDOWS BITMAP TOOL Creating and applying masks
Note The number of colors is derived from the following formula: number of colors = 2 ^ (bits per pixel).
Creating and applying masks

A mask is a bitmap that you apply to the baseline and result bitmaps in order to exclude any part of a bitmap from comparison by the bitmap tool. For example, if you are testing a custom object that is painted on the screen and one part of the object is variable, you might want to create a mask to filter out the variable part from the bitmap comparison.
Saving a mask
Masks can be saved in a file, applied to the baseline and result bitmaps for you to examine on screen only, or applied to and saved in the baseline and result bitmap files. Once masks are applied and saved, they become a permanent part of the baseline and result bitmaps. The advantage of saving the mask alone is that later you can read in the mask file and apply it to the bitmap on screen, thus allowing you to keep the bitmap in its original state. You can supply the name of a mask bitmap file (as well as its associated baseline bitmap file) as an argument to SilkTests bitmap functions. The bitmap tool supports the .msk file extension for mask files. Alternatively, you can designate a mask in the file name and use the generic .bmp extension. We recommend, however, that you use the .msk extension. Four SilkTest bitmap-related functions GetBitmapCRC, SYS_ CompareBitmap, VerifyBitmap, and WaitBitmap accept mask files as arguments.
Users Guide
417
23 USING THE WINDOWS BITMAP TOOL Creating and applying masks Prerequisites
Before using the masking feature, you must: Capture or open two bitmaps to compare. Set baseline and result bitmaps, if currently unset. Determine which sections you need to mask. (Use one or more comparison features, if necessary, to locate bitmap differences.)
This section describes how to create, edit, and apply a mask.

Two ways to create masks
You can create a mask in two ways: By converting the Differences window to a mask. A mask created this way filters out all differences. By opening a new mask window and specifying rectangular areas to mask.
How to create a mask that excludes all differences
Procedure To create a mask that excludes all differences and apply it: 1 2 3 Open a Differences window, if one is not already open, by selecting Differences/Show. Select Differences/Convert to Mask. A message is displayed: Bitmaps are now identical on screen. Click OK. The bitmap tool creates an untitled mask bitmap from the Differences window, swapping black and white, and applies the mask to the baseline and result bitmaps. The following figure shows the mask bitmap created from the Differences window for the open baseline and result bitmaps.
418
Users Guide
Choose one of the following actions:

If you want to Then
Keep the baseline and result bitmaps with the mask applied Unapply the mask Keep the mask as it is Edit the mask
Save the bitmap files. The mask is now a permanent part of the bitmap files. Close the mask bitmap window. Saving is optional. Save the mask file. Select File/Save and close the mask bitmap window (which unapplies the mask). See How to edit an applied mask on page 421.
How to create a mask that includes selected areas
To create a mask that excludes some differences or just selected areas, and apply it: 1 Select Edit/New Mask. The bitmap tool creates an empty Mask Bitmap child window that is the same size as the baseline bitmap.
Users Guide
419
Using the Differences window to help you locate differences, place the mouse cursor in the baseline bitmap window at the position where you want to begin creating the mask. As you press and hold the left mouse button, drag the mouse cursor to outline a rectangle. Then release the left mouse button. The following figure provides an example of a user-created mask for the baseline bitmap of the Font dialog. Note that the rectangular outline in the baseline map changes to a filled-in rectangle. The mask bitmap window also contains a like-sized rectangle in the same relative location.
3 4
Repeat step 2 until you have completed the mask. If you want to delete a portion of the mask, place the mouse cursor in the baseline bitmap window at the position where you want to begin editing. While pressing the Shift key and then the left mouse button, drag the mouse cursor over the area of the existing map that you want to delete, and then release the Shift key and the left mouse button. The area of the mask overlapped by the rectangle outline disappears in both the baseline and mask bitmap window.
420
Users Guide
Select Edit/Apply Mask. The bitmap tool applies the mask to the result bitmap and closes the Differences window.
Choose one of the following actions:

If you want to Then
Keep the baseline and result bitmaps with the mask applied Unapply the mask Keep the mask as it is Edit the mask
Save the bitmap files. The mask is now a permanent part of the bitmap files. Close the mask bitmap window. Saving is optional. Save the mask file. Close the mask bitmap window (which unapplies the mask), but save the file when prompted. See How to edit an applied mask below.
How to apply a mask
Procedure To apply a saved mask: 1 2 Open the mask bitmap file, if not open, and select Bitmap/Set Mask. Select Edit/Apply Mask.
How to edit an applied mask
You can edit a mask after it has been applied: To add to the mask, place the mouse cursor in the baseline bitmap window at the position where you want to begin adding to the mask. As you press and hold the left mouse button, drag the mouse cursor to outline a rectangle. Then release the left mouse button. To delete part of the mask, place the mouse cursor in the baseline bitmap window at the position where you want to begin deleting part of the mask. While pressing and holding the Shift key, drag the mouse cursor over the area of the existing map that you want to delete, and then release the Shift key and the left mouse button.
Users Guide
421
422
Users Guide
4rh 2e C t p a
Calling Windows DLLs From 4Test Scripts

This chapter explains how to call a DLL from within a 4Test script. The chapter covers these topics:
Topic Page
What you will learn
Declaring a DLL function Aliasing a DLL name Using DLL support files installed with SilkTest Passing arguments to DLL functions
423 424 425 426
Declaring a DLL function

The following 4Test code declares a DLL function named MemManInfo. It takes a single out parameter named MemManInfo of type MEMMANINFO, and returns a boolean value.
dll "toolhelp.dll" BOOL MemManInfo (out MEMMANINFO MemManInfo)
As this example shows, a declaration for a DLL begins with the keyword dll. The general format is: dll dllname.dll prototype [prototype]...
Users Guide
423
24 CALLING WINDOWS DLLS FROM 4TEST SCRIPTS Aliasing a DLL name
dllname prototype
Prototype syntax
The name of the dll file that contains the functions you want to call from your 4Test scripts. A function prototype of a DLL function you want to call.
A function prototype has this form: return-type func-name ( [arg-list] ) return-type func-name arg-list The data type of the return value, if there is one. An identifier that specifies the name of the function. A list of the arguments passed to the function, specified as follows:
[pass-mode] data-type identifier pass-mode Specifies whether the argument is passed into the function (in), passed out of the function (out), or both (inout). If omitted, in is the default. To pass by value, make a function parameter an in parameter. To pass by reference, use an out parameter if you only want to set the parameters value; use an inout parameter if you want to get the parameters value and have the function change the value and pass the new value out. The data type of the argument. The name of the argument.
data-type identifier
Limitation
You can call DLL functions from 4Test scripts, but you cannot call member functions in a DLL.
Aliasing a DLL name

If a DLL function has the same name as a 4Test reserved word, or the function does not have a name but an ordinal number, you need to rename the function within your 4Test declaration and use the 4Test alias statement to map the declared name to the actual name. For example, the exit statement is reserved by the 4Test compiler. Therefore, to call a function named exit, you need to declare it with another name, and add an alias statement, as shown here:
dll "mydll.dll" my_exit () alias exit
424
Users Guide
24 CALLING WINDOWS DLLS FROM 4TEST SCRIPTS Using DLL support files installed with SilkTest
Using DLL support files installed with SilkTest

SilkTest is installed with the following include files that contain all the declarations, data types, and constants necessary for you to call hundreds of functions within the Windows API from your scripts. These files are:
Include file Description
msw.inc
Contains use statements for the include files that apply to 16-bit Windows: mswconst.inc, mswtype.inc, mswfuncs.inc, mswmsg.inc, and mswutil.inc. By including this one file in your 4Test scripts, you have access to all the information in the other include files.
msw32.inc
Contains use statements for the include files that apply to 32-bit Windows: mswconst.inc, mswtype.inc, mswfun32.inc, mswmsg32.inc, and mswutil.inc. By including this one file in your 4Test scripts, you have access to all the information in the other include files.
mswconst.inc
Declares constants you pass to DLL functions. These constants contain style bits, message box flags, codes used by the GetSystemMetrics function, flags used by the GetWindow function, window field offsets for the GetWindowLong and the GetWindowWord functions, class field offsets for the GetClassLong and GetClassWord functions, and menu function flags. Contains 4Test declarations for 16-bit functions in the user.exe, krnl386.exe, and toolhelp.dll files. Contains 4Test declarations for 32-bit functions in the user32.dll and kernel32.dll files. Declares 16-bit Microsoft Window messages, control messages, and notification codes. Declares 32-bit Microsoft Window messages, control messages, and notification codes. Declares many data types commonly used in the Windows API.
mswfuncs.inc mswfun32.inc mswmsg.inc mswmsg32.inc mswtype.inc
Users Guide
425
24 CALLING WINDOWS DLLS FROM 4TEST SCRIPTS Passing arguments to DLL functions
Include file
Description
mswutil.inc
Contains the following utility functions: PrintMemManInfo PrintTaskList PrintWindowDetail GetStyleBitList PrintStyleBits
Passing arguments to DLL functions

This section discusses:
Data types of arguments
The valid data types for arguments passed to DLL functions How to pass string arguments How to pass arguments to functions that expect pointers How to pass arguments that can be modified by the DLL function How to pass window handles to a DLL function
Since DLL functions are written in C, the arguments you pass to these functions must have the appropriate C data types. In addition to the standard 4Test data types, SilkTest also supports these ten C data types: char, int, short, and long unsigned char, unsigned int, unsigned short, and unsigned long float and double
Any argument you pass must have one of these data types (or be a record that contains fields of these types).
Strings
The char* data type in C is represented by the 4Test STRING data type. The default string size is 256 bytes. The following code fragments show how a char array declared in a C struct is declared as a STRING variable in a 4Test record:
// C declaration typedef struct { ... char szName[32]; ... }
426
Users Guide
24 CALLING WINDOWS DLLS FROM 4TEST SCRIPTS Passing arguments to DLL functions // 4Test declaration type REC is record ... STRING sName, size=32 ...
To pass a NULL pointer to a STRING, use the NULL keyword in 4Test. If a DLL sets an out parameter of type char* to a value larger than 256 bytes, you need to initialize it in your 4Test script before you pass it to the DLL function. This will guarantee that the DLL does not corrupt memory when it writes to the parameter. For example, to initialize an out parameter named my_parameter, include the following line of 4Test code before you pass my_ parameter to a DLL:
my_parameter = space(1000) Pointers
When passing pointers to C functions, use these conventions: Pass a 4Test string variable to a DLL that requires a pointer to a character (null terminated). Pass a 4Test array or list of the appropriate type to a DLL that requires a pointer to a numerical array. Pass a 4Test record to a DLL that requires a pointer to a record. 4Test records are always passed by reference to a DLL. Note You cannot pass a pointer to a function to a DLL function.
Arguments that can be modified by the DLL function
An argument whose value will be modified by a DLL function needs to be declared using the out keyword. If an argument is sometimes modified and sometimes not modified, then declare the argument as in and then, in the actual call to the DLL, preface the argument with the out keyword, enclosed in brackets. For example, the third argument (lParam) to the SendMessage DLL function can be either in or out. Therefore, it is declared as follows:
// the lParam argument is by default an in argument dll "user.dll" LRESULT SendMessage (HWND hWnd, UINT uiMsg, WPARAM wParam, LPARAM lParam)
Then, to call the DLL with an out argument, you use the keyword out, enclosed within brackets:
SendMessage (Open.hWnd, WM_GETTEXT, 256, [out] sText)
For more information, see dll declaration in online Help.

Window handles
If a parameter takes a window handle, use the hwnd property or the GetHandle method of the AnyWin class to get the window handle you need.
Users Guide
427
24 CALLING WINDOWS DLLS FROM 4TEST SCRIPTS Passing arguments to DLL functions
428
Users Guide
5rh 2e C t p a

Using SilkTest with PVCS allows you to manage your SilkTest test files with a powerful software configuration management tool. You can retrieve and run the latest version or any earlier version of a script, suite, include file, or testplan at any time in a consistent, repeatable manner. You can keep track of who makes a change, when, and why, and also ensure that files arent inadvertently overwritten or lost. Note This chapter assumes that you are familiar with PVCS. If not, you should consult your PVCS documentation.
What you will learn

Topic Page
Overview of using PVCS Setting up PVCS with SilkTest Using PVCS with SilkTest Possible problems
429 432 432 438
Overview of using PVCS

Benefits of using PVCS
You can use PVCS to: Lock revisions to prevent other users from modifying them and to prevent the loss of changes when files are updated simultaneously Reconstruct any revision of a file; you can go back to any previous revision of a work file quickly and easily Prevent unauthorized access to files by defining privileges, which set access levels for each user
Users Guide
429
25 USING PVCS WITH SILKTEST Overview of using PVCS

Assign a symbolic name to specific revisions of a group of files and retrieve those revisions of the files using that label Keep detailed information on who, when, what, and why changes were made Compress files when they are stored and keep only deltas (changes) between revisions
The PVCS connection to SilkTest works in conjunction with the PVCS Version Manager. It uses pre-existing PVCS configuration files, including the master PVCS configuration file (master.cfg) if it exists. It is also preconfigured to work easily with SilkTest test files. Using the PVCS connection to SilkTest, you can: Store and retrieve SilkTest test files from within SilkTest Set up PVCS archive directories from within SilkTest Retrieve any previous version of a file Give a revision of a file a label and later retrieve that version of the file using the label Retrieve files either just for use in running a test or with a lock that allows the file to be revised View the changes made from the last revision of the archive View the logged history of an archive Note Only the most commonly used PVCS functions are available from within SilkTest. If you need to perform any other operations or setup, you must run Version Manager.
Key terms
To review some key terms: The SilkTest test files (scripts, suites, testplans, and include files) that you use are in a working directory and are stored as archives by PVCS in an archive directory. PVCS has special configuration files which control how work files and archives are handled and where the archive and reference directories are found. Every working directory has a corresponding unique archive directory. Working directories can also share an archive directory if you have some test files that are shared for testing several applications. In every working directory there is a PVCS configuration file called vcs.cfg, whose main purpose is to point to the corresponding archive directories.
SilkTest/PVCS configurations
430
Users Guide
25 USING PVCS WITH SILKTEST Overview of using PVCS
Scenario 1: One working directory, one archive directory In the following scenario, all of the test files archives are in one archive directory.
Archive Directory ArchDir 1 Working Directory Tests
vcs.cfg
Scenario 2: Archive directory shared between tests In the following scenario, the tests for two applications have some shared files which are kept in a shared archive directory. The configuration file for the working directory for each application would have two archive directories in its archive directory list.
Archive Directories ArchDir 1 Working Directories App1 tests
vcs.cfg ArchDir 2 (shared)
App2 tests
ArchDir 3
vcs.cfg
The SilkTest/PVCS default configuration file
Supplied with SilkTest is a default PVCS configuration file that has been customized for archiving SilkTest test files. It is named pvcsqap.cfg and is installed in the Windows directory. This file is the source from which SilkTest creates all your the local vcs.cfg files that manage your SilkTest work files. Before you can perform any PVCS operations under SilkTest on the files in a working directory, a local vcs.cfg file must exist. The easiest way to generate one and assign some corresponding archive directories is with the Archive
Users Guide
431
25 USING PVCS WITH SILKTEST Setting up PVCS with SilkTest
Setup feature in SilkTest (see Setting up your archives on page 433). All the PVCS directives in pvcsqap.cfg will be copied to your new local configuration file (vcs.cfg) in the working directory along with the archive and reference directories you assign. These local vcs.cfg files are used in conjunction with the PVCS master configuration file (master.cfg) and override specific settings for handling SilkTest test files.
Modifying pvcsqap.cfg
If you want to customize pvcsqap.cfgfor example, add or change some directivesyou should modify the file before you have SilkTest generate your local vcs.cfg files from it. Changes you make to the pvcsqap.cfg file are used only when SilkTest creates a new local configuration file.
Setting up PVCS with SilkTest

To use the PVCS connection to SilkTest, you need PVCS Version Manager version 5.2.10 or later. Warning When running 32-bit SilkTest, you need to be running 32bit PVCS. Procedure To set up PVCS to use with SilkTest: 1 2 3 4 Rename nsnwnt.dll file to nsnwnt.sav in your PVCS directory (unless you are running on a NOVELL network). Modify pvcsqap.cfg in your Windows directory if you want to customize it (see the preceding section). Verify that you have a VCSID= userID environment variable defined. (Installation of PVCS should have put this in your autoexec.bat file.) If you have already been using PVCS to manage your test files, copy your generic PVCS configuration file as vcs.cfg into each working directory.

This section describes: How SilkTest initializes PVCS Setting up your archives Checking out files Checking in files
432
Users Guide
25 USING PVCS WITH SILKTEST Using PVCS with SilkTest
Using version labels Viewing the archive log Viewing differences
Initializing PVCS in SilkTest

When you start SilkTest, it will try to detect, load, and initialize PVCS. If this fails, there will be no PVCS archive support. If PVCS support is activated, the SilkTest menu bar is updated as follows: Two new menu items are added under File: Check Out and Check In. One new menu item is added under Options: PVCS Options. It is a cascading menu with the choices Archive Setup, Label Versions, View History Log, and View Differences.
The first time you access PVCS after starting SilkTest, PVCS security will be invoked. If PVCS security has been set up so there is a PVCS access database with password protection, you will be asked to log in. After you have accessed PVCS through SilkTest for the first time, you will not be asked to log in again. However, each time you try to check out or check in a file, PVCS will verify that you have the appropriate privileges for accessing the specific archive.
Setting up your archives

The first thing you will do is set up your archives. The major objective of archive setup is to assign archive and reference directories for each working directory. Procedure To set up your archives: 1 In SilkTest, select Options/PVCS Options/Archive Setup. The Archive Setup dialog is displayed.
Users Guide
433
2 3
Click on the Change Dir button and navigate to a working directory where you have test files and click OK. Set up the archive and reference directories. To add a new archive directory, specify it in the Archive Directory field at the bottom of the dialog. The path to the archive directory can either be typed or inserted by clicking the Browse button and selecting a directory. You can optionally specify a reference directory in the Reference Directory field. Click the Add button to append it to the list of archive/reference directory entries in the list box. The archive directories listed in the Archive/Reference Directory list box are the only ones where files in the current working directory can be retrieved from or stored to. The list of archive and reference directories is kept in the local PVCS configuration file (vcs.cfg) in the working directory. If you make any changes to the list of archive/reference directories and there is a configuration file in the current working directory the file will be modified; otherwise, a new file will be created based on the SilkTest/PVCS configuration file (pvcsqap.cfg). Selecting an existing entry in the Archive/Reference list box allows you to edit it, delete it, or change its position. Note The only modifications that archive setup makes to an existing vcs.cfg file is to the archive directory and reference directory settings.
4 5
Repeat these steps for your other working directories. Click the Close button.
434
Users Guide
25 USING PVCS WITH SILKTEST Using PVCS with SilkTest Naming archives
Note the following about naming archives: Archive names must be unique in an archive directory. For example, if a PVCS archive suffix of .??v is used, files with names of myfile.res and myfile.rem will access the same archive (myfile.rev). Warning Do not archive SilkTest script object files that have the .ino extension. If you do, you will have ambiguous archive files, because both an include file and an object file would be archived as file.inv. If the archive names are not unique within all the archive directories in the list, the first archive directory in the list with an archive of that name is the one that is used. To ensure the correct archive is being accessed, you need to carefully create archive directories so they relate to the various SilkTest projects. For example, the files that make up a test suite for testing a text editor application called TextEdit could be put in an archive directory i:\archives\textedit\tests or i:\archives\tests\textedit or i:\tests\textedit\archives, depending upon how your particular PVCS system is designed.
Checking out files

To use an archived file in SilkTest, you check it out of PVCS. Procedure To check out a file: 1 Select File/Check Out or press Ctrl+t. The Check Out dialog is displayed. Only the files in the current directory that have archives are displayed in the list of files.
Select the files you want to check out. You can check out multiple files at once by highlighting them in the multiple-select list of files.
Users Guide
435
By default, the most recently stored version of a selected file is retrieved. You can retrieve earlier versions by using either a revision number or a version label. Multiple files can be checked out with a common version label, but not with a common revision number. To display the Revisions and Version Labels list boxes, click the Advanced button. Note If you want to modify a file to check in later, you must select the Lock To Modify radio button (only the most recent version of a file can be locked for modification). 3 Click OK.
Checking in files
You check in a file to store a new version of a SilkTest file in a PVCS archive. The archive keeps track of who made the changes, when they were done, and any comments you want to enter about the changes that were made. To be able to check a modified file back into an archive, it must have first been checked out with a lock. Procedure To check in a file: 1 Select File/Check In or press Ctrl+k. The Check In dialog is displayed.
Select one or more files in the directory to be checked in. (If one of the files you have selected doesnt yet have an archive, see Creating a new file archive below before proceeding.) Click OK. The following dialog is displayed so you can make comments and/or assign a version label to the new revision of the file(s) being checked in.
436
Users Guide
Creating a new file archive
If the archive for a work file does not exist, it is automatically created when you check the work file in. The new archive is created in the first archive directory in the list of archive directories for the current working directory if it is allowed by the local and master PVCS configuration files. You can also create the archive with PVCS Version Manager. Tip If you want to create a new archive in an archive directory other than the one listed first for the working directory, do the following before checking the file in: Open the Archive Setup dialog and move the directory where you want to store the new archive to the top of the list. See Setting up your archives on page 433.
Using version labels

Procedure To assign a label to the latest revision of multiple files in a working directory: 1 Select Options/PVCS Options/Label Versions. The Label Archives dialog is displayed.
Users Guide
437
25 USING PVCS WITH SILKTEST Possible problems
Select the work files and enter the label in the Version Label field. If the label has already been given to an earlier revision of a file, the label will be reassigned to the most recent version.
Viewing the archive log

Procedure To see the history log of the archive for the currently active file in SilkTest, select Options/PVCS Options/View History Log. SilkTest displays the following information on each revision for the archive: revision number, date of change, who made the change, any comments made about the change, any version label given to the revision, and which revisions are locked and by whom. The archives log is put into a newly created temporary file named arclogxx.tmp in the working directory and opened in the editor.
Viewing differences
Procedure To see the differences between the currently active file in SilkTest and the last revision of its archive, select Options/PVCS Options/ View Differences. The differences are put into a new file named arcdifxx.tmp in the current working directory and opened in the editor.
Possible problems
This section lists some of the problems you might encounter using the PVCS extension to SilkTest. If you get PVCS ERROR: PVCS_E_CANT_OPEN_CONFIG_FILE, you do not have a pvcsqap.cfg file in your Windows directory. If you get PVCS ERROR: PVCS_E_LAF..., the PVCS file islv.ini in the Windows directory has problems. Only one program can use the PVCS DLLs at a time. So if you are running PVCS Version Manager, SilkTest might not be able to use the PVCS extension.
438
Users Guide
If you get General Protection Violation in Module win32s16.dll while trying to perform a PVCS function and you are running on Windows 95, Windows 98, or NT and are not running on a Novell network, you didnt rename the nsnwnt.dll in your PVCS directory.
Users Guide
439
440
Users Guide
Command Reference
IVtP I a r
In this part

Chapter Page
Chapter 26, Command Line Syntax Chapter 27, Menu Commands
443 447
Users Guide
441
442
Users Guide
6rh 2e C t p a
Command Line Syntax

This chapter explains the syntax of the partner command, which you can use to start the SilkTest executable from the command line. There are two ways to start the SilkTest executable program from the command line: In the Run dialog by selecting Run from the Start menu. From the command-line prompt in a DOS window.
What you will learn
Syntax of the partner command

The syntax is: partner [-m mach] [[-q] [-query query name] [-p mess] [-resexport] [-r] scr.t/suite.s/plan.pln/ link.lnk [args]]
Users Guide
443
26 COMMAND LINE SYNTAX Syntax of the partner command The options
The following table lists all the options to the partner command.
Option Description
-m
Specifies the target machine. The default is the current machine. (On UNIX, get the machine name using the hostname command.) Call the 4Test built-in function Connect to connect to a different machine at runtime. Must be the last option specified, followed only by the name of a script (and, optionally, arguments that the script takes), a suite, testplan, or link file. If you specify a link file, tells SilkTest to resolve the link and attempt to open the link target. Otherwise, tells SilkTest to run the specified script, suite, or testplan, optionally passing args as arguments to a script file Quits SilkTest after the script, suite, or testplan completes. Specifies a query. Must be followed by the name of a saved query. Tells SilkTest to perform an Include/Open All, then Testplan/Mark By Named Query, then Run/Marked Tests. Provided for use with a Windows shell program that is running SilkTest as a batch task. The option enables another Windows program to receive a message containing the number of errors that resulted from the script(s) run. SilkTest broadcasts this message using the Windows PostMessage function, with the following arguments: hWnd = HWND_BROADCAST uiMsg = RegisterWindowMessage (mess) wParam = 0 lParam = number of errors
-r
-q -query
-p
To take advantage of the -p option, the shell program that runs SilkTest should first register mess, and should look for mess while SilkTest is running. -resexport Tells SilkTest to export the most recent results sets to .rex files automatically. Specifying -resexport has the same effect as if each script run invokes the ResExportOnClose function during its execution. The name of the SilkTest script, suite, testplan, or link file to load, run, or open.
script.t/ suite.s/ plan.pln/ link.lnk
444
Users Guide
26 COMMAND LINE SYNTAX Examples
Option
Description
args
Optional arguments to a script file. You can access the arguments using the GetArgs function and use them with your scripts. For more information, see Passing arguments to a script on page 147.
Examples
To load SilkTest, type:
partner
To edit the test.inc include file, type:

partner test.inc
To load SilkTest and run the test.s suite, type:

partner -r test.s
To load SilkTest on system sys1 and run the test.t script, type:
partner -m sys1 -r test.t
To load SilkTest and run the test.t script with arguments, type:
partner -r test.t arg1 arg2
To load SilkTest, run tests.pln, and export the most recent results set from tests.res to tests.rex, type
partner -q -resexport -r tests.pln
To load SilkTest and run the tests marked by the query named query3 in tests.pln, type
partner -query query3 -r tests.pln
Users Guide
445
26 COMMAND LINE SYNTAX Examples
446
Users Guide
7rh 2e C t p a
Menu Commands
This chapter provides reference information about the menus in SilkTest, including the Include and Testplan menus of the testplan editor. Menus and menu items are listed in alphabetical order, not in the order they appear on the menu bar. The following table describes each of the menus.
Menu Use the commands on this menu to
What you will learn
SilkTest and the testplan editor menus
Breakpoint
Add or remove a breakpoint, which is a line of 4Test code where execution stops so that you can check the status of the application. A breakpoint appears as a large bullet in the left margin. On color monitors, it appears in red. The Breakpoint menu appears only in debugging mode. Find and fix runtime errors. The Debug menu appears only in debugging mode. Cut, copy, delete, and paste 4Test statements; undo changes; go to a specified line in a window; locate and replace text strings; and locate errors in 4Test code associated with a line in a results window or errors window. You can also toggle Visual 4Test on and off. Open, save, close, and print files. You can also run suites and run and debug 4Test scripts. Learn about functions, methods, and properties; access online help. Open, lock, unlock, save, and close include files containing subplans and master testplans.
Debug Edit
File Help Include
Users Guide
447
27 MENU COMMANDS
Menu
Use the commands on this menu to
Options
Set general options; set editor font and colors; set recorder, runtime, Agent, and class map options; enable and disable options for extensions; set fault trapping options; manage property sets; create, save, and close sets of runtime, Agent, and class map options and Help files; and enable and disable the QuickStart Wizard. Manipulate the outline (expand, collapse, change indentation) and add and delete comments. The Outline menu is active if a script file is active and you are displaying it using Visual 4Test (as opposed to Classic 4Test, which was used in previous releases of QA Partner).
Outline
Record
Record the following: Window declarations (GUI object descriptions) Application states Entire testcases 4Test statements Window locations and tags Classes
Results
View, delete, and extract the results of one or more script executions from a results file. You can also open the script associated with a given set of results. Compile, run, debug, and halt scripts, suites, and testplans. Define testplan attributes, invoke the Testplan Detail dialog, manage manual tests, find the next marked test in a testplan, go to the script associated with a test, insert a testplan template, mark and unmark tests, and generate completion reports. Display different elements of the running script, for example, local and global variables, the call stack, and breakpoints. You can also execute expressions. The View menu appears only in debugging mode. See a list of all open windows and make one active. Change the appearance and layout of child windows.
Run Testplan
View
Window
448
Users Guide
27 MENU COMMANDS Breakpoint menu
Breakpoint menu
The Breakpoint menu contains the following commands: Add... Delete... Delete All Toggle Note For complete information about debugging testcases, see Chapter 7, Using the Debugger.
Breakpoint/Add...
Available only in debugging mode. Opens the Add Breakpoint dialog, which allows you to add a breakpoint at any executable line of a function. To set a breakpoint, double-click on a module from the Modules list box, which displays all currently loaded scripts and include files. When you select a module, the Function field displays all the functions and testcases in the selected module. Select the function that you want to enter a breakpoint into. SilkTest enters the function name in the Breakpoint field. Select OK to have SilkTest set a breakpoint on the first line of that function. A breakpoint is denoted as a large bullet.
Breakpoint/Delete...
Available only in debugging mode. Opens the Delete Breakpoint dialog, which lets you delete one ore more breakpoints. The dialog lists all breakpoints by file name and line number.
Breakpoint/Delete All
Available only in debugging mode. Deletes all breakpoints, first prompting you for confirmation.
Users Guide
449
27 MENU COMMANDS Debug menu
Breakpoint/Toggle
Available only in debugging mode. Adds a breakpoint on the current line, if none exists, or deletes a breakpoint, if one exists, on the current line. A breakpoint is denoted as a large bullet.
Debug menu
The Debug menu contains the following commands: Abort Continue Exit Finish Function Reset Run Run to Cursor Step Into Step Over Note For complete information about debugging testcases, see Chapter 7, Using the Debugger.
Debug/Abort
Available only in debugging mode when you have executed scripts on a target machine. Terminates execution of the script you are debugging.
Debug/Exit
Available only in debugging mode. Exits debugging mode.
450
Users Guide
27 MENU COMMANDS Debug menu
Debug/Finish Function
Available only in debugging mode. Executes the script until the current function returns.
Debug/Reset
Available only in debugging mode. Frees memory and all variables, and clears the call stack of the script you are debugging.
Debug/Run and Debug/Continue

Available only in debugging mode. Runs the script in the debugging window until the first breakpoint, if any, is reached. Execution stops just before the line with the breakpoint. SilkTest marks the current line (the next line to be executed) with a triangle. The menu item changes from Run to Continue. When SilkTest completes the execution of the script, it displays a message box indicating that the script has terminated. During script execution, SilkTest displays a transcript window, which is similar to the results window. Unlike the results file, however, the output from debugging a script is not saved in a file, there are no statistics, and all the information is expanded automatically. The transcript window contains the script name, the testcase names, and a list of the errors encountered and their line numbers. At the bottom of the transcript window is a text field in which you can enter any statement to execute. The results of each statement you execute appear in the transcript window.
Debug Run To Cursor

Sets a temporary breakpoint (indicated by a hollow red circle in the margin) on the line containing the cursor. SilkTest immediately runs the script, stopping at the current line. The breakpoint is cleared after it is hit.
Users Guide
451
27 MENU COMMANDS Edit menu
Debug/Step Into
Available only in debugging mode and after using Debug/Run and execution has stopped at a breakpoint. Executes the current line of 4Test code in the active script or in a file called by the active script. If the current line has a breakpoint, SilkTest executes the line. If the current line contains a function call, control passes into the function; SilkTest stops at the first statement.
Debug/Step Over
Available only in debugging mode, and after using Debug/Run and execution has stopped at a breakpoint. Executes the current line of 4Test code in the active script or in a file called by the active script, without stepping into any functions called by the current line. Control stops at the next statement.
Edit menu
The Edit menu contains the following commands: Copy Cut Delete Find... Find Error Find Next Find Next Difference Goto Line... Paste Redo Replace... Undo Visual 4Test
452
Users Guide
Edit/Copy
Copies the selected text to the clipboard. Use with Edit/Paste.
Edit/Cut
Deletes the selected text from the active SilkTest window and copies it to the clipboard. Use with Edit/Paste.
Edit/Delete
Deletes the selected text. Unlike the Cut command, Delete does not copy text to the clipboard.
Edit/Find...
Opens the Find dialog. SilkTest searches in the active window for the text string you specify in the Find What text field. The default string is the last string you searched for. You can also specify the direction of the search (by default, down) and whether or not the search should be case sensitive (by default, case insensitive). If the search is successful, SilkTest highlights the first occurrence of the text string, expanding outlined code, if necessary, to do so. Select Find Next to have SilkTest locate the next occurrence of that text string. If the search is not successful, SilkTest displays an informational message.
Users Guide
453
Edit/Find Error
Available in the active error window or results window. How this command works depends on whether an error window or results window is active:
If this window is active Then SilkTest...
An errors window or script
In errors window, scans downward from the cursor position until it finds a line referring to a script file. It looks for that file among open editor windows (or opens it if it is not found) and places the cursor on the appropriate line of the script. In script, places the cursor on the appropriate line. Positions the cursor to the next error message or warning in the results window.
A results window
By default, Find Error finds both errors and warnings. If you want to skip over warnings, deselect the Find Error Stops at Warning check box in the Runtime Options dialog. See Options/Runtime... on page 482. You can select Find Error with the accelerator key F4.
By pressing F4 when You can
An errors window or script is active A results window is active
Step through each source line that contains an error. As you view the code, SilkTest displays the relevant error message on the status line. Cycle through each error message or warning in the results file
Edit/Find Next
Repeats the last find or replace operation (see Edit/Find... and Edit/ Replace...). This menu item changes to Replace Next after a replace operation, or to Replace All again after a global replace operation.
454
Users Guide
Edit/Find Next Difference

Available only when a Difference Viewer is the active window. Finds the next discrepancy between an actual and expected value. For more information, see Finding application logic errors on page 154.
Edit/Go to Line...
Opens the Go to Line dialog, which locates a specified line. In the outline editor, SilkTest expands the outline before it locates the specified line number. Once you click OK, SilkTest positions the cursor at the beginning of the specified line in the active window.
Edit/Paste
Inserts text that you have cut or copied to the clipboard into the active SilkTest window at the current insertion point. Note You cannot paste graphics into a SilkTest window.
Edit/Redo action
Reverses the undo you just accomplished through Edit/Undo. The Redo menu item describes the undo. For example, after undoing a cutting action, the Redo menu item becomes Redo Cut. When you select Edit/Redo action, SilkTest performs the action again; in this example, it cuts some text again.
Edit/Replace...
Opens the Replace dialog. When you click the Replace pushbutton, SilkTest replaces the string you specify in the Find What field with the string in the Replace With field. The default string is the one you specified in the last replacement operation. If there are no more occurrences of the text string to replace, SilkTest displays an information message.
Users Guide
455
27 MENU COMMANDS File menu
Edit/Undo action
Backs out of the most recent change you have made in the editor. When you make a change in the editor, the Undo menu item describes the action. For example, if you had just cut some text, the menu item would read Undo Cut. When you select Edit/Undo action, SilkTest undoes the editing action, in this example, the cutting action, and also changes the Redo menu item to describe the undone action, for example, Redo Cut. To reverse the undo you just did, use Edit/Redo.
Edit/Visual 4Test
If checked, starts Visual 4Test, an outline editor mode. All the commands on the Outline menu become available. Default is checked. To start the plain editor, select the menu item; the check mark is removed. To have the plain editor start by default, change the Outline Editor by Default field in the General Options dialog. See Options/General... on page 477. Note Changing from the plain editor to Visual 4Test might not work due to syntax errors.
File menu
The File menu contains the following commands: Check In Check Out Close Debug... Exit New... Open... Open Selection Page Setup Print... Printer Setup...
456
Users Guide
Quit Run... Save Save As... Save All Save Object File
In addition, the File menu contains a dynamic menu item of the form n operation file-name that allows you to repeat the most recent open, run, or debugging operation on a file name.
File/Check In
Available if PVCS is installed only. Checks a file into PVCS. For more information, see Chapter 25, Using PVCS with SilkTest.
File/Check Out
Available if PVCS is installed only. Checks a file out of PVCS. For more information, see Chapter 25, Using PVCS with SilkTest.
File/Close
Closes the active window. If you made changes to the window since the last time you saved the file, SilkTest prompts you to save the file.
File/Debug...
Opens the Debug dialog, from which you can choose a script to debug. Once you click OK, SilkTest opens the selected file in debugging mode. Special debugging commands are available. See the menu items on the Debug, Breakpoint, and View menus. For complete information about debugging testcases, see Chapter 7, Using the Debugger.
Users Guide
457
File/Exit
Closes SilkTest. SilkTest prompts you to save any changes made to files since the last time you saved them.
File/n operation file-name

Lists the most recent files you opened, ran, or debugged. You can click on an item to repeat the operation on the specified file. Items are prefaced by an integer, n, where the most recent operation is listed first. For example, the most recent file operation is prefaced by1. You can select an item by its number; for example, to repeat the operation specified by the second item, press Alt+F+2. By default, SilkTest displays four files. You can modify this number by editing File History Size on the General Options dialog. See Options/ General... on page 477.
File/New...
Opens the New dialog, which you use to create a new filea test frame, testplan, 4Test script, 4Test include file, suite, or text file. If you are using the testplan editor, you can click the wizard icon to invoke the QuickStart Wizard. When you select any of these file types except test frame, SilkTest opens a new editing window. The title bar of the new editing window displays the file type youve chosen and the string untitled. When you select test frame, SilkTest displays the New Test Frame dialog, which allows you to create a test frame file for an open application.
For more information about See
Test frames and include files Testplans Scripts Suites
Chapter 4, Recording a Test Frame Chapter 2, Creating Testplans Chapter 5, Designing and Recording Testcases Chapter 6, Running Tests and Interpreting Results
458
Users Guide
For more information about
See
QuickStart Wizard
Chapter 1, Overview
File/Open...
Opens the Open dialog, from which you select an existing file. If the specified file is already open, SilkTest makes its window the active window, rather than opening another copy of the file.
File/Print...
Opens the Print dialog, which you use to print all or part of the contents of the active editor or results window.
To
Action
Print the entire file or Print a range of lines
Select All Lines in the Print Range group box. or Highlight the lines you want to print and select Selected Lines Only in the Print Range group box.
Users Guide
459
To
Action
Print a header and/or footer Type literal text, variables, or a combination in the on each page Header and/or Footer fields: &f prints the file name &p prints the page number Example: If you type the following in the header field: Script: &f and the file name is testapp.t, the printed header will be: Script: testapp.t Print entire contents of file Check the Print Fully Expanded check box. or Print the file as shown on the screen Adjust the margins or Uncheck the Print Fully Expanded check box. Enter new values for the left, right, top, and bottom margins in the Margins group box. Default value is 0.50 inch. Select the output resolution you want from the Print Quality list box. Click the Font pushbutton. The Editor Font dialog appears. See page 476. Click the Setup pushbutton. The standard Printer Setup dialog appears.
Change the output resolution Change the font family, size, and style of the text Reconfigure the printer options
File/Printer Setup...
Opens the Printer Setup dialog, which you use to select printer settings, such as the printer, paper size and source, and page orientation of the output.
File/Run...
Opens the Run dialog, from which you select a script, suite, or testplan (if available) to run. Note SilkTest will run a script, suite, and testplan that has an extension of .t, .s, or .pln, respectively.
460
Users Guide
27 MENU COMMANDS Help menu
File/Save
Writes the contents of the active window to disk. If you open a file that is owned by someone else or that someone else currently has open, SilkTest opens a view-only copy of the file and changes the menu item File/Save to File/Save Object File. You cannot save the source file (since it is locked), but you can select File/Save Object File to write a a new object file. For more information, see Using object files on page 134.
File/Save As...
Opens the Save As dialog, which you use to save the contents of the active window with the name you specify.
File/Save All
Writes to disk all the open files that have changed since the last time you saved them.
File/Save Object File

Available only when a script or include file is view-only. Writes the contents of the current include file or script to disk in compiled form. Useful when you are working with a file that you dont have write access to and want to create an object file.
Help menu
The Help menu contains the following commands: Help Topics Library Browser... Online Users Guide... About...
Users Guide
461
27 MENU COMMANDS Include menu
Help/Help Topics...
Starts SilkTest online Help. For information on its contents, look at the topic About this Help.
Help/Library Browser...
Starts the Library Browser, an online facility for quickly looking up documentation for built-in and user-defined 4Test methods, properties, and functions. Note For information about using the Library Browser, see The Library Browser on page 38. For information about modifying the contents of the Library Browser, see Chapter 18, Modifying the Library Browser.
Help/Online Users Guide...

Displays an online version of this manual. For more information, see Online Users Guide on page 41.
Help/About ...
Opens a dialog that lists the releases of software that are currently running.
Include menu
The Include menu contains the following commands:
462 Users Guide
Acquire Lock Close Close All Open Open All Release Lock Save
27 MENU COMMANDS Include menu
Include/Acquire Lock
When the cursor is positioned on an open, read-only testplan or subplan, locks the subplan. The gray bar changes from gray to yellow, indicating that you have read-write access to the file.
Include/Close
When the cursor is positioned on an open subplan in a testplan, closes the subplan.
Include/Close All
Closes all open subplans in the testplan.
Include/Open
When the cursor is positioned on an include statement in a testplan, opens the subplan in the testplan.
Include/Open All
Opens all subplans (indicated by include statements) within the active testplan.
Include/Release Lock
When the cursor is positioned on a open, locked subplan or testplan, unlocks the subplan or testplan. The margin bar for the testplan or subplan turns from yellow to gray. The testplan editor prompts you to save any unsaved changes.
Users Guide
463
27 MENU COMMANDS Options menu
Include/Save
When the cursor is positioned on a subplan, writes the subplan to disk. When the cursor is positioned on a master testplan, writes the testplan, but not the subplans, to disk. Note To save the master testplan and all subplans, use File/Save.
Options menu
The Options menu contains the following commands: Agent... Class Map... Close Options Set Editor Colors... Editor Font... Enable/Disable Wizard Extensions General... Open Options Set... Property Sets... PVCS Options Recorder... Runtime... Save New Options Set...
In addition, the Options menu contains a dynamic menu item of the form n option-file-name that allows you to cause the nth options set loaded to be the current options set in effect.
Options/Agent...
Opens the Agent Options dialog, which allows you to set global options relating to these categories: timing, verification, closing windows, bitmaps, faults, compatibility, extensions, and other. Select the tab for a category to display a panel of options for that category.
464 Users Guide
The following figure shows the Timing category.
Whenever you select an option from a panel, the dialog displays the 4Test statement that represents the selected option. For example, if you select the Window Timeout (Seconds) option in the Timing category,
the following 4Test code is displayed in a field at the bottom of the dialog.
The statement uses the SetOption method to operate on the Agent object. OPT_WINDOW_TIMEOUT is the internal 4Test name of the Window Timeout option, and 5 is the current, default value of the option, five seconds. This code is useful if you want to set a local value for a given Agent. You simply place the statement in the main routine of your script, making sure to set the value you want. The 4Test Statement Which Sets the Option field displays the code. You can either type it yourself or select the Copy to Clipboard pushbutton to have SilkTest copy the statement to the clipboard.
Creating multiple sets of Agent options In some situations you might want to have more than one set of Agent options; for example, if youre working on multiple projects, you might want to have a custom set of options for each project. In this case, you need to save your options in a file other than the
Users Guide
465
default file, which is partner.ini. SilkTest saves your options automatically in partner.ini whenever you exit the dialog. To save your custom options in another file, see Options/Save New Options Set... on page 486.
Timing options
Window Timeout (Seconds) Specifies the number of seconds SilkTest waits for a window to appear and be in the correct state. If a window does not appear in the correct state within the specified timeout, SilkTest raises an exception. The correct state of the window depends on how you set the options on the Verification tab, which determine whether SilkTest checks whether a window is enabled, active, exposed, or unique. The default is 5 seconds (unless you have enabled enhanced support for Visual Basic, in which case the default is 20 seconds for 32-bit applications and 10 seconds for 16-bit applications). Window Retry Interval (Seconds) Specifies the number of seconds SilkTest waits between attempts to verify a window, if the window does not exist or is in the incorrect state. SilkTest continues trying to find the window until the time specified by the Window Timeout option is reached. The correct state of the window depends on how you set the options on the Verification tab, which determine whether or not SilkTest checks whether a window is enabled, active, exposed, or unique. The default is 0.06 seconds. Keyboard Event Delay (Seconds) Specifies the delay used before each keystroke in a script. Default is 0 seconds. Mouse Event Delay (Seconds) Specifies the delay used before each mouse event in a script. The delay affects moving the mouse, pressing buttons, and releasing buttons. Default is 0 seconds. App Ready Timeout (Seconds) Specifies the number of seconds that the Agent waits for an application to become ready. If the application is not ready within the specified timeout, SilkTest raises an exception. To require the Agent to confirm the ready state of an application, check the Verify That an Application is Ready (Requires an Extension) option on the Verification tab. This option applies only if the application or extension knows how to communicate to the Agent that it is ready. To find out whether the extension has this capability, see the documentation that comes with the extension. Default is 10 seconds.
466
Users Guide
App Ready Retry Interval (Seconds) Specifies the number of seconds SilkTest waits between attempts to verify that an application is ready. SilkTest continues to test the application for readiness if it is not ready until the time specified by the App Ready Timeout option is reached. Default is 0.1 seconds.
Verification options
Verify That Windows Are Active If checked, verifies that windows are active before interacting with them. See the topic on active and enabled status in SilkTest online help for information about how this option affects various SilkTest methods. Default is checked. Verify That Windows Are Enabled If checked, verifies that windows are enabled before interacting with them. See the topic on active and enabled status in SilkTest online help for information about how this option affects various SilkTest methods. Default is checked. Verify That Windows Are Exposed If checked, verifies that windows are exposed (that is, not covered, obscured, or logically hidden by another window) before interacting with them. Default is checked. Verify That a Tag Uniquely Identifies a Window If checked, verifies that each tag used by the script matches only one window. Default is checked. Verify That Coordinates Passed to a Method Are Inside the Window If checked, verifies that the coordinates passed to a method are inside the window before the mouse is pressed. If checked and coordinates fall outside the window, SilkTest raises the E_COORD_OUTSIDE_WINDOW exception. Typically, you use the checking feature unless you need to be able to pass coordinates outside of the window (such as negative coordinates). Note that the MoveMouse, PressMouse, and ReleaseMouse methods never verify their coordinates. Default is checked. Verify the Class Tag for Methods of Class Control If checked, verifies that objects are of the specified type before interacting with them. This option is unchecked and disabled for SilkTest.
Users Guide
467
Verify That an Application Is Ready (Requires an Extension) If checked, synchronizes the Agent with the application under test. Calls made to the Agent will not proceed until the application is ready. This option applies only if you have an extension enabled in the Options/Extensions dialog. Default is checked.
Close options
List of Buttons Used to Close a Window Specifies a list of strings representing the list of buttons used to close windows closed with the Close, CloseWindows, and Exit methods. The preferred way to specify these buttons is with the lsCloseWindowButtons variable in the objects declaration. Default is Cancel, Close, Exit, Done. Keystrokes Used to Close a Dialog Box Window Specifies a list of strings representing the keystroke sequence used to close dialogs (used by Close, CloseWindows, and Exit). The preferred way to specify this keystroke sequence is with the lsCloseDialogKeys variable in the objects declaration. Default is <Esc>. List of Menus Used to Close a Window Specifies a list of strings representing the list of menu items used to close windows with Close, CloseWindows, and Exit. The preferred way to specify these menu items is with the lsCloseWindowMenus variable in the objects declaration. Default is File/Exit*, File/Quit*. List of Buttons Used to Close a Confirmation Window Specifies a list of strings representing the list of buttons used to close confirmation dialogs (that is, dialogs or message boxes that appear when closing windows with the Close, CloseWindows, and Exit methods). The preferred way to specify these buttons is with the lsCloseConfirmButtons variable in the objects declaration. Default is NO. Name of Close Item On System Menu Specifies a list of strings representing the list of menu items on the system menu used to close windows with the Close, CloseWindows, and Exit methods. Default is Close.
Bitmap options
Bitmap Match Count Specifies an integer representing the number of consecutive snapshots that must be the same for the bitmap to be considered stable. Snapshots are taken up to the number of seconds specified by the
468
Users Guide
Bitmap Match Timeout option, with a pause specified by Bitmap Match Interval occurring between each snapshot. This option affects the CaptureBitmap, GetBitmapCRC, VerifyBitmap, and WaitBitmap methods. Default is 5. Bitmap Match Interval (Seconds) Specifies a number representing the time interval between snapshots to use for ensuring the stability of the bitmap image. The snapshots are taken up to the time specified by the Bitmap Match Timeout option. This option affects the CaptureBitmap, GetBitmapCRC, VerifyBitmap, and WaitBitmap methods. Default is 0.1 seconds. Bitmap Match Timeout (Seconds) Specifies a number representing the total time allowed for a bitmap image to become stable. During the time period, SilkTest takes multiple snapshots of the image, waiting the number of seconds specified by the Bitmap Match Interval option between snapshots. If the timeout period is reached before the number of bitmaps specified by the match count option, SilkTest stops taking snapshots and raises the exception E_BITMAP_NOT_STABLE. This option affects the CaptureBitmap, GetBitmapCRC, VerifyBitmap, and WaitBitmap methods. Default is 5 seconds. Bitmap Compare Tolerance (Pixels) Specifies an integer representing the number of pixels of difference below which two bitmaps are considered to match. If the number of pixels that are different is smaller than the number specified with this option, the bitmaps are considered identical. The maximum tolerance is 32,767 pixels. The bitmap pixel tolerance is used by the VerifyBitmap and WaitBitmap methods, as well as the SYS_ CompareBitmap function. Default is 0 pixels.
Compatibility options
Use these options only if you are upgrading from a previous release of SilkTest and want to use functionality that matches the previous release. Don't Group Radio Buttons into a List If unchecked, treats radio buttons as a group of class RadioList. Otherwise, uses the Release 1.x method of treating radio buttons as individual objects. Default is unchecked. Use Release 1.x Version of the Class Library If checked, uses QA Partner Release 1 versions of GetChildren, GetClass, and GetParent. Otherwise uses Release 2 and later versions.
Users Guide 469
Default is unchecked. Use Release 1.x Window Tags If checked, generates and operates on tags compatible with SilkTest releases earlier than QA Partner Release 2. Otherwise, uses Release 2 or later tags. Default is unchecked. The Release 2 algorithm affects tags that use index numbers and some tags that use captions. In general, Release 2 tags are more portable, while the earlier algorithm generates more platform-dependent tags. Use this option only if you must run old scripts without any changes. Don't Trim Spaces When Getting Items in Controls If checked, leading and trailing spaces are not trimmed from items on windows. Default is unchecked. Add Window Tags to the Record Menu If checked, includes the Record Window Tags menu item on the Record menu. Selecting the Record Window Tags menu item opens the Record Window Tags dialog. This dialog is equivalent to the QA Partner Release 1.x Paste Window dialog, allowing you to paste Release 1.x style window tags into your script. Default is unchecked.
Other options
Tolerance to Use When Sizing Windows (Pixels) Specifies an integer representing the number of pixels allowed for a tolerance when a resized window does not end at the specified size. For some windows and GUIs, you cant always resize the window to the particular size specified. If the ending size is not exactly what was specified and the difference between the expected and actual sizes is greater than the tolerance, SilkTest raises an exception. Windows cannot be sized smaller than will fit comfortably with the title bar. Default is 0 pixels. Tolerance to Use When Moving Windows (Pixels) Specifies an integer representing the number of pixels allowed for a tolerance when a moved window does not end up at the specified position. For some windows and GUIs, you cant always move the window to the particular pixel specified. If the ending position is not exactly what was specified and the difference between the expected and actual positions is greater than the tolerance, SilkTest raises an exception.
470
Users Guide
The tolerance can be set through the Control Panel, by setting the desktop window granularity option. If the granularity is zero, you can place a window at any pixel location. If the granularity is greater than zero, the desktop is split into a grid of the specified pixels in width, determining where a window can be placed. In general, the tolerance should be greater than or equal to the granularity. Default is 0 pixels. Events Used to Invoke Popup Menus Specifies a string, which is the command (keystrokes or mouse buttons) used to display pop-up menus. To use mouse buttons, specify <button1>, <button2>, or <button3> in the command sequence. Default is <Button2><Up><Down>. Pick Menus Before Getting Menu Item Information If checked, picks the menu before checking whether an item on it exists, is enabled, or is checked. Note: You might see menus pop up on the screen even though your script does not explicitly call the Pick method. Default is unchecked. Pick Dropdowns Before Getting Item Information If checked, drops down the combo box before trying to access the combo boxs contents. This is usually not needed, but some combo boxes only get populated after they are dropped down. If you are having problems getting the contents of a combo box, check this option. Default is unchecked Consider Case When Matching Items in Controls Check this option to consider case when looking for an item in a control. Default is unchecked. Show Windows Which Are out of View If checked, allows controls which are not currently scrolled into view to be seen by the Agent. This option is useful for testing Web applications. If unchecked, controls not currently in view are invisible. Default is checked. Automatically Scroll Window into View If checked, scrolls a control into view before recording events against it or capturing its bitmap. This option is useful for testing Web applications. This option applies only if Show Windows Which Are out of View is enabled. This option is useful for testing Web applications in which a dialog contains a scroll bar.
Users Guide 471
Default is checked. Warning Segue strongly recommends that you keep the default settings for Show Windows Which Are out of View and Automatically Scroll Window into View. If you do change the values, save them in a custom options set. Enable communication with SilkBean If checked, enables communication with the SilkBean for all scripts in a test session. The SilkBean is a utility that allows you to perform cross-platform testing of 100% pure Java controls in standalone Java applications. For more information about the SilkBean, see the online Help on Testing Java Applications with the SilkBean. Default is unchecked.
Options/Extensions...
Opens the Extensions dialog, which you use to enable extensions and fault trapping for applications under test on your host machine. When you first open the Extensions dialog, it displays the runtime environments for all extensions that you have installed or that come with SilkTest. The environments listed in the Extensions dialog are used for running the applications you wish to test. The environments that you will use for testing applications appear in the Application column. The browser and Java runtime environments appear because browser support and Java support come with SilkTest. By default, extensions for testing applications in each runtime environment are disabled.
Application
Displays the browser and runtime environments that the installation program installed and recognizes automatically. If you installed all development environments, the installation program automatically adds the following runtime environments to the Extensions dialog. For runtime environments that do not use standard names or for stand-alone applications, you must add the name of the executable or DLL file to the dialog yourself. For more information, see Chapter 3, Enabling Extensions for Applications Under Test.
Primary Extension
The Primary Extension field displays Enabled, Disabled, or None (for not applicable), or names the primary extension to use for applications launched in the runtime environment listed in the Application column. Click on an entry in the field to activate a drop-down list box. Then click on the down arrow to see the choices.
472
Users Guide
27 MENU COMMANDS Options menu Other Extensions
Check the boxes that represent the other extensions that are required for testing the application that is launched in the runtime environment listed in the Application column. Java To enable Java support for an application running on the host machine, check the Java check box. When Java is enabled, you may also click the Java pushbutton to select Java options. See Options later in this section for a description of Java options. ActiveX To enable ActiveX and Visual Basic 5 support for an application running on the host machine, select the ActiveX check box. When ActiveX is enabled, you may also click the ActiveX pushbutton to select specific ActiveX options (none available as of this writing). Fault Trap To enable fault trapping for an application on the host machine, check the Fault Trap check box. You may also click the Fault Trap pushbutton to turn on/off specific fault trapping options. For more information, see Enabling fault trapping on page 217.
Options
The pushbuttons in this group display options for different extensions. Extension The Extension pushbutton opens the Extension Options dialog. The option displayed in the dialog depends on the extension type you selected.
If you selected You can
Netscape 4 or an application based on Netscape 4 Internet Explorer 4 or 5 or an application based on Internet Explorer 4 or 5
Enable/disable JavaScript or enable/ disable the Java plug-in. See Testing Web Applications with SilkTest. Enable/disable browser-specific image names or enable/disable the Java plug-in. See Testing Web Applications with SilkTest.
Java Active only when Java support is enabled, the Java pushbutton opens the Extension Options dialog. Select Redirect Java Console Output and enter a file name to redirect output from the Java console to a local file where you can more easily scroll and copy the text. For more information, see the online Help on Testing Java Applications and Applets. ActiveX Active only when ActiveX support is enabled (ActiveX is checked), the ActiveX pushbutton opens the Extension Options dialog. See online Help for information on the option, Ignore VB and SS Frames.
Users Guide
473
Fault trap Fault options determine the actions taken by SilkTest when a general protection fault is encountered. Active only when fault trapping is enabled (the Fault Trap check box is selected), the Fault Trap pushbutton opens the Fault Trap Options dialog. The first four options, Fault Actions, set SilkTests behavior when a fault occurs, as described in the following table:
Fault Action Description Default
Log Fault to Results File Log Fault to faultrap.log Capture Desktop Bitmap
If checked, logs faults information to the checked results file. If checked, saves faults information in the file faultrap.log in the SilkTest installation directory. If checked, captures a bitmap of the desktop when the fault occurs and saves it in the file faultrap.bmp in the SilkTest installation directory. If checked, chains to the next fault handler installed on the system, for example, passes the fault to a debugger. Otherwise, SilkTest logs the fault, raises an exception, shuts down the offending application, and continues. Not applicable to 32-bit environments. checked
unchecked
Chain to Next Handler
unchecked
The next five options, Fault Log Information, relate to logging fault information, as described in the following table:
Fault Log Information Description Default
Call Stack Registers Memory Information System Information Task List
If checked, logs the current calling stack. checked Not applicable to 32-bit environments. If checked, logs registers. If checked, logs memory management information. If checked, logs information about the system. If checked, logs a list of all programs currently running. checked checked checked checked
474
Users Guide
27 MENU COMMANDS Options menu New
Click the New pushbutton to add one or more runtime environments to the dialog for testing applications. For more information, see Chapter 3, Enabling Extensions for Applications Under Test. Active only when you select a runtime environment in the Application column that you entered manually. (You cannot duplicate runtime environments that are installed by default.) Click the Duplicate pushbutton to add a new application that has all the same settings (primary and other extensions, fault trapping, and other options) as the selected application, but with a different name. For more information, see Chapter 3, Enabling Extensions for Applications Under Test. Active only when you select a runtime environment in the Application column that you entered manually. (You cannot remove runtime environments that are installed by default.) Click the Remove pushbutton to delete an runtime environment from the dialog. For more information, see Chapter 3, Enabling Extensions for Applications Under Test. Active only when you select a runtime environment with an enabled Primary Extension. Click the Details pushbutton to see detailed information about the runtime environment, including version, help text, and executable module(s).
Duplicate
Remove
Details
Options/Class Map...
Opens the Class Map dialog, which you use to map custom classes to standard classes supported by SilkTest. By establishing a class mapping, you can use the methods and properties of existing standard classes on the appropriate custom classes. You can also display the Class Map dialog while you are recording declarations by clicking the Class Map pushbutton in the Record Window Declarations dialog. Note For more information about mapping custom classes to standard classes, see Mapping custom classes to standard classes on page 94.
Options/Close Options Set

Available only when a set of custom Agent, runtime, and class map options is in effect. Deactivates the current options set. The file name disappears from the title bar. The default options are now in effect.
Users Guide
475
Options/Editor Colors...
Opens the Editor Colors dialog, shown in the following illustration, which you use to set the screen colors for various elements of 4Test code, results information, and the testplan (if available). You can select any of the screen colors provided or create your own by modifying the RGB values of these colors. Modify the values by using the slider bar to change the relative value of red, green, or blue, or enter specific values in the field next to each color. When you are satisfied, click OK. If you want to revert to the default colors, click Reset.
Editor Item Select an item from the list box. You can set colors for the following categories: 4Test code Results information Testplans Difference Viewer
Editor Item Color Select one of the 16 colors from the palette for the selected editor item. If you are satisfied with this color, click OK. Otherwise, you can modify the RGB values of the selected colors by sliding the bar to the left or right until you get the color you want or by entering specific RGB values.
Options/Editor Font...
Opens the Editor Font dialog, shown below, which allows you to select a screen font in the family, size, and style of your choice. As you make your selections, the Sample field is updated.
476
Users Guide
When you are satisfied, click OK. SilkTest changes the font family, size, and style for all open windows.
Font Select a font family from the available fonts displayed in the list box. The default is Courier. Font Style Select a style from those that are available for the selected font, for example, regular, italic, bold, and bold italic. Default is regular. Size Select a point size from those that are available for the selected font. Default is 9. Sample This read-only area displays a line of text in the font family, style, and point size currently selected and changes dynamically as you make selections from the list boxes in the dialog.
Options/Enable/Disable Wizard
Associated with the testplan editor. Enables or disables the QuickStart Wizard, which guides you through the creation of test frames, testplans, and testcases. For more information, see Getting started on page 19.
Options/General...
Opens the General Options dialog, which you use to set such aspects of general system behavior as the editor and your workspace.
Users Guide
477
.
Editor
Save Files Before Running If checked, writes all open modified files to disk before executing a script, suite, or testplan. If unchecked, only writes the individual script, suite, or testplan to disk before running it if it has been modified. Default is checked. Create Backups If checked, SilkTest creates a backup file each time you save a file. The backup file has an underscore ( _ ) appended to the extension. Default is checked. Visual 4Test by Default If checked, SilkTest starts the outline editor when you open a script or include file. Otherwise, SilkTest opens the file in the plain editor. Default is checked. Width of Tabs Specifies the number of spaces in a tab stop. Default is 4 spaces.
Workspace
Show Toolbar If checked, SilkTest displays the tool bar. Default is checked. Show Toolbar Tips If checked, SilkTest displays on-screen descriptions of toolbar buttons (ToolTips) when you place the mouse pointer over the buttons. SilkTest uses the standard Windows ToolTips display mechanism. You can customize the appearance of the ToolTips using the standard Windows Display Properties dialog. Default is checked. File History Size Specifies the number of recent file actions, from 0 to 9, to display on the File menu. See File/n operation file-name on page 458. Default is 4.
478
Users Guide
27 MENU COMMANDS Options menu 4Test
Include File Extensions Specifies the default extensions that SilkTest supports for include files. Files with other extensions will be treated like text files. They are displayed in the plain editor only. Defaults are inc and lib.
Testplan
Data File for Attributes and Queries Specifies the default path for the testplan initialization file. The file extension is arbitrary. Available only if you have installed the testplan editor. Default is testplan.ini.
Help
Help Files for Library Browser Specifies the files that SilkTest uses to compile its help file (4test.hlp) for the Library Browser. Separate all entries with commas. Default is 4test.txt, which includes information for all the built-in classes and functions. For more information about the Library Browser, and how to update its contents with information about your own classes and functions, see Chapter 18, Modifying the Library Browser.
Options/n option-file-name
Displays a list of from 1 to n file names. Each is a custom options file that you have loaded during the current work session. File names are prefaced by an integer, n, where the last-loaded options file is listed first. You can click on a file name to have a new set of options in effect. Alternatively, you can select a file name by its number; for example, to load the options in the second file in the list, press Alt+O+2.
Options/Open Options Set...

Opens the Open Options Set dialog, a standard file-opening dialog that you can use to load the set of custom Agent, runtime, and class map options and Library Browser Help files that you want to be in effect for the current suite, script, or testplan. After you load a set of options, SilkTests title bar displays the name of the options file, which has an .opt file extension. Example: If you create a custom options set when Java support is enabled, then the next time you open the options set, the Java Help file will be loaded automatically into the Library Browser.
Users Guide
479
Options/Property Sets
Opens the Property Sets dialog, which allows you to create, modify, combine, and delete property sets, which are used to verify properties in testcases. For more information, see Configuring your own property sets on page 121.
Options/PVCS Options
Available if PVCS is installed only. Displays cascading menu that allows you to manage your SilkTest test files with PVCS. For more information, see Chapter 25, Using PVCS with SilkTest.
Options/Recorder
Opens the Recorder Options dialog, shown below, which allows you to specify settings that SilkTest uses when recording.
General Options
Change Hotkey To Ctrl+Shift If checked, SilkTest waits for you to press Ctrl+Shift for verifications, instead of the default Ctrl+Alt. Default is unchecked. Record Multiple Tags If checked, SilkTest records multiple tags whenever recording. SilkTest records the tags that are specified in the Record Window Declarations Options dialog. For more information, see About tags on page 84. Default is checked.
Verification
Verify Using Properties If checked, verify test application using properties (instead of attributes). Default is checked.
480
Users Guide
Note This option is checked automatically if you have enabled enhanced support for Visual Basic; these features both require properties for verification. You cannot uncheck the Verify Using Properties option without disabling enhanced support for Visual Basic. Data File for Property Sets Specifies the file containing the definitions for the property sets being used. For more information about verification using properties and property sets, see Verifying using properties on page 115.
Recorded Events
Record Only Low Level Events If checked, specifies that you want to record events at a lower level for selected controls. For example, you might want to record a click in a CheckBox, instead of recording an actual selection. If you specify that you want to record only lowlevel events in check boxes, SilkTest would record something like this if you check a check box:
Find.CaseSensitive.Click (1, 41, 10)
Normally, SilkTest would record this:

Find.CaseSensitive.Check ()
Default is unchecked for all controls. Ignore Mouse Move Events If checked, SilkTest does not record mouse movements when you select the Record Testcase and Record Actions commands. If unchecked, SilkTest records mouse movements that cannot be built into higher-level actions and that occur while a mouse button is pressed. Typically, you leave this checked unless you are testing an application, such as a drawing application, where mouse movements themselves are significant. Default is checked. Dont Record BeginDrag/EndDrag If checked, SilkTest does not record BeginDrag and EndDrag methods when you press a mouse button on an object and do a drag operation on a listview, treeview, or list box. Default is unchecked.
Users Guide
481
Options/Runtime...
Opens the Runtime Options dialog, shown below, which allows you to specify settings that SilkTest uses when it runs a script.
4Test
Agent Name Specifies the name of the Agent on the target machine, that is, the machine on which you want to run a script or suite. Specify a value only if you want to run your script on a machine other than the host machine. Note If you dont know the Agent name, click on the Agent icon on the target machine. Network Specifies the networking protocol to use when testing applications across a network. Arguments Specifies the arguments, if any, that you want to pass to the script at runtime. Separate multiple arguments with spaces. For example, suppose your script takes two arguments: the number of iterations to perform and the name of a test data file. In this case, you would enter 5 test1.dat For more information, see Passing arguments to a script on page 147.
482
Users Guide
Use Path Specifies one or more paths along which SilkTest searches for include files. Include files can be named in the Use Files field or in a scripts use statement. If you specify a path, SilkTest searches the current directory and then each of the directories in the path named here. If you do not specify a search path, SilkTest searches the current directory only. Type one or more paths, separated by commas. The syntax for a path is the same as that used by the native operating system. Use Files Specifies the names of one or more include files for SilkTest to automatically load at startup. Do not specify files in this field if you intend to include a use statement for the files in a script. Type the file name(s) as you would any other operating system path. Separate file names with a comma. Tip You can also click the Add button and select an include file to add from a standard file selection dialog. Objfile Path Specifies the location where SilkTest reads and writes object files from. Leave the field empty if you want to store object files in the same directory as their corresponding source files Specify an absolute path if you want to store all object files in the same directory Specify a relative path if you want object files to be stored in a directory relative to the directory containing the source files
For more information, see Using object files on page 134. Gui Targets Specifies the platforms that you want to compile your scripts and include files for (using conditional compilation). You can specify as many GUI targets as you want; separate each GUI specifier by a comma. For more information, see Conditional compilation on page 271. Default Browser Specifies the Web browser to use when testing Web applications. Save Object Files During Compilation If checked, creates an object file from a script or include file when it is compiled. (An object file is always created for a script or include file when it is saved.) For more information, see Using object files on page 134. Enable Y2K Date Transformation Rules If checked, SilkTest will transform dates as specified in transformation rules you define.
Users Guide 483
For more information, see Testing applications that have been made Year 2000 compliant on page 195. Compiler constants... Opens the Compiler Constants dialog, where you can define constants and assign them values. You can use the defined constants in scripts and include files anywhere you can specify an expression. Constants are evaluated and replaced with their values at compile time. To define a constant, specify its name in the Constant Name field and its value in the Value field, then click Add. You can edit or delete an existing constant by selecting it and clicking Edit or Remove. Y2K Rules... Opens the Y2K Date Transformation Rules dialog where you can define rules that SilkTest should use to transform dates. For more information, see Testing applications that have been made Year 2000 compliant on page 195.
Results
Directory/File Specifies the name of the directory, or the name of the directory and file that stores the results of script runs. Note SilkTest always assigns the extension .res to all results files. If you supply a different extension, SilkTest will override it. If you leave the field empty (the default), SilkTest gives the results file the same name as the script and stores it in the same directory as the script. If you supply only a directory name, SilkTest gives the results file the same name as the script and stores it in the directory you specify. History Size Specifies an integer representing how many sets of results to keep for a script. Once this number is reached, SilkTest automatically deletes the oldest set of results each time it generates new results. A value of 0 saves all results files. Default is 5. Write to Disk After Each Line If checked, writes the results file to disk whenever the script generates output, as in the case of a print statement. Selecting this option ensures that in the event of system failure, SilkTest will produce a results file containing output up to the time of system failure. The disadvantage of selecting this option is that file I/O slows down script execution. Default is checked.
484
Users Guide
Find Error Stops at Warning If checked, Edit/Find Error locates error messages and warnings in results files. Otherwise, the command locates error messages only. Default is checked. Show Overall Summary If checked, displays the summary of results for the entire script, suite, or testplan, including the start and elapsed time of execution, and the total number of errors and warnings. For more information on how this option is used, see Results/Show Summary on page 506 and Results/Hide Summary on page 504. Default is checked. Log Elapsed Time, Thread, and Machine For Each Output Line If checked, records this information in the results file for each line that is written. Default is unchecked. Note To view this information, make the results file active, then select Results/View Options and check Elapsed time, Thread number, and Current machine.
Execution
Minimize While Running If checked, SilkTest runs minimized while you run a script, suite, or testplan. Default is unchecked. Show Detailed Status Window If checked, SilkTest displays the Runtime Status window while you are running a script or suite when SilkTest is not minimized. Default is checked. Save Status Window Position If checked, SilkTest remembers the position of the Runtime Status window if you move it during script execution. The next time you run a script, the Runtime Status window will display in the new position. Default is unchecked, in which case the Runtime Status window always comes up at the same location.
Debugging
Print Agent Calls If checked, specifies whether or not you want the results file for each test run to include a list of all method calls made by your script. Each entry includes the method name and the arguments passed to it. This is a useful feature for debugging because it tells you exactly which methods were actually called by the running program.
Users Guide
485
27 MENU COMMANDS Outline menu
Default is unchecked. Print Tags With Agent Calls If checked and Print Agent Calls is also checked, includes tags with the method calls in your results files. Default is unchecked.
Options/Save New Options Set...

Opens the Save Options Set As dialog, which you use to save the current set of Agent, runtime, and class map options and Library Browser Help files. The options set file has an .opt file extension. After you click OK, the options in this file are in effect. SilkTests title bar displays the name of the options file.
Outline menu
The Outline menu contains the following commands: Collapse Collapse All Comment Block Expand Expand All Move Left Move Right Transpose Down Transpose Up Uncomment Block
Outline/Collapse
When the current line or selected line(s) is preceded by a minus sign (-), hides subitems.
486
Users Guide
27 MENU COMMANDS Outline menu
Outline/Collapse All
Hides all subitems in the window.
Outline/Comment Block
Available when the active window is a 4Test script, include file, or plan. Places a single-line comment (indicated by //) at the beginning of the current line or selected lines.
Outline/Expand
When a plus sign (+) precedes the current line or selected lines, displays the hidden subitems (children) of the current line or selected lines.
Outline/Expand All
Displays all subitems in the window.
Outline/Move Left
Moves the current line, selected lines, or the text cursor (if line is empty) to the left. Repeat to achieve the desired outdent. If you shift the code to an invalid position in a script or include file, SilkTest marks the invalid line(s) with an X.
Outline/Move Right
Moves the current line, selected lines, or the text cursor (if line is empty) to the right. Repeat to achieve the desired indent. If you shift the code to an invalid position in a script or include file, SilkTest marks the invalid lines with an X.
Users Guide
487
27 MENU COMMANDS Record menu
Outline/Transpose Down
Exchanges the current line or selected lines with the next line. Adjust the indent or outdent using Edit/Move Right and Edit/Move Left.
Outline/Transpose Up
Exchanges the current line or selected lines with the previous line. Adjust the indent or outdent using Edit/Move Right and Edit/Move Left.
Outline/Uncomment Block
Available when the active window is a 4Test script, include file, or plan. Removes a single-line comment (indicated by //) from the current line or selected lines.
Record menu
The Record menu contains the following commands: Actions... Application State... Class... Method... Testcase... Window Declarations... Window Identifiers... Window Locations...
Record/Actions...
Opens the Record Actions dialog, shown below, which records the actions you perform to test an application, translates them into 4Test statements, and displays them in the Recorded Actions window. In addition, you can add a 4Test verification statement or method for a GUI object.
488 Users Guide
Typically, you then insert (paste) the statements into a script file at the cursor position, where you can edit them and write additional statements. If no file is open, SilkTest inserts the actions into an untitled script window.
Recorded Actions
The Recorded Actions list box displays your interactions with your application as 4Test statements. Furthermore, the list box provides all the editing capabilities of the SilkTest editor, so you can manipulate any actions you record. 4Test statements have this format: fully-qualified-object-name.method. After recording one or more actions, you might want to Add a verification statement or a method for a GUI object by positioning the mouse cursor over the GUI object and pressing Ctrl+Alt. Halt recording to bring your application to another state before continuing to record actions. (This avoids recording of extraneous statements.) Press the Pause Recording pushbutton. Insert the statements in the Recorded Actions list box into a script file. Press Paste to Editor, or Copy to Clipboard and related commands. Delete the statements from the list box, for example, because you recorded the wrong actions. Press Clear All.

Keep on Top
If checked, keeps the Record Actions dialog in the foreground even when you are tracking the cursor over your application. Default is checked.
Pause Recording/ Resume Recording
Pause Recording/Resume Recording is a toggle pushbutton that suspends or continues the recording of your interactions with your application. Typically, you use Pause Recording to halt recording while you bring your application to the correct state before beginning recording again. Typically, you use Resume Recording after pasting recorded actions into the editor. To add a verification statement or method for an object, position the mouse cursor over the object and press Ctrl+Alt. (Press Ctrl+Shift if you changed the hotkey in Recorder Options.)
Ctrl+Alt (Ctrl+Shift)
Users Guide
489
For more information, see Recording the verification stage on page 112.
Paste to Editor
The Paste to Editor pushbutton inserts the 4Test code in the Recorded Actions list box into the active SilkTest editing window. The Record Actions dialog is cleared. Each time you select Paste to Editor, SilkTest inserts the recorded actions at the end of the same file. Note If you use an editor other than that provided with SilkTest to write your test scripts, use Copy to Clipboard to insert actions rather than Paste to Editor.
Copy to Clipboard
The Copy to Clipboard pushbutton copies the recorded actions in the list box to the clipboard. The Record Actions dialog is cleared. Use Edit/Paste to insert the actions into an editing window of your choice. Empties the Recorded Actions window. Closes the Record Actions dialog and redisplays the SilkTest editing window. If you have not pasted your recorded actions to the editor, SilkTest prompts you to do so.
Clear All Close
Record/Application State...
Opens the Record Application State dialog, which allows you to define an application state routine that SilkTest runs before it executes your testcase. If the current window is not a script or include file, SilkTest prompts you to create a new include file.
Application State Name Specifies the name of the application state you are going to record. The application state name must be less than 64 characters.
490
Users Guide
Based On Specifies an application state upon which this application state is based. Select an application state from the Based On drop-down list. The default is the last application state you specified in this field. Paste to Editor Closes the Record Application State dialog and places the 4Test code, if any, in the associated file. Cancel Closes the Record Application State dialog without placing any code in the associated file. Start Recording Closes the Record Application State dialog, displays a message box indicating that the base state is being set, and displays the Record Status window.
The presence of the Record Status window indicates that you can begin recording an application state. The Status field flashes the word Recording. In addition, the Record Status window lets you Temporarily pause recording, for example, to leave your desk or begin another task, and then resume recording. Press Pause Recording and then Resume Recording in the Record Status window. Add verification statements to your application state routine. Position the mouse pointer over the GUI object you want to verify and press Ctrl+Alt. The Record Status window shows the object that the mouse pointer is over. See what youve recorded and then resume recording. Press Pause Recording to review the Application State Code field in the dialog and then press Resume Recording in the dialog. End recording. Press Done.
Pause Recording/Resume Recording is a toggle pushbutton that suspends or continues the recording of your interactions with your application. Typically, you use Pause Recording to halt recording while you bring your application to the correct state before beginning recording again. Typically, you use Resume Recording after pasting your recording into the editor.
Users Guide
491
27 MENU COMMANDS Record menu Ctrl+Alt
To add a verification statement or method, position the mouse cursor over the object and press Ctrl+Alt. (Press Ctrl+Shift if you changed the hotkey in Recorder Options.) For more information, see Recording the verification stage on page 112.
Done
When you have finished recording an application state or just want to see what you have recorded, press the Done pushbutton on the Record Status window. SilkTest redisplays the Record Application State dialog. The Application State Code field contains all the 4Test code youve just recorded. You can take the following actions:
If Then
All the information in the window is complete and what you want. The code is not what you want.
Click OK. SilkTest closes the Record Application State dialog and places the new application state in your file. Edit the code in the Application State Code field.
The application state name is not Edit the name in the Application State Name the one you want. field. The application state on which this application state is based is not the one you want. The application state routine is not yet finished. Delete the code in the Application State field, select a new application state from the drop-down list, and click Resume Recording to rerecord the application state. Click Resume Recording. SilkTest opens the Record Status window. You can continue to record your application state.
Record/Class...
Records a new Visual Basic, ActiveX, or Java class declaration. For more information, see the online Help.
Record/Method...
Available only when an include file or script is in the active window and the insertion point is within a class or object declaration (methods are necessarily part of a class or object definition). Opens the Record Method dialog, which allows you to record a method for a class or window declaration.
492
Users Guide
Method Name Specifies the name of the method. You can add a new method name or select a name from the drop-down list: BaseState, Close, Invoke, or Dismiss. Paste to Editor Closes the Record Method dialog and places the method in the associated include file at the cursor position. Cancel Closes the Record Method dialog without placing any code in the associated include file. Start Recording Closes the Record Method dialog and displays the Record Status window. For more information about this dialog, see Record/ Application State... on page 490.
Pause Recording/Resume Recording is a toggle pushbutton that suspends or continues the recording of your interactions with your application. Typically, you use Pause Recording to halt recording while you bring your application to the correct state before beginning recording again. Typically, you use Resume Recording after pasting your recording into the editor. To add a verification statement or method, position the mouse cursor over the object and press Ctrl+Alt. For more information, see Recording the verification stage on page 112.
Ctrl+Alt
Done
When you have finished recording a method or just want to see what you have recorded, click Done in the Record Status window. SilkTest redisplays the Record Method dialog. The Method Code field contains all the 4Test code youve just recorded. You can take the following actions:
If Then
All the information in the window is correct and complete.
Click OK. SilkTest closes the Record Method dialog and places the new method in your include file.
Users Guide
493
If
Then
The code is not what you want.
Edit the code in the Method Code field.
The method name is not what you Edit the name in the Method Name field. want. The method is not yet finished. Click Resume Recording. SilkTest opens the Record Status window. You can continue to record your method.
Record/Testcase...
Compiles the currently active script file or testplan, if not already compiled, and opens the Record Testcase dialog. If the current window is not a script, SilkTest prompts you to open a script file or create a new script file before opening the dialog.
The Record Testcase dialog allows you to record an entire testcase, specifying the application state of your choice and including verification statements. You can also control whether or not SilkTest displays a status window when driving the application to the specified base state. If this status window obscures critical controls in your application, you can suppress it by unchecking the Show AppState status window check box. For complete information about recording testcases, see Chapter 5, Designing and Recording Testcases.
494
Users Guide
Record/Window Declarations...
Opens the Record Window Declarations dialog, which allows you to record descriptions, called window declarations, of the GUI objects in your application and insert them into a declarations file, called an include file. SilkTest uses the declarations in the include file to identify the objects named in your test scripts. The declaration is a combination of class, identifier, and tag. The Record Window Declarations dialog, shown below, is composed of a Window Declaration list box, the Window Detail group box, the Tag Information group box, a check box and several pushbuttons. As you move the mouse pointer over your application, the contents of the dialog change to reflect the object under the pointer. When you position the mouse pointer over the object you want to declare and press Ctrl+Alt, the Record Window Declarations dialog freezes the current contents, in effect, taking a snapshot of the declaration. The following figure shows a snapshot of the declaration for the Text Editor.
Window Detail
The Window Detail group box (upper left) displays the class, identifier, and tag of the object highlighted in the list box (bottom of the dialog). You can edit the class and identifier of an object displayed in the list box by selecting it. The Window Detail group box contains the following fields: Class The class is the GUI object type and determines which methods can operate on the object.
Users Guide
495
SilkTest assigns the class CustomWin to all custom objects. If the custom object can be mapped to a standard SilkTest class, you should create the mapping before you paste (insert) the declarations into an include file. Click the Class Map... pushbutton to display the Class Map dialog, shown below, to change the class of a custom object.
Identifier The identifier is a unique and logical, generalized platformindependent name for referring to an object. The identifier is the name you use to refer to an object in your testcase. It is arbitraryyou can use any unique identifier you want for an object. By default SilkTest constructs the identifier from the objects label or caption, removing any embedded spaces or accelerators. You can edit the identifier, for example, to make it more descriptive. The change will be reflected in the list box. To change the default identifier, click the Options... pushbutton. Tag A tag is the actual internal name SilkTest uses to refer to an object. By default SilkTest constructs the tag from the objects caption and Window ID, but it can also be based on the objects prior text, index, and location, as shown in the Tag Information group box. To include other components in the objects tag, check the appropriate check boxes in the Tag Information group box. The tag is updated in the Window Detail group box. To change how SilkTest constructs the default tag, click the Options pushbutton.
Tag Information
The Tag Information group box (upper right) displays the components that SilkTest can use to construct the tag for the object. For more information about tags, see About tags on page 84.
Window Declaration
The Window Declaration list box (across bottom of dialog) displays the declarations youve recorded. It shows the class, identifier, and tag of the recorded object and its child (subordinate) objects in the GUI hierarchy. The
496
Users Guide
declarations for child objects are embedded within the declaration of the parent (superior) object and are shown indented in the list box. To modify the declaration of an object displayed in the list box, select it. Its class, identifier, and tag are displayed in the Window Detail group box, where you can edit this information. By default, custom objects of class CustomWin are shown in red on color monitors (bold on black-and-white monitors). This highlighting serves as a visual reminder for you to map the class of custom objects to a standard class supported by SilkTest. Also, by default, custom objects of class Ignore are not included in the declarations. To change these default behaviors, click the Options pushbutton.
Resume Tracking
The Resume Tracking pushbutton continues the recording of object declarations. Typically, you use Resume Tracking after pasting declarations into the editor. When you are ready to record another declaration, press Resume Tracking. The Paste to Editor pushbutton inserts the frozen declarations in the Window Declaration list box into an untitled file if the current window is not a script or include file. The Record Window Declarations dialog is cleared. Each time you select Paste to Editor, SilkTest inserts the declarations at the end of the same file. Note Use Copy to Clipboard instead of Paste to Editor if you want to paste declarations elsewhere in the file, or if you use an editor other than that provided with SilkTest to write your test scripts.
Paste to Editor
Copy to Clipboard
The Copy to Clipboard pushbutton copies the declarations in the list box to the clipboard. The Record Window Declarations dialog is cleared. Use Edit/ Paste to insert the declarations into a different editing window or to insert them into the current window at the location of your choice. If checked, the Keep on Top check box keeps the Record Window Declarations dialog in the foreground even when you are tracking the cursor over your application. Default is checked. The Close pushbutton closes the Record Window Declarations dialog. If you have pasted declarations into a file, SilkTest displays the editing window that contains your declarations. The Options pushbutton opens the Record Window Declarations Options dialog, which lets you customize settings for the default tag, identifier, and the recording window.
Keep on Top
Close
Options
Users Guide
497
The Default MultiTags group box controls the default manner in which SilkTest constructs the tag. The default is to construct the tag from the objects Caption and Window ID. Note If you deselect Record Multiple Tags, the Default Multitags group box becomes the Default Tag group box, and the check boxes for the tag types become radio buttons, allowing you to choose only one tag component. Select one or more of the following choices (select only one if you deselected Record Multiple Tags):
Tag form Description
Caption Prior text Index
The objects caption. The closest static text above or to the right of the object. An objects index number. The index number is the order of appearance of the object, from top left to bottom right, in relation to its sibling objects of the same class. Example: If a dialog has four pushbuttons, the one closest to the top left of the dialog has the index #1 and the one closest to the bottom right has the index #4.
Window ID
The internal number used by the GUI itself to identify the object. For applications built in SQL Windows and Visual Basic, the Window ID is the internal control name of the object. The x,y location of the object. This approach is primarily useful when testing graphical controls. For more information on graphical controls, see Chapter 17, Supporting Custom Objects.
Location
498
Users Guide
The Window Declaration Identifiers group box controls the default manner in which SilkTest constructs the identifiers in the declaration.
Choose this radio button If you want the basis of the identifier to be
Use the Caption
An objects caption. Default. If there is no caption, SilkTest constructs the identifier from the class name and the index number. For example, if an object is of class PushButton and its index number is #1, SilkTest generates the identifier PushButton1.
Use the Window ID
The internal number used by the GUI itself to identify the object. Note SilkTest uses the Window ID as the basis of the identifier only if it is a textual internal control name obtained from a development tool like SQL Windows and Visual Basic.
The Window Declaration Viewer group box lets you determine how SilkTest displays and declares objects of custom classes.
Check box Description
Mark Custom Classes
Displays custom classes in red in the list box. The highlighting serves as a visual reminder to map classes to standard classes before pasting the declaration into an editor window. Default is unchecked. Includes custom objects of class Ignore in the declaration. Default is unchecked.
Show Ignored Windows
Record/Window Identifiers...
Opens the Record Window Identifiers dialog, shown below, which records a 4Test window identifier as a fully qualified GUI object name.
Users Guide
499
Window Identifier
When you place the mouse cursor over a GUI object, the Window Identifier text field displays the 4Test identifier for the object. The actual name recorded by SilkTest depends on whether or not you have previously recorded declarations for the GUI objects you want to test. If you have compiled the include file for the GUI objects, then a fully qualified object name is composed of the identifier of the object and as many of its ancestors as necessary to uniquely identify the object. Example: The fully qualified name of the Open menu item is:
TextEditor.File.Open
The ancestor of the Open menu item is the File menu. The ancestor of the File menu is the top-level GUI object, TextEditor, the application itself. If you have not previously compiled the include file for the GUI objects, then SilkTest creates a fully qualified object name on the fly (called dynamic instantiation) based on the class and tag of the object. For more information, see Recording without window declarations on page 138.
When the text field displays the identifier you want to record, you can halt tracking and insert (paste) it into a script file. Press Ctrl+Alt (Ctrl+Shift if you changed the hotkey in Recorder Options) to freeze the contents of the text field, and then press Paste to Editor, or Copy to Clipboard and related commands, to insert the identifiers into a script.
Keep on Top
If checked, always keeps the Record Window Identifiers dialog in the foreground. Default is checked.
Resume Tracking
The Resume Tracking pushbutton continues the recording of identifiers. Typically, you use Resume Tracking after pasting an recorded identifier into the editor. When you are ready to record another identifier, select Resume Tracking. The Paste to Editor pushbutton inserts the recorded identifier into the active SilkTest editing window. The identifier is inserted at the cursor position. The Record Window Identifiers dialog is cleared.
Paste to Editor
500
Users Guide
Note If you use an editor other than that provided with SilkTest to write your test scripts, use Copy to Clipboard to insert identifiers rather than Paste to Editor.
Copy to Clipboard
The Copy to Clipboard pushbutton copies the identifier in the Window Identifier list box to the clipboard. The window is cleared. Use Edit/Paste to insert the identifiers into an editing window of your choice. The Close pushbutton closes the Record Window Identifiers dialog.
Close
Record/Window Locations...
Opens the Record Window Locations dialog, shown below, which you use to record the x, y location of a graphical control, such as a tool bar. A location is recorded relative to the screen, frame, and client window.
Location
When you move the mouse cursor over the active application, the text field displays the 4Test identifier of the GUI object tracked by the mouse. The Screen, Frame, and Client radio buttons display the x, y location of the GUI object relative to the screen, frame, and client window, respectively. Select the location of your choice. Then press Ctrl+Alt (Ctrl+Shift if you changed the hotkey in Recorder Options) to halt recording. The Paste to Editor pushbutton inserts the relative location of your choice into the active SilkTest editing window. The location is inserted at the cursor position. The Record Window Locations dialog is cleared. Note If you use an editor other than that provided with SilkTest to write your test scripts, use Copy to Clipboard to insert locations rather than Paste to Editor.
Paste to Editor
Copy to Clipboard
The Copy to Clipboard pushbutton copies the location of the object named in the text field to the clipboard. The window is cleared. Use Edit/Paste to insert the location into an editing window of your choice. If checked, the Keep on Top check box keeps the Record Window Locations dialog in the foreground even when you are tracking the cursor over your application.
Users Guide 501
Keep on Top
27 MENU COMMANDS Results menu
Default is checked.
Resume Tracking
The Resume Tracking pushbutton continues the recording of locations. Typically, you use Resume Tracking after pasting a location into the editor. When you are ready to record another location, select Resume Tracking. When you are ready to record another location, select Resume Tracking. The label changes to Pause Tracking,
Close
The Close pushbutton closes the Record Window Locations dialog.
Results menu
The Results menu contains the following commands: Compact Compare Two Results... Convert to Plan... Delete... Export... Extract... Find Next Difference Goto Source Hide Summary Mark Failures in Plan Merge... Pass/Fail Report... Select... Send to Silk Radar... Show Summary Update Expected Value View Differences... View Options...
502
Users Guide
Results/Compare Two Results...

Available only when the active window is a results window for a testplan. Opens the Compare Two Results dialog, which allows you to see results that have changed from a previous run of the testplan. For more information, see Comparing different runs of a testplan on page 161.
Results/Compact
Available only when the active window is a results window. Removes unused space in a results file, thereby reducing the file size.
Results/Convert to Plan...
Available only when the active window is a results window and you have run a script, a suite, or a single testcase from a script. Opens the Convert Results to Plan dialog, which allows you to transform a results file into a testplan. For more information, see Converting the data in a results file into a plan on page 232.
Results/Delete...
Available only when the active window is a results window. Opens the Delete Results dialog. When you select a set of results and click OK, SilkTest deletes it.
Results/Export...
Available only when the active window is a results window. Opens the Export Results dialog, which you can use to export your results to a structured file that is suitable for further processing by an application such as a spreadsheet. For more information, see Exporting structured information on page 158.
Users Guide
503
Results/Extract...
Available only when the active window is a results window. Opens the Extract Results dialog, which allows you to place selected information from a results file in ASCII format into a new editor window or a file or send the information to a printer. For more information, see Storing results in an unstructured ASCII format on page 156.
Results/Goto Source
Available only when the active window is a results window. Opens a script file associated with the current results file, if it is closed, and makes it the active window. If the cursor was positioned at an error message in the results file, SilkTest positions the cursor at the error line in the script. If the cursor was positioned at the results for a particular testcase, SilkTest positions the cursor at the beginning of the testcase in the script file.
Results/Hide Summary
Available only when the active window is a results window. Hides the display of results summary of the current suite, script, testplan, or testcase. To have the summary hidden by default, see Show Overall Summary on page 485.
Results/Mark Failures in Plan

Available only when the active window is a the testplan editor results window. Marks all testcases in the testplan that generated errors during the last plan execution and makes the testplan the active window. A black stripe in the margin denotes the marked testcases. Mark Failures in Plan is useful if you want to fix errors and rerun only failed tests.
504
Users Guide
Results/Merge...
Available only when the active window is a the testplan editor results window. Opens the Merge Results dialog, which you use to combine the active results file with the results file of your choice. For more information, see Merging testplan results sets on page 160.
Results/Next Error Difference

Available only when a results file produced by running a testplan is the active window. After you issue Results/Compare Two Results, locates the next difference between two results files that is due to the pass/fail state of a test changing (skipping over differences resulting from the addition or removal of tests). For more information, see Comparing different runs of a testplan on page 161.
Results/Next Result Difference

Available only when a results file produced by running a testplan is the active window. After you issue Results/Compare Two Results, locates the next difference between two results files. For more information, see Comparing different runs of a testplan on page 161.
Results/Pass/Fail Report...
Available only when a results file produced by running a testplan is the active window. Opens the Results Pass/Fail Report dialog, which generates an onscreen report on the number and percentage of tests that have passed. For more information, see Generating a testplan Pass/Fail report on page 162.
Users Guide
505
Results/Select...
Available only when the active window is a results window. Opens the Select Results dialog, which allows you to chose which set of results to display. You can also use this dialog to add a comment to individual results sets. By default, SilkTest saves the results of five executions per script, suite, or testplan. To change the default number of results saved, edit the History Size option in the Runtime Options dialog, available by selecting Runtime on the Options menu.
Results/Send to Silk Radar...

Available only when the active window is a results window. Opens the Send Results to Silk Radar dialog, which you can use to send your results directly to Silk Radar, the Segue product that you can use to manage your applications bug reports, enhancement requests, and documentation issues. For more information, see Sending results to SilkRadar on page 158.
Results/Show Summary
Available only when the active window is a results window. Displays the results summary for the current suite, testplan, script, or testcase, including the start and elapsed time, and error totals. By default, a summary is shown for the script. To change this default behavior, see Show Overall Summary on page 485.
Results/Update Expected Value

Available only when the active window is a results window. When an error message is selected, replaces the expected value in the testcase with the actual value. Note that this command updates data within a testcase, not data passed in from the testplan. Makes the associated script file active.
506
Users Guide
27 MENU COMMANDS Run Menu
Results/View Differences...
Available only when the active window is a results window, and the current line of the results file displays a box icon preceding an error message. Opens the Difference Viewer when you click on the icon. For more information, see Finding application logic errors on page 154.
Results/View Options...
Displays a dialog that lets you specify which information you want displayed in the results window and how you want the information sorted.
Run Menu
The Run menu contains the following commands: Abort Application State... Compile Compile All Debug Run Run All Tests Run Marked Tests Show Status Testcase...
Run/Abort
Available only when a program is being run or debugged on a target machine other than the host machine. Terminates the script, suite, testcase, or testplan that is currently executing.
Users Guide
507
Run/Application State...
Opens the Run Application State dialog, which allows you to run or debug an application state defined in your test frame file or the active script file. If you click Debug, SilkTest displays the file in which the application state resides and enters debugging mode. Special debugging commands are available on the Breakpoint, Debug, and View menus. In addition, SilkTest places a breakpoint at the first line of the application state.
Run/Compile
Compiles the testplan, suite, or script and all dependent files (such as include files), if they have changed since they were last compiled. It also compiles the following if needed: the files listed in the Use Files field in the Runtime Options dialog, the compiler constants declared in the Runtime Options dialog, and the include files loaded at startup. Compile-time errors are automatically written to an error window. Compiling a suite is equivalent to compiling each script in the suite individually. Compare to Run/Compile All.
Run/Compile All
Compiles the script or suite and all dependent include files, even if they have not changed since they were last compiled. It also compiles files listed in the Use Files field in the Runtime Options dialog and the compiler constants declared in the Runtime Options dialog. Finally, it compiles the include files loaded at startup, if needed. Compile-time errors are automatically written to an error window. Compiling a suite is equivalent to compiling each script in the suite individually. Compare to Run/Compile.
508
Users Guide
Run/Debug
Available only when the file in the active window is a script. Reads the script in the active window into a debugging window and enters debugging mode. Special debugging commands are available. For more information, see Chapter 7, Using the Debugger.
Run/Run
Compiles and runs the 4Test script, suite, or testplan in the active window. Compilations errors are written to an errors window. Once the file is compiled, all runtime errors (exceptions), reports, and other results information are automatically written to the corresponding results file (filename.res), which SilkTest displays in a results window after the script, suite, or testplan terminates. SilkTest will not run a script, suite, or testplan whose file name has an extension other than .t, .s, or .pln.
About saving files
SilkTest always saves the suite, script, or testplan before running it if you made any changes to it since the last time you saved it. By default, SilkTest also saves all other open modified files whenever you run a script, suite, or testplan. To prevent this automatic saving of other open modified files, deselect the Save Files Before Running check box in the General Options dialog. See Options/General... on page 477.
Run/Run All Tests

Available only for the testplan editor. Executes all the tests in the testplan, first expanding any subplans.
Run/Run Marked Tests

Available only for the testplan editor. Executes only the marked tests in the testplan, first expanding any subplans.
Users Guide
509
27 MENU COMMANDS Testplan menu
For information on the commands for marking tests, see Results/Mark Failures in Plan on page 504, Testplan/Mark on page 513, Testplan/Mark by Query on page 514, and Testplan/Mark by Named Query on page 514. For general information on marking, see Marking a testplan on page 243.
Run/Show Status
Hides or shows the Runtime Status window when running a script, suite, testcase, or testplan on a target machine other than the host machine.
Run/Testcase...
Available only when the active window contains either a testcase or a results file in which the current line is a testcase. Opens the Run Testcase dialog, which displays all the testcases contained in the current script. You can select a testcase to run or debug. If you select Debug, SilkTest displays the file in which the testcase resides and enters debugging mode and places a breakpoint at the first line of the testcase. Note Debugging mode makes available all the debugging commands on the Breakpoint, Debug, and View menus.
Testplan menu
The Testplan menu contains the following commands: Completion Report... Define Attributes... Detail... Find Next Mark Go to Script Insert Template... Manual Tests... Mark Mark All Mark by Named Query
510
Users Guide
Mark by Query Unmark Unmark All
Testplan/Completion Report...
Opens the Testplan Completion Report dialog, which you use to generate a report on the number of complete tests. For more information, see Measuring testplan completion on page 258.
Testplan/Define Attributes...
Opens the Define Attributes dialog, which allows you to define or rename the testplan editor attributes as well as add, rename, or remove attribute values. For more information, see Chapter 11, Categorizing and Marking Testplans.
Testplan/Detail...
When the cursor is positioned at a test or group description, opens the Testplan Detail dialog. Use this command to add important information to the current test description or group description.
The dialog has a list box and a panel for the following categories: Test Execution
Users Guide
511
Test Attributes Symbols
To display a particular tab, select the appropriate tab.

List box
The list box at the top of the dialog provides context. The black arrow icon in the left margin identifies the test at the cursor position in the testplan. This is the test description you are adding details to, for example, character search in the preceding figure. Any details will apply to this test and any tests that inherit details from this test. Group descriptions that apply to this test are shown (in blue) at the top of the list box. The Test Execution tab, shown in the preceding figure, helps you specify valid script, testcase, and testdata statements. Script Specifies the full path of the script that you want to appear on the current line of the testplan. To browse for the correct script, click the Scripts pushbutton, which displays the Testplan Detail -- Scripts dialog, a standard file-selection dialog. the testplan editor writes the full path, relative path, or just the file name of the selected script into the testplan, beginning the line with the reserved word script, followed by a colon. Testcase Specifies the name of the testcase that you want to appear on the current line of the testplan. To browse for the correct testcase, press the Testcases pushbutton, which displays the Testplan Detail -- Testcase dialog that displays the testcases contained in the script specified in the Script field. The testplan editor writes the selected testcase into the testplan, beginning the line with the reserved word testcase, followed by a colon. Test Data Specifies the data values that you want to pass to the testcase. Separate each value with a comma. The testplan editor writes the data to the current line of the testplan, beginning the line with the reserved word testdata, followed by a colon. For more information on specifying data values, see Chapter 10, Adding Data to a Testplan.
Test Execution tab
Test Attributes tab
The Test Attributes tab allows you to assign attribute values to your testplan. For more information, see Chapter 11, Categorizing and Marking Testplans.
Symbols tab
The Symbols panel allows you to add, edit, or delete a symbol associated with the test. For more information, see Specifying data that is shared by multiple tests on page 226.
512
Users Guide
Testplan/Find Next Mark

Locates the next mark in the active testplan.
Testplan/Goto Script
Opens the script file associated with the current line of the testplan and makes it the active window.
Testplan/Insert Template...
Opens the Insert Testplan Template dialog, which you use to create a template, or outline, of your testplan. For more information, see Using a template of the GUI hierarchy as an outline on page 58.
Testplan/Manual Tests...
Opens the Update Manual Tests dialog, which you can use to document your manual tests and have them included in Completion and Pass/Fail reports. For more information, see Documenting manual tests in the testplan on page 65.
Testplan/Mark
Marks one or more tests in the testplan, depending upon the position of the insertion cursor. The following table shows how to use Testplan/Mark.
To mark Then do this and select the Mark command
A single test A group of related tests
Click on the test description Click on the group description
Two or more adjacent tests Select the test descriptions of the tests (by holding and their subordinate tests and dragging the mouse button)
Users Guide
513
For more information, see Marking a testplan on page 243.
Testplan/Mark All
Marks all the tests in the testplan.
Testplan/Mark by Named Query

Opens the Mark by Named Query dialog, which allows you to create, combine, edit, or delete named queries. For more information, see Creating and modifying testplan queries on page 246.
Testplan/Mark by Query
Opens the Mark by Query dialog, which allows you to construct a query from attributes, symbols, the script, the testcase, and test data. For more information, see Creating and modifying testplan queries on page 246.
Testplan/Unmark
Unmarks one or more tests and any subordinate tests, depending on the position of the insertion cursor. The following table shows how to use Testplan/Unmark.
To unmark Then do this and select the Unmark command
A single test A group of related tests Two or more adjacent tests and their subordinate tests
Click on the test description Click on the group description Select the test descriptions of the tests
514
Users Guide
27 MENU COMMANDS View menu
Testplan/Unmark All
Unmarks all the tests in the testplan.
View menu
The View menu contains the following commands: Breakpoints Call Stack Collapse Data Expand Data Expression Global Variables Local Variables Module... Transcript Note For complete information about debugging testcases, see Chapter 7, Using the Debugger.
View/Breakpoints
Available only in debugging mode. Displays in the Breakpoints window, which lists the module names and line numbers where breakpoints are set. Double-click on an entry to move the focus to that line in the script.
View/Call Stack
Available only in debugging mode. Displays the current call stack in the Call Stack window. A call stack is a listing of all the function calls that have been called to reach the current function in the script you are debugging.
Users Guide
515
View/Collapse Data
Available only in debugging mode. Collapses the data about a variable. A plus (+) sign in front of a variable name indicates that the data about the variable is already collapsed. A minus sign (-) in front of a variable name indicates that the data can be collapsed. A period (.) indicates that all the data about a variable is already displayed. Note You can also collapse the data about a variable by doubleclicking on the minus (-) symbol.
View/Expand Data
Available only in debugging mode. Expands the data about a variable. A minus (-) in front of a variable name indicates that more data is available. A plus (+) in front of a variable name indicates that the data about it is already expanded. A period (.) indicates that all the data about a variable is already displayed. Note You can also expand the data about a variable by doubleclicking on the (+) symbol.
View/Expression
Available only in debugging mode. Opens an Expression window in which you can evaluate an expression and check the result. To evaluate an expression, type it in the input area and press Enter. The result appears directly beneath the expression. You use the Expression window to query the application you are running, thus performing actions different from those specified in your script. If you type an identifier name, the result is the value that variable currently has in the running script. If you type an expression, the result is the value of that expression. Any function you specify must return a value, and must be in scope (accessible) at the current source line. You can reference class properties and methods that return values in expressions, as long as the declaration for the class they belong to is included in one of the modules used by the script being debugged.
516
Users Guide
If an expression evaluates to a complex value, like an array, SilkTest may display its result in collapsed form. Use View/Expand Data or View/Collapse Data to manipulate the display, or double-click on the plus (+) and minus (-) symbols.
View/Global Variables
Available only in debugging mode. Displays the Globals window, which lists all global variables that are in scope (accessible) from the current source line and their current values. You can set a new value for the variable in the Set Variable text field. If a variable is uninitialized, SilkTest labels it <unset>. If a variable has a complex value, like an array, SilkTest may display its result in collapsed form. Use View/Expand Data or View/Collapse Data to manipulate the display, or double-click on the plus (+) and minus (-) symbols. For more information on 4Test scope rules, see online Help.
View/Local Variables
Available only in debugging mode. Displays the Locals window, which lists all local variables that are in scope (accessible) in the current function declaration and their current values. You can set a new value for the variable in the Set Variable text field. If a variable is uninitialized, SilkTest labels it <unset>. If a variable has a complex value, like an array, SilkTest may display its result in collapsed form. Use View/Expand Data or View/Collapse Data to manipulate the display, or double-click on the plus (+) and minus (-) symbols. For more information on 4Test scope rules, see online Help.
View/Module...
Available only in debugging mode. Opens the View Module dialog, which lists the modules used by the script being debugged. Double-click on a module name to view it in a debugging window. The list includes all the modules that loaded by startup.inc, so you can set breakpoints in GUI functions, classes, and so forth.
Users Guide
517
27 MENU COMMANDS Window menu
View/Transcript
Available only in debugging mode. Makes the transcript window for the current debugging session the active window. SilkTest sends error information and output from print commands to the transcript window. The contents of the transcript window are not written to disk.
Window menu
The Window menu contains the following commands: Arrange Icons Cascade Close All Next Previous Tile Horizontally Tile Vertically
Window/Arrange Icons
Arranges the icons in a row in the SilkTest main window.
Window/Cascade
Layers one SilkTest child window on top of the other. Except for the top window, which is completely visible, only the title bar of each window is showing.
Window/Close All
Closes all open windows. If any file has changed since you last saved it, SilkTest prompts you to save the file before closing it.
518
Users Guide
Window/n
Lists all open windows. Each window in the history is numbered from 1 to n and is identified by its file type and the window title bar. If the file has never been saved, then the title bar is (untitled). Otherwise the file name is shown. The windows are numbered in the order you opened them, with first window numbered 1. If more than nine windows are open, the More Windows... item appears; click More Windows to open the Select Window dialog, which lists all open windows. The currently active window is preceded by a check mark. To make another window active, do one of the following: Select a window from the menu. Select a window by its number; for example, to select the second window, press Alt+W+2. Cycle through the history by using Window/Next or Window/Previous.
Window/Next
Makes the next window in the numbered window history the active window. Repeat this command to cycle through the window history.
Window/Previous
Makes the previous window in the window history the active window. Repeat this command to cycle through the window history.
Window/Tile Horizontally
Makes each child window of the SilkTest main window the same size and positions them in a north to south orientation along the y axis. Each window is as wide as the SilkTest window frame and is as tall as the height of the display divided by the total number of child windows.
Users Guide
519
Window/Tile Vertically
Makes each child window of the SilkTest main window the same size and positions them in an east to west orientation along the x axis. Each window is as tall as the SilkTest window frame and is as wide as the width of the frame divided by the total number of child windows.
520
Users Guide
Appendices
I IVtP I a r
In this part

Chapter Page
Appendix A, Testplan Editor Keywords Appendix B, Using Drag-and-Drop Appendix C, Supporting Owner-Draw List Boxes Appendix D, Useful Multitestcase Code Templates Appendix E, Glossary
523 527 529 531 535
Users Guide
521
522
Users Guide
Testplan Editor Keywords

AA xp i d n e
Keywords
Just as the 4Test language has a set of keywords that you use to construct testcases, the testplan editor also has its own distinct set of keywords that you use when creating testplans. The keywords are described in the following table.
Use this keyword To accomplish this task
comment
Specify a comment that will display in the results file. For more information, see Adding comments that display in the results on page 55.
include
Specify the name of an include file that contains part of the testplan, typically a subplan. For more information, see Dividing a testplan into a master plan and subplans on page 259.
script
Associate a 4Test script file with one or more tests in the plan. For more information, see Linking the testplan to scripts and testcases on page 59.
testcase
Associate a 4Test testcase with one or more tests in the plan. For more information, see Linking the testplan to scripts and testcases on page 59.
testdata
Associate data with a testcase in the testplan. When the testplan is run, the testplan editor passes the data to the associated testcase. For more information, see Specifying data that is unique to a single test description on page 223.
Note Attribute names are also considered keywords.
Users Guide
523
A TESTPLAN EDITOR KEYWORDS
Statements
You use the testplan editor keywords to construct statements, using this syntax: keyword : value keyword value One of the the testplan editor keywords. A comment, script, testcase, include file, attribute name, or data value.
For example, this statement associates the script myscript.t with the plan:
script : myscript.t
Spaces before and after the colon are optional.

Symbol definition statements
Use symbols to define data that is shared by a group of tests in the plan. Symbol definitions follow these syntax conventions: The symbol name can be any valid 4Test identifier name, but must begin with the $ character. The symbol value can be any text. When the testplan editor encounters the symbol, it expands it (in the same sense that another language expands macros). For example, the following the testplan editor statement defines a symbol named Color and assigns it the STRING value Red:
$Color = "Red"
To use a $ in a symbol value, precede it with another $. Otherwise, the compiler will interpret everything after the $ as another symbol. For example, this statement defines a symbol with the value Some$String:
$MySymbol = "Some$$String"
To assign a null value to a symbol, do not specify a value after the equals sign. For example:
$MyNullSymbol =
To indicate that a test is incomplete when generating a testplan completion report, assign the symbol the ? character. For example:
$MySymbol = ?
Note If a symbol is listed in a testcases argument list, but is not assigned a value before the testcase is actually called, the testplan editor generates a runtime error that indicates that the symbol is undefined. To avoid this error, assign the symbol a value or a ? if the data is not yet finalized. For more information on 4Test syntax features (identifier names, data types, or the list constructor operator) see online Help.
524
Users Guide
The # operator
When a # character precedes a statement, the statement will double as a test description in the testplan. This helps eliminate possible redundancies in the testplan. For example, the following test description and script statement:
Script is test.t script:test.t
can be reduced to one line in the testplan:

#script: test.t
The testplan editor considers this line an executable statement as well as a description. Any statements that follow this description in the testplan and that trigger test execution must be indented.
Comments
Use two forward slash characters to indicate that a line in a testplan is a comment. For example:
// This is a comment
Comments preceded by // do not display in the results file. You can also specify comments using the comment statement; these comments will display in the results files. For more information, see Adding comments that display in the results on page 55.
Users Guide
525
526
Users Guide
Using Drag-and-Drop
BA xp i d n e
SilkTest supports drag-and-drop operations on Windows 95, Windows 98, and Windows NT. Drag-and-drop operations have three distinct parts: Selecting an item by pressing a mouse button Moving, or dragging, the item Releasing the mouse button, thereby dropping the item at a target location
The target location can be a logical location, that is, an identifiable object in a listview, treeview, or listbox, or it can be a physical location specified by x, y coordinates in a window. Five new methods support drag-and-drop operations: BeginDrag method BeginDragAt method EndDrag method EndDragAt method DragMouse method
BeginDragAt and EndDragAt are general methods that work for any window. They move an item to a physical target location and operate between windows of the same application or a different application. Segue recommends that you use care when recording drag-and-drop operations. Do the testcase setup carefully, and while recording, avoid extraneous movements. EndDrag and BeginDrag apply only to list box, listview, and treeview controls. They move an item to a logical target location and operate between windows of the same application or a different application.
Users Guide
527
B USING DRAG-AND-DROP
DragMouse combines the functionality of the begin and end drag methods. However, DragMouse operates only within a single window. For more information, see online Help.
528
Users Guide
Supporting Owner-Draw List Boxes

Cp xe A i d n
An owner-draw list box is a list box that has the owner-draw style bit set. This is distinct from a custom object that looks like a standard list box but is not. The following procedure describes how developers can modify an application so that SilkTest can access the text of a standard list box that is owner-draw and that does not have the HasStrings style bit turned on. (If the HasStrings style bit of the owner-draw list box is turned on, then you do not need to make the modifications described here.) The procedure entails modifying each owner-draw list boxs parent window procedure so that SilkTest can query the parent about what is in the list box. Procedure To turn on the HasStrings style bit of the owner-draw list box: 1 2 Include owndraw.h, supplied with SilkTest, in your source files. For each owner-draw list box, add code, such as that shown below, to its parents WndProc.
LONG FAR PASCAL ListParentWndProc (HWND hWnd, UINT uiMsg,WPARAM wParam, LPARAM lParam) { //Use a static for the registered message number static UINT uiMsgGetItem Text = 0; LPGETITEMTEXTSTRUCT LpGetItemText; USHORT usItem; PSZ pszItemText; //Register the QAP_GETITEMTEXT message if it is not registered if (uiMsgGetItem Text == 0) uiMsgGetItemText = RegisterWindowMessage("QAP_GETITEMTEXT");
Users Guide
529
C SUPPORTING OWNER-DRAW LIST BOXES
switch (uiMsg) { ... default; //Process the QAP_GETITEMTEXT message if (uiMsg == uiMsgGetItemText) { //lParam points to a LPGETITEMTEXTSTRUCT structure lpGetItemText = (LPGETITEMTEXTSTRUCT) lParam; //Perform the requested action switch (lpGetItemText->Action) { case ODA_HASTEXT: //Tell the QAP driver if your list box contains text if (your list box has text) lpGetItemText->bSuccess = TRUE; else lpGetItemText->bSuccess = FALSE; break; case ODA_GETITEMTEXT: //Return the text for the requested list item //(lpGetItemText->itemID is the index of the item in the //list box -- the same number passed to LB_GETITEMDATA) usItem = UINT (lpGetItemText->itemID); pszItemText = <pointer to text of item[usItem]>; strncpy (lpGetItemText->lpstrItemText, pszItemText, lpGetItemText->nMaxItemText); lpGetItemText->lpstrItemText[lpGetItemText->nMaxItemText-1] = \0; lpGetItemText->bSuccess = TRUE; break; case ODA_GETITEMTEXTSIZE: //Return the length of the requested list item usItem = UINT (lpGetItemText->itemID); lpGetItemText->nMaxItemText = <length of item[usItem]> + 1; lpGetItemText->bSuccess = TRUE; break; ... } } } }
530
Users Guide
Useful Multitestcase Code Templates

Dp xe A i d n
This appendix contains two templates that you might find useful as a basis for your multitestcases.
What you will learn
This appendix contains the following section:

Topic Page
Parallel template Client/Server template
531 532
Parallel template
This template is stored as parallel.t in the Examples subdirectory of the SilkTest installation directory. The code tests a single application that runs on an externally defined set of machines. This multitestcase template accepts a list of machine names. The application whose main window is MyMainWin is invoked on each machine. The same operations are then performed on each machine in parallel. If any testcase fails, the multitestcase will be marked as having failed; however, a failed testcase within a thread does not abort the thread. You can use this template by doing three edits: Include the file that contains your window declarations. Substitute your applications MainWin name defined in your MainWin window declaration for the templates MyMainWin MainWin name.
Users Guide
531
D USEFUL MULTITESTCASE CODE TEMPLATES Client/Server template
Insert the calls to one or more tests, or to the main function, where indicated.
use "myframe.inc" multitestcase MyParallelTest (LIST of STRING lsMachines) STRING sMachine // Connect to all machines in parallel: for each sMachine in lsMachines spawn SetUpMachine (sMachine, MyMainWin) rendezvous // Set app state of each machine, invoking if necessary: SetMultiAppStates() // Run testcases in parallel for each sMachine in lsMachines spawn SetMachine (sMachine) // Call testcase(s) or call main() rendezvous
Client/Server template
This template is stored as multi_cs.t in the Examples subdirectory of the SilkTest installation directory. This testcase invokes the server application and any number of client applications (based on the list of machines passed to it) and runs the same function on all clients concurrently, after which the server will perform end-of-session processing. You can use this template by doing the following edits: Include the files that contains your window declarations for both the client application and the server application. Substitute your server applications MainWin name defined in your MainWin window declaration for the templates MyServerApp MainWin name. Substitute your client applications MainWin name defined in your MainWin window declaration for the templates MyClientApp MainWin name.
532
Users Guide
Replace the call to PerformClientActivity with a function that you have written to perform client operations and tests. Replace the call to DoServerAdministration with a function that you have written to perform server administrative processing and/or cleanup.
use "myframe.inc" multitestcase MyClientServerTest (STRING sServer, LIST of STRING lsClients) STRING sClient // Connect to server machine: SetUpMachine (sServer, MyServerApp) // Connect to all client machines in parallel: for each sClient in lsClients spawn SetUpMachine (sClient, MyClientApp) rendezvous // Set app state of each machine, invoking if necessary: SetMultiAppStates() // Run functions in parallel on each client: for each sClient in lsClients spawn // Make client do some work: [sClient] PerformClientActivity() rendezvous // Perform end-of-session processing on server // application: [sServer] DoServerAdministration()
Users Guide
533
534
Users Guide
Glossary
EA xp i d n e
Agent
The 4Test Agent is the software process that translates the commands in your 4Test scripts into GUI-specific commands. In other words, it is the Agent that actually drives and monitors the application you are testing. One Agent can run locally on the host machine. In a networked environment, any number of Agents can run on remote machines. The state you expect your application to be in at the beginning of a testcase. This is in addition to the conditions required for the base state. See base state. In the testplan editor, site-specific characteristics that you can define for your testplan and assign to test descriptions and group descriptions. Each attribute has a set of values. For example, you define the Developer attribute and assign it the values of Bob, Carol, Ted, and Alice, the names of the QA engineers in your department. Attributes are useful for grouping tests, in that you can run or report on parts of the testplan that have a given attribute value; for example, all tests that were developed by Bob can be executed as a group. In SilkTest, an attribute is a characteristic of an application that you verify in a testcase. Used in the Verify Window dialog.
application state
attributes
base state
The known, stable state you expect the application to be in at the start of each testcase. See DefaultBaseState. In debugging mode, a list of functions and testcases executing at the time an error occurred in a script. Listed in reverse order, from the last one executed back to the first. Subordinate object in the GUI hierarchy. A child object is either logically associated with or physically contained by its superior object, the parent. For example, the File menu, as well as all other menus, are physically contained by the main window. See parent object. GUI object type. The class determines which methods can operate on an object. Each object in the application is an instance of a GUI class.
call stack
child object
class
Users Guide
535
E GLOSSARY
class mapping
Association of nonstandard custom objects with standard objects understood by SilkTest. A windows internal area less the scroll bars, title bar, and borders. Nonstandard object that SilkTest does not know, by default, how to interact with. Generalized testcase that receives many combinations of data from 4Test functions and/or a testplan. Variable defined within a class or window declaration. See window declarations. Built-in application state function that returns your application to its base state. By default, the built-in DefaultBaseState ensures that the application is running and is not minimized, the main window of the application is open, and all other windows (for example, dialogs and messages boxes) are closed. See also base state. Dual-paned display-only window that lists every expected value in a testcase and its corresponding actual value. Highlights all occurrences where expected and actual values differ. You display the Difference Viewer by selecting the box icon in the results file. Creation of a fully qualified object name for a GUI object based on the objects class and tag. Done automatically whenever the recorded object has no associated declaration defined in the test frame file. See also fully qualified object name. A library of reusable functions that allow code, data, and resources to be shared among programs using the module. Programs are linked to the module dynamically at runtime. Signal that something did not work as expected in a script. Logs the error in the results file. See test frame file. Name that uniquely identifies a GUI object. The actual format depends on whether or not a window declaration has been previously recorded for the object and its ancestors. See also dynamic instantiation. In the testplan editor, one or more lines in an outline that describe a group of tests, not a single test. Group descriptions by default are displayed in black. See also test description.
client area custom object
data-driven testcase
data member declarations DefaultBaseState
Difference Viewer
dynamic instantiation
dynamic link library (DLL)
exception
frame file fully qualified object name
group description
536
Users Guide
E GLOSSARY
hierarchy of GUI objects Hungarian notation
Parent-child relationships between GUI objects. Naming convention in which a variables name begins with one or more lowercase letters indicating its data type. For example, the variable name sCommandLine indicates that the data type of the variable is STRING. Name used in test scripts to refer to an object in the application. Logical, GUI-independent name. Identifier is mapped to the tag in a window declaration. File that contains window declarations, and can contain constant, variable, and other declarations. Testing that determines the actual (not simulated) impact of multi-machine operations on an application, the server, the network, and all related elements. To make an application suitable for a specific locale: for example, to include foreign language strings for an international site. In the testplan editor, a test that is documented but cannot be automated and, therefore, cannot be run within the testplan. You might chose to include manual tests in your testplan in order to centralize the testing process. To indicate that a test description is implemented manually, you use the keyword value manual in the testcase statement. In the testplan editor, a technique used to work with one or more tests as a group. A mark is denoted by a black stripe in the margin bar of the testplan. Marks are temporary and last only as long as the current work session. Tests that are marked can be run or reported on independently as a subset of the total plan. In the testplan editor, that portion of a testplan that contains only the top few levels of group descriptions. Within the master plan you can expand (display) its subplans, which contain the remaining levels of group description and test description. The master plan/subplan approach allows multi-user access to a testplan, while at the same time maintaining a single point of control for the entire project. A master plan file has a .pln extension. See also subplan. Dialog box that has only static text and pushbuttons. Typically, message boxes are used to prompt a user to verify an action (such as Save changes before closing?) or to alert a user to an error. Operation, or action, to perform on a GUI object. Each class defines its own set of methods. Methods are also inherited from the classs ancestors.
identifier
include file
load testing
localize an application
manual test
mark
master plan
message box
method
Users Guide
537
E GLOSSARY
minus (-) sign
In a file, an icon that indicates that all information is displayed. Click on the minus sign to hide the information. The minus sign becomes a plus sign. A dialog box that presents a task that must be completed before continuing with the application. No other part of the application can be accessed until the dialog box is closed. Often used for error messages. A dialog box that presents a simple or ongoing task. May be left open while accessing other features of the application, for example, a Search dialog box. Tests that deliberately introduce an error to check an applications behavior and robustness. For example, erroneous data may be entered, or attempts made to force the application to perform an operation that it should not be able to complete. Generally a message box is generated to inform the user of the problem. Indented declarations that denote the hierarchical relationships of GUI objects in an application. A script file or include file that is executable but not editable or viewableas opposed to a source file, which is editable but must be compiled before it is executed. In the testplan editor, a structured, usually hierarchical model that describes the requirements of a testplan and contains the statements that implement the requirements. The outline supports automatic, context-sensitive coloring of testplan elements. In SilkTest, the outline is a 4Test editor mode that supports automatic, context-sensitive coloring and indenting of 4Test elements.
modal
modeless
negative testing
nested declarations
object file
outline
parent object
Superior object in the GUI hierarchy. A parent object is either logically associated with or physically contains its subordinate object, the child. For example, the main window physically contains the File menu as well as all other menus. Testing to verify that an operation in an application performs within a specified, acceptable period of time. Alternately, testing to verify that space consumption of an application stays within specified limits. In a file, an icon that indicates that there is hidden information. You can show the information by clicking on the plus sign. The plus sign becomes a minus sign. Different classes or objects performing the same named task, but with different execution logic.
performance testing
plus (+) sign
polymorphism
538
Users Guide
E GLOSSARY
properties
Characteristics, values, or information associated with an object, such as its state or current value. User-selected set of characteristics that are compared to the attributes, symbols, or execution characteristics in a testplan. When the set of characteristics matches a test, the test is marked. (See mark.) This is called marking by query. For example, you might run a query in order to mark all tests that are defined in the find.t script and that were created by the developer named Bob. A built-in, automatic mechanism to ensure the application is in a known state. If the application is not in the expected state, a message is logged to the results file and the problem is corrected. The recovery system is invoked before and after each testcase is executed. A set of baseline tests that are run against each new build of an application to determine if the current build has regressed in quality from the previous one. In SilkTest, a file that lists information about the scripts and testcases that you ran. In the testplan editor, a results file also lists information about the testplan that you ran; the format of a results file mimics the outline format of the testplan it derives from. The name of the results file is script-name.res or testplan-name.res. A collection of related 4Test testcases and functions that reside in a script file. A file that contains one or more related testcases. A script file has a .t extension, such as find.t. Tests that constitute a quick set of acceptance tests. They are often used to verify a minimum level of functionality before either accepting a new build into source control or continuing QA with more in-depth, time-consuming testing. In the testplan editor, lines that implement the requirements of a testplan. The testplan editor has several different statements: testcase, script, testdata, include, and attribute. Statements consist of one of the preceding keywords followed by a colon and a value. In SilkTest, a statement is a method or function call or 4Test flow-of control command (such as if..then) that is used within a 4Test testcase.
query
recovery system
regression testing
results file
script
script file
smoke test
statement
status line
Area at the bottom of the window that displays the status of the current script, the line and column of the active window (if any), and the name of the script that is currently running. When the cursor is positioned over the toolbar, it displays a brief description of the item.
Users Guide
539
E GLOSSARY
stress testing
Tests that exercise an application by repeating the same commands or operation a large number of times. Testplan that is referenced by another testplan, normally the master testplan, by using an include statement. Portion of a testplan that lives in a separate file but can be expanded inline within its master plan. A subplan may contain the levels of group description and test description not covered in the master plan. A subplan can inherit information from its master plan. You add a subplan by inserting an include statement in the master plan. A subplan file has a .pln extension, as in subplan-name.pln. See also master plan. A file that names any number of 4Test test script files. Instead of running each script individually, you run the suite, which executes in turn each of your scripts and all the testcases it contains. In the testplan editor, used in a testplan to pass data to 4Test testcases. A symbol can be defined at a level in the testplan where it can be shared by a group of tests. Its values are actually assigned at either the group or test description level, depending on whether the values are shared by many tests or are unique to a single test. Similar to a 4Test identifier, except that its name begins with a $ character. Objects actual name or index as it appears in the GUI. The name by which SilkTest locates and identifies objects in the application. A hierarchical outline in the testplan editor that you can use as a guide when creating a new testplan. Based on the window declarations in the frame file. In the testplan editor, a terminal point in an outline that specifies a testcase to be executed. Test descriptions by default are displayed in blue. See also group description. Contains all the data structures that support your scripts: window declarations, user-defined classes, utility functions, constants, and variables, as well as other include files. In a script file, an automated test that ideally addresses one test requirement. Specifically, a 4Test function that begins with the testcase keyword and contains a sequence of 4Test statements. It drives an application to the state to be tested, verifies that the application works as expected, and returns the application to its base state. In a testplan, a testcase is a keyword whose value is the name of a testcase defined in a script file. Used in an assignment statement to link a test description in a testplan with a 4Test testcase defined in a script file.
subplan
suite
symbols
tag
template
test description
test frame file
testcase
540
Users Guide
E GLOSSARY
testplan
In general, a document that describes test requirements. In the testplan editor, a testplan is displayed in an easy-to-read outline format, which lists the test requirements in high-level prose descriptions. The structure can be flat or many levels deep. Indentation indicates the level of detail. A testplan also contains statements, which are keywords and values that implement the test descriptions by linking them to 4Test testcases. Large testplans can be divided into a master plan and one or more subplans. A testplan file has a .pln extension, such as find.pln. See also master plan and subplan. A named location in which you can store a piece of information. Analogous to a labeled drawer in a file cabinet. 4Test code that checks that an application is working by comparing an actual result against an expected (baseline) result. Descriptions of all the objects in the applications graphical user interface, such as menus and dialogs. Declarations are stored in an include file which has a .inc extension, typically the frame.inc file. See also test frame file. Predefined identifiers for referring to parts of the window. Associated with common parts of MoveableWin and Control classes, such as LeftEdge, MenuBar, ScrollBar.
variable
verification statement
window declarations
window part
Users Guide
541
E GLOSSARY
542
Users Guide
Index
Symbols
X (letter X) meaning of 487 # (pound sign) character 150, 233, actual values compared with expected values 455, 503, 507 replacing expected values with 155, 506 Add Breakpoint dialog 449 Add menu item (Breakpoint) 449 adding breakpoints 449, 450 Adobe Acrobat Reader 41 Agent menu 28 Agent menu item (Options) 464 Agent Options dialog recovery system and 294 agents about 28 class for 305 closing options for 475 connecting to 362 definition 27, 306, 339, 351 opening options set for 479 options for 464 predefined identifier for 306 saving options set for 486 specifying name 482 version number 29 alias statement 424 aliasing DLL names 424 analyzing deviations 503 test results 148155 Application field on Extensions dialog 472 Application State menu item (Record) 187, 490492 Application State menu item (Run) 508 application state status window displaying 494 suppressing 111, 494 application states behavior of 186 definition 186 how recorded in script file 133 recording 185189, 490492 running 508 testing 189 applications GUI-specific differences 279 porting 267 restart after failure 348, 380 starting on different GUIs 273 starting up 380 testing internationalized 283286 applying masks 417421 area charts 164 arguments passing to a script 147 Arrange Icons menu item (Window) 518 arranging icons 518 ASCII files exporting Completion reports 258,
525
# character in tags 85 $ (dollar sign) character 226 meaning of double 524 $ character in tags 85 - (minus sign) meaning of 486, 516, 517 l (bullet) meaning of 447 * (asterisk) 123 in queries 248 + (plus sign) meaning of 487, 516, 517 . (period) meaning of 516, 517 // (double slash) meaning of 487, 525 ? (question mark) in queries 248 to indicate unset symbol 230, 249 @ character in tags 85 ^ character in tags 85 | character in tags 88
511
ASCII format translating results into 156, 158,
504
ASCII text editor, using to create scripts and testplans 35 assigning to plan attributes 240 script 60 testcase 60 assignment statements format of 240 inserting manually 63 scope of 50 See also statements. asterisks 123, 248 attributes, testplan assigning to plan 240 defining 237240, 511 definition of 236 identifying inherited 257 including in queries 249 storing 237 uses of 236 using in query 514 using in reports 505 attributes, verification about 130 defining for a class 311 using under Windows 114 avoiding testcase failure 155, 506
Numerics
4test.hlp 329 4test.txt 329
A
Abort menu item (Debug) 450 Abort menu item (Run) 507 aborting testcases 145 accelerators for changing outline indents 54 access statement 368 accessing 368 Acquire function 368 Acquire Lock menu item (Include) 262, 463 Actions menu item (Record) 488490 when to use 139 actions, recording mouse movements during 481 active, verifying window as 467
B
backups option for creating 478 bar charts 164 base states definition 290 baseline bitmap
Users Guide
543
definition 410 setting 413 BaseState method recording 298 binary files 134 Bitmap menu 403 bitmap tool 401421 capturing bitmaps in 408412 comparing bitmaps in 412417 opening 406 opening from a results file 153,
C
C language data types supported in 4Test 33 call stack clearing 451 definition 515 looking at 152 viewing 515 captions GUI-specific 274 use in tag 93 capturing bitmaps 408412 Zoom window 416 Cascade menu item (Window) 518 cascading windows 518 chaining to next fault handler 474 changing built-in classes 304 indenting in outline 54 tags 93 char data type (C) 33 charting reports 164 Check In command, File menu 433 Check Out command, File menu 433 Class Map dialog 496 Class Map menu item (Options) 475 Class Map option 318 class map options closing 475 opening set of 479 saving set of 486 classes changing built-in 304 creating class that maps to several 277 custom, marking 499 defining 314 defining attributes for 311 defining properties for 308 definition 29, 83, 495 deriving new 323 for non-windows 305 hierarchy in 304 logical 305 mapping custom 318, 495 marking custom 475 of main window 83 verifying, option for 467 class-property pair, specifying 123 cleanup stage of testcase recording 132 clearing call stack 451 client/server basic concepts 337 configurations 343
407
bitmaps agent options 127 capturing 408412 capturing during fault, option 474 comparing 412417 comparison failure 153 functions 411 options for agent 468 saving 410 verifying 126, 411 bitview program for Windows 401421 Blue Express enabling extension 68 Bookmarks 41 box icon 153 meaning of 507 breaking lines of code 35 Breakpoint menu 449450 description 447 breakpoints adding 174, 449, 450 adding temporary 175 definition 173, 447 deleting 175, 449, 450 using Step Into with 452 using Step Over with 452 viewing 175, 515 browser extensions 70 and recovery system 78, 102, 141,
144
disabling 75, 104, 143 enabling 68, 102, 142 enabling on host machine 73 enabling on target machine 72 browsers, setting default 483 Built-in data types 33 BUTTON class 319 buttons used to close confirmation window 468 used to close windows 468
client/server applications effect on testing strategy 337, 351 example code 384398 testing concurrently 374 testing in parallel 376 testing serially 373 using timers with 374 clipboard class for 305 copying to 453 cutting text to 453 pasting from 455 predefined identifier for 306 Close All menu item (Include) 463 Close All menu item (Window) 518 Close menu item (File) 457 Close menu item (Include) 463 close options for agent 468 Close Set menu item (Options) 475 CloseWindows method recovery system and 292 closing all windows 518 files 457 options set 475 set of options 475 subplans 463 windows, by recovery system 292 code breaking lines of 35 collapsing 486, 487 commenting 487 deleting comments from 488 expanding 487 coding template 384 Collapse All menu item (Outline) 487 Collapse Data menu item (View) 516 Collapse menu item (Outline) 486 collapsing code 486 data about variables 516 colors in expanded subplan 261 in results file 151 of margin bar 262 of symbols 232 setting in editor 476 using to indicate outline structure 54 Combine Property Sets dialog 123 combining queries 250 results sets 160 combo boxes dropping down before accessing contents 471
544
Users Guide
command line interface -m option 444 options for partner command 444 -p option 444 -q option 444 -query option 444 -r option 444 -resexport option 444 starting from 443 commands 447520 for changing outline indenting 54 for marking 244 See also menus. commands. See menus. Comment Block menu item (Outline) 487 comments 476 adding 487, 525 adding to results sets 151 deleting 488 in testplans 55 Compact menu item (Results) 503 Compare Two Results menu item (Results) 161, 503 comparing bitmaps 412417 results files 161, 503, 505 compatibility options for agent 469 Compile All menu item (Run) 508 Compile menu item (Run) 508 compiled files 134 compiler constants 484 compiling 508 conditional 271 completion definition 258 Completion menu item (Testplan) 511 Completion report 511 how to generate 258 concurrency features of 4Test 363,
376
concurrency testing 340 code example 387 concurrent programming techniques 347 concurrent scripts basic concepts 349 constructs used in 376 rendezvous statement 349 synchronizing machines 349 when to use 349 writing 363 concurrent tests use of semaphore in 368 conditional compilation 271 using constants 271
configuration testing 340 configuring for LAN Manager 358 for NetBIOS 359 for remote testing 354 for TCP/IP 356 naming PC Agents 355, 359 the basic process 353 constants sCmdLine 90 using in scripts and declarations 271, 484 wInvoke 96 Continue menu item (Debug) 451 controls evenly sized and spaced 320 GUI-specific 275, 276 supporting additional 275 unevenly sized and spaced 322 converting Differences window to mask 418 Copy menu item (Edit) 453 copying text 453 crashes of application under test, handling 217 creating classes 323 data-driven 190233 masks 417421 master plan 259 methods 306, 307 property sets 121 subplans 262 testplans 4765 critical statement 365 Ctrl+Alt, if it doesnt work 113 cursor class for 305 predefined identifier for 306 custom classes mapping to standard 475, 495 marking 499 custom objects definition 317 supporting 317325 verifying class of 467 customizing colors 151 CustomWin class 323 Cut menu item (Edit) 453 cutting text 453
D
data associating with objects 98 shared by multiple tests 226
storing in external files 194 verifying that an application can handle invalid 194 Data types C, support for 33 summary of built-in 33 database initializing 373 manipulating from testcases 346 data-driven testcases 190233 deadlock 4Test handling of 370 script deadlock definition 370 Debug dialog opening 457 Debug menu 450452 description 447 Debug menu item (File) 457 Debug menu item (Run) 509 debugger 171, 171182 collapsing variable data in 516 entering 509 evaluating expressions in 516 exiting 450 files in, list of 458 files in, selecting 457 options for 485 viewing breakpoints in 515 viewing call stack in 515 viewing modules in 517 viewing variables in 517 debugging mode entering 457 declaration files. See include files. declarations. See window declarations. declaring DLL functions 423 default system behavior options for setting 464 DefaultBaseState function 300 DefaultScriptEnter function 300 DefaultScriptExit function 300 DefaultTestcaseEnter function 300 DefaultTestcaseExit function 300 defects tracking 158 Define Attributes dialog 240 Define Attributes menu item (Testplan) 511 defining attributes 237240, 511 data for objects 98 queries 246250, 514 Delete All menu item (Breakpoint) 449 Delete Breakpoint dialog 449
Users Guide
545
Delete menu item (Breakpoint) 449 Delete menu item (Edit) 453 Delete menu item (Results) 503 Delete Results dialog 503 deleting attributes 237 breakpoints 449, 450 property sets 125 queries 247, 253 results files 503 text 453 derived keyword 307 designing sound testcases 106 Detail menu item (Testplan) 511512 Details pushbutton on Extensions dialog 475 Developer/2000 enabling extension 68 devising test requirements 108 dialogs Add Breakpoint 449 Class Map 496 close method for 295 Combine Property Sets 123 Debug 457 Define Attributes 240 Delete Breakpoint 449 Delete Results 503 Edit Property Set 124 Editor Colors 476 Editor Font 477 Extensions 472 Find 453 General Options 478 Go to Line 455 invoking 96 Mark by Query 514 modal, definition 105 New 458 Print 459 Printer Setup 460 Property Sets 122 Record Actions 489 Record Application State 188, 490 Record Method 492 Record Status 491 Record Testcase 110 Record Window Declarations 495 Record Window Declarations Options 498 recording declarations for 81, 91 Replace 455 Runtime Options 482 Select Results 506 Test Plan Completion Report 511 Window Locations 501
Difference Viewer displaying 154, 507 finding differences in 455 setting editor colors for 476 Differences menu 404 Differences window converting to mask 418 definition 414 directories GUI-specific navigation 280 disabling extensions 67 distributing testing workload on network 337, 351 DLL calling from scripts 423427 documentation about methods, functions, and properties 462 dollar sign character 226 double data type (C) 33 drag-and-drop 527528 not recording 481 Duplicate pushbutton on Extensions dialog 475 duplicating extension settings 475 dynamic instantiation 138
errors avoiding 506 finding 152 fixing 155 system, handling 217 viewing information about 518 See also exceptions ExceptData function 215 exceptions definition 210 handling 209 Execute field, transcript window 177 executing. See running. Exit menu item (Debug) 450 Exit menu item (File) 458 exiting 458 debugger 450 threads 365 Visual 4Test 456 Expand All menu item (Outline) 487 Expand Data menu item (View) 516 Expand menu item (Outline) 487 expanding code 487 expected values compared with actual values 455, 503, 507 replacing with actual values 155,
506
E
Edit menu 453456 bitmap tool 403 description 447 Edit Property Set dialog 124 editing attributes 237 property sets 124 queries 247, 252 window declarations 495 editor setting general options for 478 Editor Colors dialog 476 Editor Colors menu item (Options) 476 Editor Font dialog 477 Editor Font menu item (Options) 476 editor, using ASCII editor 35 Enable Y2K date transformation rules field 483 enabled, verifying window as 467 enabling extensions 67, 472 error messages finding 152 GUI-specific 280 opening bitmap tool from 407
Export menu item (Results) 503 exporting results 158 exposed, verifying window as 467 Expression menu item (View) 516 Expression window evaluating expressions in 516 expressions evaluating 179, 516 Extension Enabler 69 Extension Enabler dialog 72 extensions associating with applications 472 disabling 67 enabling 67, 68, 472 enabling manually 68 enabling on host machine 69 enabling on target machine 68 enabling other extensions 473 enabling primary extensions 472 how extensions work with applications 67 menu for 28 options 473 Extensions dialog 70, 472 Application field 472 Details pushbutton 475 Duplicate pushbutton 475 New pushbutton 475
546
Users Guide
Options pushbutton 473 Other Extensions check boxes 473 Primary Extension field 472 Remove pushbutton 475 Extensions menu item (Options) 472 extensions, file bitmap files 410 default for include file 479 mask files 417 options file 486 results file 509 Extract menu item (Results) 504 extracting results 156, 504
F
failures marking 504 fault trapping 217 capturing desktop bitmap 474 chaining to next handler 474 logging faults to results files 474 logging memory management information 474 logging registers 474 logging system information 474 logging task list 474 logging the calling stack 474 faultrap.log 474 faulttrap.bmp 474 file extensions default for include files 479 file history modifying size of 458 File menu 457461 description 447 FileReadValue function 194 files closing 457 creating backups 478 creating new 458 list of loaded 479 list of, by operation 458 modifying size of history 478 opening 459 opening new 458 printing 459 property sets 121 saving 461 saving all 461 saving automatically 147 saving before running 478 saving before running, option for 147 saving object 461 saving with different name 461
setting default include file 483 view-only 461 See also include files, results files, scripts, suites, test frame file, test plans, initialization files. Find dialog 453 Find menu item (Edit) 453 Find Next Difference menu item (Results) 154, 455 Find Next Mark menu item (Testplan) 513 icon equivalent 245 Find Next menu item (Edit) 454 finding error messages 152 errors in scripts 152 line number 455 text 453 Finish Function command 177 Finish Function menu item (Debug) 451 float data type (C) 33 fonts setting in editor 476 footers printing 459 4Test language setting editor colors for 476 4Test statements recording 488490 functional testing 340 functions finding return value of 516 finishing execution of 451 online documentation for 327, 462 passing arguments to DLL 426 using Step Into on 452 using Step Over on 452 viewing calls to 515 fuzzy verification 118
Goto Script menu item (Testplan) 513 Goto Source menu item (Results) 153 GPFs 217 graphical controls supporting 320 tags, editing 93 graphing reports 164 group descriptions changing level of 54 default color of 54 definition 50 GUI conditional compilation, based on 271 hierarchy of objects in 91, 92 inserting template of hierarchy 58 porting tests to another 267281 specifiers 89, 267 testing objects in 267
H
headers printing 459 Help About menu item 462 Help menu 462 description 447 hHost constant 366 Hide Summary menu item (Results) 504 hidecalls keyword 312 hiding Runtime Status window 485, 510 hierarchical outline structure 51 history size of on File menu 478 size of results 484 host machine definition 27, 339, 351 enabling extensions on 69 executing system commands 366 Hungarian notation xiii
G
General menu item (Options) 477 General Options dialog 478 for setting initialization file 262 general protection faults 217 GetGUIType function 279 global variables 517 basic concept 350 protecting access to 367 with GUI specifiers 270 Global Variables menu item (View) 517 Go to Line dialog 455 Go to Line menu item (Edit) 455
I
icons arranging 518 for changing indenting 54 for marking 245 identifiers changing 93 definition 78, 84, 496 finding current value of 516 of main window 84 predefined 306 recording 499501
Users Guide
547
setting default 499 if statement versus switch 279 include files automatically loading 483 conditionally loading 272 creating 458 default extensions for 479 default path of 483 definition 495 for DLL support 425 object files 134 startup 517 Include menu 447, 463464 include statement adding to master plan 262 indentation changing 35, 54 importance of 50 index number definition 498 inheritance definition 29 property sets and 123 subplans and 261 inherited symbols identifying 232 initialization files 237, 247 partner.ini 448 sharing 262 Insert Template menu item (Testplan) 513 inserting. See pasting. int data type (C) 33 international characters, applications containing 284 internationalized applications 283
keyboard event delay option 466 keystrokes used to close dialog box window 468 keywords 523 setting editor colors for 476
M
machine handle operator 366 main function 192 main window recording declarations for 78 Manager Class widgets, ignored by default 95 managing testing progress 511 manual keyword to testcase statement 65, 248 manual tests including in plan 65 mapping custom classes to standard classes 94 mapping classes 318, 475, 495 margins adjusting 459 Mark All menu item (Testplan) 514 Mark by Named Query menu item (Testplan) 247, 514 Mark by Query dialog 514 Mark by Query menu item (Testplan) 247, 514 Mark Failures in Plan menu item (Results) 156, 504 Mark menu item (Testplan) 513 icon equivalent 245 marked tests printing 246 running 147, 509 marking by query 246 test failures 504 testplans 243246 tests 245, 513, 514 ways of 235 marking commands comparison of 244 marks appearance 243 definition 235 finding next 513 masks commands for 403 creating and applying 417421 definition 402 saving 417 master plan adding include statements to 262 definition 259 dividing testplan into 259 editing in multi-user environment 262 MDICLIENT class 319
L
labels use in tag 93 LAN Manager networks 359 languages handling multiple 283286 layering windows 518 level 1 tests definition 106 level 2 tests definition 106 Library Browser about 327 modifying 329 saving Help files in options set 479 Library Browser menu item (Help) 462 lie continuation character 35 line charts 164 lines breaking 35 finding 455 moving 487, 488 list boxes supporting custom 325 list outline structure 51 load testing 341 local values of symbols 228 local variables 517 Local Variables menu item (View) 517 localizing tags 284 locating. See finding. location suffix adding to tag 320 locations, windows recording 501 locks releasing 463 setting 262, 463 login windows, handling 297 long data type (C) 33 low level recording 481 lsCloseConfirmButtons variable 468 lsCloseDialogKeys variable 468 lsCloseWindowButtons variable 468 lsCloseWindowMenus variable 468
286
intersection method of combining queries 250 Invoke method 96 IPX/SPX platforms for 352 ITE (Integrated Test Environment) 27,
339
J
Java enabling extension 68 jumping to a difference in a bitmap 415
K
keyboard accelerators 54
548
Users Guide
measuring testplan completion 258 memory clearing 451 menu hierarchy 91, 92 GUI-specific 275 menu items picking menu before checking menu item 471 used to close windows 468 menus 447520 bitmap tool 403405 Breakpoint 449450 Debug 447, 450452 Edit 447, 453456 File 447, 457461 Help 447, 462 Include 447, 463464 Options 448, 464486 Outline 448, 486488 Record 448, 488502 Results 448, 502507 Run 448, 508510 Testplan 448, 511515 View 448, 515518 Window 448, 518520 window declarations for 90 Merge menu item (Results) 505 merging results sets 160 message boxes generic 92 recognizing extra pushbuttons in 99 window declaration for 92 Method menu item (Record) 492 methods adding 304 adding for derived classes 323 calls to, in results file 485 changing 304 defining 306 defining for a GUI object 96 definition 29 deriving new from existing 307 online documentation for 327, 462 recording 492 redefining 307 using cross-platform names 278 verifying coordinates of 467 VerifyProperties 118 Microsoft Windows messages to support custom list boxes 325 networking tips 359 Microsoft Word converting outline to testplan via 55
minimizing window during run 485 minus sign (-) meaning of 486, 516, 517 modal dialog definition 105 modifying attributes 237240 window declarations 96 Module menu item (View) 517 modules viewing in debugger 179, 517 mouse event delay option 466 mouse movements ignoring while recording 481 Move Left menu item (Outline) 487 Move Right menu item (Outline) 487 moving lines 487 multiple tags definition 84 disabling 88 what to do on platforms that dont support them 145 multitag statement 86 multithreaded script writing 363 multithreading 376
Next Error Difference menu item (Results) 505 Next menu item (Window) 519 Next Result Difference menu item (Results) 505 nongraphical controls supporting 323 null values in symbols 524
O
object files 134 saving 461 saving during compilation, option 483 setting path for 483 object tests definition 106 object-oriented tools concepts of 304 objects associating data with 98 custom See custom objects supporting custom 323 test implementation 267 online documentation 462 Online Users Guide 41 full-text search requirements 41 how to use 41 troubleshooting 41 Open All menu item (Include) 463 Open menu item (File) 459 Open menu item (Include) 463 Open Set menu item (Options) 479 opening existing file 458, 459 new file 458 previously opened windows 519 set of options 479 operators scope resolution 307 options agent 464 class map 475 closing set of 475 debugging 485 editor colors 476 editor font 476 general system 477 opening set of 479 for printing files 459 recorder 480 runtime 482486 Options menu 464486 description 448 Options pushbutton on Extensions
N
named queries versus unnamed 246 naming attributes, rules for 237 navigating through a testplan 256 NetBIOS/NetBEUI naming Agents 359 platforms for 352 Network dialog 28 network testing 351370 concurrent client/server tests 374 kinds of 339 Microsoft Windows tips 359 scenarios 337, 351 serial client/server tests 373 networking enabling 357, 359 naming Agents on target machines 359 naming PC Agents 355, 359 networking protocols 352 configuring for 354 selecting 354 New dialog 458 New menu item (File) 458 New pushbutton on Extensions dialog 475
Users Guide
549
dialog 473 OSF/Motif if Ctrl+Alt doesnt work 113 Other Extensions check boxes on Extensions dialog 473 outline editor. See Visual 4Test. Outline menu 486488 commands for changing indents 54 description 448 outline of testplan 4858 changing levels in 54 default colors 54 developing in Microsoft Word 55 structuring 5055 template of GUI hierarchy as 58 owner-draw list boxes 529530
P
parallel processing features 363 parallel statement 364 parent statement 92 partner executable program syntax 443 partner.ini file 448, 465 Pass/Fail Report menu item (Results) 505 Pass/Fail reports 162 Paste menu item (Edit) 455 pasting text 455 path setting default for include file 483 peak load testing 342 performance testing 342 period (.) meaning of 516, 517 plain editor starting 456 platforms configuring for Agent 354 porting applications to other 267
281
possible protocols (table) 353 specifiers 89 plus sign (+ ) meaning of 487, 516, 517 polymorphism definition of 308 popup menus events used to invoke 471 port number conflicts 356, 357, 358 portability 274 of controls 275, 276 porting tests to other GUIs 267281 pound sign character 150, 233, 525 PowerBuilder
accessing internal names 499 enabling extension 69 Previous menu item (Window) 519 Primary Extension field on Extensions dialog 472 Print Agent Calls option 177 Print dialog 459 Print menu item 459 Printer Setup dialog 460 Printer Setup menu item (File) 460 printing command for 459 marked tests 246 reports 505, 511 priority of threads 365 progress report 511 properties defining for a class 308 definition 32 online documentation for 327, 462 properties, verification 115 and attributes 114 property sets combining 123 creating 121 definition 116 deleting 125 editing 124 file names 121 how defined 121 initialization file 121 predefined 116 where stored 121 Property Sets dialog 122 protocols configuring for 352 networking 352 selecting 354 supported (table) 353 pushbuttons extra, in message boxes 99 PVCS using 429 PVCS Options command, Options menu 433 pvcsqap.cfg file 431
storing 247 query marking commands compared 246 question marks in unset symbols 230, 249 wildcard character 248 quitting debugger 450
R
raise statement 217 read-only text color of 261 Record Actions dialog 488, 489 Record Application State dialog 188,
490
Record menu 488502 description 448 Record Method dialog 492 Record Status dialog 491 Record Testcase dialog 110, 494 Record Window Declarations dialog 495 Record Window Declarations Options dialog 87, 497, 498 Record Window Identifiers dialog 499 Record Window Locations dialog 501 recording application operations 363 application states 185189, 490
492
BaseState method 298 bitmap verification statement 411 4Test statements 488490 methods 492 testcases 109133, 494 verification statements 112 window declarations 78, 318, 495499 window identifiers 499501 window locations 501 without declarations 138 recording statement 133 recovery system 289299, 348, 380 modifying default 298, 492 red plus sign 152 Redo 455 refreshing local subplan copy 264 Release 1.x compatibility options for 469 Release function 368 Release Lock menu item (Include) 463 releasing subplan lock 463 Remove pushbutton on Extensions dialog 475 removing applications from Extensions
Q
QAP_VERIFY_KEY environment variable 113 queries combining 250 defining 246250, 514 deleting 253 editing 252
550
Users Guide
dialog 475 rendezvous statement 364 repeating last find or replace operation 454 Replace All Again menu item (Edit) 454 Replace dialog 455 Replace menu item (Edit) 455 Replace Next menu item (Edit) 454 replacing text 455 reporting results 370 reports charting 164 Completion 258, 511 graphing 164 Pass/Fail 162, 505 using results export as basis for 158 using results extract as basis for 156 reraise statement 213 Reset menu item (Debug) 451 resetting variables 451 results 370 reporting distributed 370 sort orders 370 results bitmap definition 410 setting 413 results files 148162 compacting 503 comparing 161, 505 converting into a plan 232 definition 149 deleting 503 exporting information from 158 extracting from 504 extracting information from 156 file name of 509 hiding summary 504 including descriptions in 525 logging faults to, option 474 merging 505 merging results sets 160 method calls in, option 485 navigating to script from 513 reducing size of 169 results sets in 151 running associated testcase from 510 selecting 506 sending to SilkRadar 158 setting default frequency of saves 484 setting default number of 484 setting default path 484 setting editor colors for 476
showing summary 506 tags and method calls, options 486 Results menu 502507 description 448 results sets commenting 151 deleting 151 merging 160 selecting 151 Run All Tests menu item (Run) 509 Run Marked Tests menu item (Run) 509 Run menu 508510 description 448 Run menu item (Debug) 451 Run menu item (File) 460 Run menu item (Run) 509 Run To Cursor command, Debug menu 451 running application states 508 files, list of 458 marked tests 147 scripts 460, 509 suites 460, 509 testcases 144 testplans 146, 460, 509 Runtime menu item (Options) 482
486
runtime options closing 475 opening set of 479 saving set of 486 Runtime Options dialog 482 test frames and 80 Runtime Status window hiding and showing 510 remembering position 485 showing or hiding, option 485
S
Save All menu item (File) 461 Save As menu item 461 Save menu item (File) 461 Save menu item (Include) 464 Save New Set menu item (Options) 486 Save Object File menu item (File) 461 saving attributes 237 files before running 478 queries 247 subplans 264, 464 saving files 461 automatically 147
before running, option for 147 bitmaps 410 creating backups 478 masks 417 options set 486 Scan menu item (Differences) 415 bitmap tool 404 sCmdLine constant 90 scope declaration 368 scope of statements 50 scope resolution operator 307 script files collapsing code in 487 expanding code in 487 script statement inserting via testcase recording 136 specifying 512 using in query 248 ways to insert 60 ScriptEnter function 300 ScriptExit function 300 scripts aborting run while debugging 450 calling DLLs from 423427 collapsing code in 486 compiling 508 compiling and running 509 concurrent 376 controlling order of execution 192 creating file for 458 creating in ASCII text editor 35 debugging 171 entering debugger 509 expanding code in 487 finding errors in 152 multithreaded 376 navigating to 513 object files 134 passing arguments to 147 porting 267 running 460 running in debugger 451 saving 133 viewing output of 179 scrolling windows Agent options for 471 sDir constant 90 searching. See finding. Select menu item (Results) 506 Select Results dialog 506 Select Window dialog 519 selecting results files 506 SEMAPHORE data type 368 semaphores code example 370
Users Guide
551
synchronizing thread with 368 Send to Silk Radar menu item (Results) 506 serial testing of client and server 373 server initializing 373 Set attributes adding and removing assignments 242 SetMachine function 366, 373 setting colors in editor 476 locks 262, 463 runtime options 482486 setting statement 87 setting Year 2000 transformations at runtime 484 settings, printer 460 setup stage of testcase recording 112 share key word 367 shareable variables 367, 368 short data type (C) 33 Show AppState status window check box 111, 494 Show menu item (Differences) bitmap tool 404 Show Status menu item (Run) 510 Show Summary menu item (Results) 506 showing Runtime Status window 485, 510 SilkRadar, sending results to 158 slash characters 525 meaning of 487 smoke tests definition 106 spawn statement 364 specifiers, platform specific 89 spreadsheets, exporting results to 158 standard classes mapping custom classes to 475 starting applications on different GUIs 273 plain editor 456 SilkTest or QA Partner 443 Visual 4Test 456 startup.inc 517 statements default color of 54 doubling as descriptions in results file 525 identifying inherited 257 parent 92 scope of 50 syntax of 524
See also assignment statements. STATIC class 319 statistics for bitmap comparison 416 Step Into command 176 Step Into menu item (Debug) 452 Step Over command 177 Step Over menu item (Debug) 452 stopping testcases 145 stress testing 341 strings replacing tags with variable 284 setting editor colors for 476 stripe in margin meaning of 243 subplans accessing most recent 264 closing 463 creating 262 default color of 54 definition 259 dividing testplan into 259 inheritance and 261 opening 463 opening from master plan 262 saving 264, 464 setting lock for 463 suites compiling and running 509 creating file for 458 running 460 summary, results hiding 504 showing 506 supporting graphical controls 320 nongraphical controls 323 suppressing display of application state status window 111 switch statement versus if 279 symbols box icon 507 bullet 447 definition 226 identifying inherited 232, 257 indicating unset 249, 524 local values 228 meaning of color in 232 minus sign 486, 516, 517 period 516, 517 plus sign 516, 517 sideways triangles 451 slashes 487 syntax conventions for 524 unknown values 230
using in queries 249 syntax partner executable program 443 Windows bitmap executable program 406
T
tabs setting width of 478 tags adding location suffix to 320 changing 93 choosing 87 definition 84, 496 editing 496 forms for 93 GUI-specific 274 how generated 87 internationalizing 284 list of with method calls, in results files 486 of main window 84 setting default 498 specific to a class 87 verifying as unique identifier 467 target machines aborting program execution on 507 connecting to 362 definition 27, 339, 351 driving applications on 366 Runtime Status window on 510 testing on single 372 testing serially on multiple 372 taskbar Agent menu 28 TCP/IP naming PC Agents 356 platforms for 352 template definition 58 temporary breakpoints 175 terminating script run 450 threads 365 test data specifying 512 test descriptions changing level of 54 default color of 54 definition 49 linking to statements 60 test design 371 for multi-application environment 381 for single application 380
552
Users Guide
required code 381 test frames definition 18 recording 77 Runtime Options dialog and 80 Test Plan Completion Report dialog 511 test plans setting editor colors for 476 test requirements devising 108 test results 370 Testcase menu item (Record) 110, 494 Testcase menu item (Run) 510 testcase statement inserting via testcase recording 136 specifying 512 specifying symbols as arguments to 228 using in query 248 ways to insert 60 TestcaseEnter function 300 TestcaseExit function 300 testcases associated with tests, identifying 511 avoiding failure of 155, 506 controlling order of execution 192 creating data-driven 190233 debugging 171 design principles for 106 making portable 267 mouse movements while recording 481 reading data from files 194 recording 109133, 494 recording and linking to testplan 136 running 144, 510 stages in design of 109 stopping 145 structuring 108 testdata statement 223 specifying 223, 512 syntax 225 using in query 248 testing across GUIs 267 across networks 337, 351 client and server serially 373 concurrency 340 concurrent applications 374 configuration 340 functional 340 GUI objects 267 load 341
multiple clients concurrently 374 multiple clients in parallel 376 multiple targets serially 372 nonstandard objects 317 on one remote target 372 peak load 342 performance 342 randomizing testing activity 376 strategies 362 stress 341 volume 342 Testplan Detail dialog 241 as a navigation tool 256 using to insert script and testcase statements 60 Testplan menu 511515 description 448 testplan.ini 237, 247, 262 testplans adding comments to 55, 525 adding data to 221233 attributes 236 compiling and running 509 creating 4766 creating from results file 232 creating in ASCII text editor 35 documenting manual tests in 65 finding marks in 513 importing from Microsoft Word 55 marking 243246 measuring completeness of 258 navigating through 256 navigating to from results file 153 navigating to script from 513 running 146, 460 tests associated with testcases, identifying 511 associating with scripts, testcases, and data 512 documenting manual 65 identifying inherited symbols in 232 marking 513 marking all 514 marking failed 504 pass/fail report on 505 running 509 running marked 147 running on another machine 482 specifying data shared by multiple 226 specifying data unique to one 223 stopping 145 unmarking 514 unmarking all 515
text copying 453 creating file for 458 cutting 453 deleting 453 pasting 455 replacing 455 searching for string 453 text editor, using ASCII editor 35 text fields supporting custom 323 this keyword 307 threads and semaphores 368 blocking 364, 368 defined 364 priority 365 spawning 364 starting 367 synchronizing 368 terminating 365 understanding 364 usage example 389 Tile Horizontally menu item (Window) 519 Tile Vertically menu item (Window) 520 tiling window horizontally 519 vertically 520 TimerCreate function 374 TimerStop function 374 timing options for agent 466 Toggle menu item (Breakpoint) 450 tool bar displaying by default, option 478 displaying help for, option 478 testing 320 tool bar icons for marking 245 ToolTips displaying 478 Transcript menu item (View) 518 transcript window 518 description 451 transcript, debugging 177 Transpose Down menu item (Outline) 488 Transpose Up menu item (Outline) 488 transposing lines 488 trapping faults 217, 474 capturing desktop bitmap 474 chaining to next handler 474 logging memory management information 474
Users Guide
553
logging registers 474 logging system information 474 logging task list 474 logging the calling stack 474 triangles, sideways meaning of 451
recording statements 112, 411,
492
using attributes 130 using bitmaps 126 using methods 128 using properties 115 verification options for agent 467 Verify function 211 verify stage of testcase recording 112 Verify Window dialog 114, 117 if Ctrl+Alt doesnt invoke it 113 VerifyProperties method statement added to script 118 syntax 118 VerifyValue method 194 version control 429 Version Manager, PVCS 430 version number of Agent 29 version of product finding out about 462 View Differences menu item (Results) 507 View menu 515518 description 448 viewing actual versus expected values 455 breakpoints 515 call stack 515 error information in transcript window 518 variables 178 view-only files 461 Visual 4Test outline editor and 35 printing and 459 starting by default 478 Visual 4Test menu item (Edit) 456 Visual Basic accessing internal names 499 enabling extension 69 volume load testing 342
U
Uncomment Block menu item (Outline) 488 Undo 456 undoing actions 456 uninitialized variables 178 union method of combining queries 250 Unmark All menu item (Testplan) 515 icon equivalent 245 Unmark menu item (Testplan) 514 icon equivalent 245 unmarking tests 514, 515 unnamed queries versus named 246 unset variables 178 unsigned data types (C) 33 Update Expected Value menu item (Results) 506 when to use 155 Use Files field (Runtime Options dialog) 483 Use Files field, Runtime Options dialog 80 Use Path field (Runtime Options dialog) 483 use statement Use Files field and 483 Use Path field and 483 using to assign attributes 241
V
Variable prefixes xiii variables adding to declaration 98 clearing 451 collapsing data about 516 finding value of identifier 516 global, finding value of 517 local, finding value of 517 replacing tags with string 284 shareable 367 uninitialized 178 viewing 178 verification cleanup stage 132 fuzzy 118
W
warnings skipping 152 Web applications Agent options for testing 471 wildcard characters 248 wildcards, using 123 window declarations benefits 67, 77 definition 495 editing 495
for dialogs 91 for menus 90 for message box 92 for multiple applications 384 making portable 267 mapping classes 318 modifying to map custom classes 318 modifying to support graphical controls 320 recording 78, 495499 syntax of 83 variables in 98 when to modify 96 Window Declarations menu item (Record) 495499 window ID definition 498 use in tag 93 Window Identifiers menu item (Record) 499501 when to use 139 window identifiers. See identifiers. Window Locations dialog 501 Window Locations menu item (Record) 501 when to use 139 window locations. See locations. Window menu 518520 description 448 windows arranging icons in 518 Bitmap Comparison Statistics 416 closing all 518 Differences (bitmap tool) 414 displaying next 519 displaying previous 519 Expression 516 history of 519 layering 518 list of open 519 minimizing, option 485 moving, tolerance to use when 470 options for verifying attributes of 467 recovery system closes all 292 resized, option for tolerance of 470 retry interval option for 466 Runtime Status 510 scrolling, Agent options for 471 showing or hiding Runtime Status 485 tiling horizontally 519 tiling vertically 520 timeout option for 466 Transcript 518
554
Users Guide
verifying 118 Zoom (bitmap tool) 414, 416 wInvoke constant 96 Word, Microsoft importing testplans from 55 workspace setting general options for 478 wStartup constant 82
X
X meaning of 487 X Window System changing verification keystroke 113
Y
Year 2000 compliance date transformations 195 setting Year 2000 transformations at runtime 484 testing converted applications 195
Z
Zoom menu item (Differences) bitmap tool 404 Zoom window 414 capturing 416 zooming areas of difference in bitmaps 414
Users Guide
555

SilkTestUserGuide

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SilkTestUserGuide

Uploaded by

Copyright:

Available Formats

users guide

the e-business management company version

(c) Copyright 1992 2001 Segue Software, Inc.

Warranties and Disclaimers

Where this manual fits in xi Typographical conventions xiii

Part I The Basics

Enabling Extensions for Applications Under Test

Recording a Test Frame

Why window declarations make your tests robust 78

Designing and Recording Testcases

Running Tests and Interpreting Results

Using the Debugger

Part II Enhancing Your Tests

Adding Data to a Testplan

Categorizing and Marking Testplans

Working With Large Testplans

Part III Cross-Platform Testing

Supporting Internationalized Applications

Part IV Customizing SilkTest

Extending the Class Hierarchy

Supporting Custom Objects

Modifying the Library Browser

Part V Testing Client/Server Applications

Configuring SilkTest for Client/Server Testing

Implementing Client/Server Testing

Part VI GUI-Specific Tips and Tools

Calling Windows DLLs From 4Test Scripts

Using PVCS with SilkTest

Part VII Command Reference

Part VIII Appendices

Where this manual fits in

PREFACE Where this manual fits in

4Test Language Reference

Contains descriptions of SilkTests 4Test language.

PREFACE Typographical conventions

A script file is script: myscript.t Enter myscript.t

Type install-dir\tut.t Record/Testcase

Brackets [ ] Ellipses ( ... ) Indentation Variable prefixes (i, b, n, w, ...)

PREFACE Typographical conventions

This part contains the following chapters:

17 47 67 77 101 141 171

The automated testing process

1 OVERVIEW The automated testing process

Recording a test frame

1 OVERVIEW Getting started

Using the QuickStart Wizard

1 OVERVIEW Getting started

The wizard opens.

1 OVERVIEW Getting started

1 OVERVIEW Getting started

1 OVERVIEW Getting started

10 Click Next twice. You see the following panel.

1 OVERVIEW Getting started

What has been recorded in the testplan

Using the wizard from a testplan

1 OVERVIEW Getting started

Getting started testing Web applications

1 OVERVIEW The anatomy of a basic testcase

The anatomy of a basic testcase

Testcase keyword Object-oriented commands

1 OVERVIEW SilkTest architecture

The built-in recovery system