Selecting and

Installing
Software
Testing Tools
- Aarti Gokhale.
Testing Tools – The Hammers
of Testing
• New methods of testing have been
developed to improve testing
effectiveness and testing efficiency.
• Testing, like program development,
– Generates a lot of information
– Necessitates numerous computer
executions
– Requires co-ordination and
communication between testers
• Testing tools can ease the burden of test
production (e.g. producing the test data,
test scripts) , test execution, general
information handling and
communication.
• Testing techniques are few in number
and broad in scope; that means a
technique can be used in a wider range
of situations.
• Testing tools are, however, numerous
and narrow in scope; that means each
tool provides different capabilities; but
accomplish only a very specific aspect of
whole of the testing objectives.
• Testing tools enhance the
effectiveness and efficiency of testing.
Thus, they can be compared to using
a hammer to drive-in a nail. Without
the hammer, this would not be
possible.
• However, that is only one aspect of
testing tools. Most important is the
selection of proper testing tool or set
of tools, to test in a given situation.
Selection of proper testing tool affects
the effectiveness and efficiency of
testing, as well.
• Thus, if a testing tool does not
match the testing purpose well,
then instead of helping in the
testing, the testing tool can be
counter productive. It may not help
reducing the testing effort or
speeding it, but it may add more
work to testing or may not be able
to detect satisfactory number of
defects. This also can have serious
impact on waste of time and over
spending.
• E.g. for a very large nail to be
driven into a wood, a relatively
smaller hammer would increase
efforts and chances of accidents,
and causing frustrations rather
than any benefits. Also, otherwise,
for a small size nail, one must not
use a heavy hammer.
• The testing tools can make
desirable difference and efficiency
achieved depends on selecting the
proper tool. Hence, selection of
tools is a very important decision.
• Many of the testing tools have not been in
use because of the following reasons:-
– Some of them are highly specialized
(simulation)
– The cost of use is very high (symbolic
execution)
• The disciplined manual testing techniques
have been most useful. They can be used
without too much of capital investment.
However, they require firm commitment
and consistently disciplined testing
practices.
• Careful planning, clearly stated
testing objectives, precisely defined
tools, good management,
organized record keeping, and
strong commitment are necessary
for successful testing.
Selecting and Using the Test
Tools
There are four steps as follows:-
• Step 1. Matching the tool to its use
• Step 2. Selecting a tool appropriate
to the life cycle phase
• Step 3. Matching the tool to the
skill level of the tester
• Step 4. Selecting an affordable tool
Step 1. Matching the tool to its use
• A tool is to accomplish a task. Therefore,
more suitable the testing tool to the meet
the goal of the testing task, the more
efficient the testing process can be.
• A wrong testing tool not only decreases
the efficiency, but there is no assurance
that the testing objectives will be
achieved.
• The testing technique should be selected
prior to the tool.
• Matrix test tool Vs test techniques can be
used to narrow the choice of more
appropriate tools.
• Following testing tools are useful in all
the types of testing use: Checklist, Error
Guessing, and Inspections.
• For meeting the Error Handling purpose
of testing, the following testing
techniques are useful: Acceptance Test
Criteria, Boundary Value Analysis, Cause
Effect Graphing, Checklist, Correctness
Proof, Data Dictionary, Desk Checking,
Error Guessing, Exhaustive Testing,
Flowcharts, Inspection, Instrumentation,
Integrated Test Facility, System Logs,
Test Data, Test Data Generator
• For testing purpose of Controls, following
testing tools can be used:- Cause-Effect
Graphing, Checklist, Code Comparison,
Confirmation/Examination, Correctness
Proof, Desk Checking, Error Guessing,
Exhaustive Checking, Fact Finding,
Flowchart, Instrumentation, Integrated
Test Facility, Risk Matrix, Test Data, Test
Data Generator, walkthroughs.
• For testing Compliance, following
techniques are useful: Acceptance Test
Criteria, Checklist, Compiler based
Analysis, Confirmation/Examination,
Correctness Proof, Design Reviews, Error
Guessing, Fact Finding, Inspections, Peer
Review, System Logs.
Step 2. Selecting a tool appropriate to the life
cycle phase
• The testing type varies with the phase in
which the testing occurs. Just as the
methods change, so do the tools. So, it
becomes necessary that the testing tool
selected is appropriate for the life cycle
phase in which it is used.
• As the life cycle phases progress, the tools
shift from manual methods of testing to
more automated methods.
• In fact, some of the most productive testing
(testing that detects and removes
significantly potential defects entered into
the system and those which may be very
costly later) can occur during the early
phases of the life cycle using manual tools.
• During the Requirements Phase, the
following testing tools can be used:-
Checklist, Confirmation/Examination, Desk
Checking, Error Guessing, Fact Finding,
Flowchart, Inspections, Modeling, Peer
Review, Risk Matrix, Scoring,
Walkthroughs.
• During the Design Phase, the following
testing tools can be used:- Cause-Effect
Graphing, Checklist,
Confirmation/Examination, Correctness
Proof, Design-based functional testing,
Design Reviews, Desk Checking, Error
Guessing, Executable Specs, Fact Finding,
Flow Chart, Inspections, Modeling, Peer
Review, Risk Matrix, Scoring, Walkthroughs
• During the Programming Phase, the following
testing tools can be used:- Boundary Value
Analysis, Cause-Effect Graphing, Checklist,
Compiler-based Analysis, Coverage-based Metric
Testing, Data Flow Analysis, Desk Checking, Error
Guessing, Fact Finding, Flow chart, Inspections,
Peer Review, Symbolic Execution, Test Data,
Tracing, Walkthroughs
• During the Testing Phase, the following testing tools
can be used:- Acceptance Test Criteria, Boundary
Value Analysis, Checklist, Compiler-based Analysis,
Complexity-based Metric testing,
Confirmation/Examination, Correctness Proof,
Coverage-based metric testing, Data Dictionary,
Design-based Functional testing, Disaster Test,
Error Guessing, Exhaustive Testing, Fact Finding,
Inspections, Instrumentation, Parallel Simulation,
Peer Review, Snapshot, System Logs, Test Data,
Test Data Generator, Utility Programs, Volume
Testing
• During the Installation Phase, the following
testing tools can be used:- Acceptance Test
Criteria, Checklist, Confirmation/Examination,
Error Guessing, Fact Finding, Inspections,
Instrumentation, Parallel Simulation, Peer
Review, System Logs, Utility Programs, Volume
Testing
• During the Maintenance Phase, the following
testing tools can be used:- Checklist, Code
Comparison, Desk Checking, Disaster Test,
Error Guessing, Fact Finding, Inspections,
Instrumentation, Integrated Test Facility, Peer
Review, SCARF, System Logs, Test Data, Test
Data Generator, Tracing, Utility Programs
• The tester skills and overall
understanding of the testing tasks is
important in selecting the tool.
• User Skills: Requires individuals to have
– In-depth knowledge of the application
– In-depth knowledge of the business purpose
for which the application is being developed.
– General management skills used to achieve
the mission of user area,
– Knowledge of identifying and dealing with
user problems.
• Programming Skills: Requires following
skills:-
Understanding of computer concepts,
Flowcharting
– Programming skills in the languages used by
the organization
– Debugging and documenting computer
programs
• System Skills: Requires the following
skills:-
– The ability to translate user requirements into
computer system design specifications
– Flowcharting, Problem Analysis, Design
Methodologies, Computer Operations, General
Business Skills, Error identification, and analysis
in automated applications and project
management.
– Normally posses good programming skills.
• Technical Skills: These are as
follows:-
– Understands a highly technical
specialty
– Can exhibit a reasonable performance
at work at that specialty
– E.g. systems programming, database
administration, statistics, accounting
and operating software packages
Step 4. Selecting an affordable tool:
• Affordability of a tools deals with its
cost and expected time of
completion of testing using that
tool. Even the most desirable, but
very costly and/or time-consuming
tool, may not be recommended.
• Some times one may have to buy
special hardware, software etc. for
using a tool. This may add to costs.
• The prices may vary depending
upon the features provided,
reliability of the tools and above all
vendor policy.
• The costs could be classified as
High, Medium and Low, based on
how much do they vary for the
same set/ range of features.
• A tool may be time consuming, if it
requires specific training, and/or it
requires additional steps of actions
from a tester or it has any
unavoidable manual interventions
involved, etc. Also, if the tool
documentation is not of adequate
quality, the tester may take longer
to complete a testing task.
 Appointing Managers for
Testing Tools
• The objectives are as follows:-
– Assign someone accountability to oversee the
tool usage.
– Create a source of competency about how to
use the tool
– Provide a training ground for future managers
• Managing a tool involves budgeting,
planning, training, and related
managerial responsibilities.
• There are three steps in appointing a
tool manager, as follows:-
– Step-1: Tool Manager Selection
– Step-2: Assign the Tool Manager Duties
– Step-3: Limiting the Tool Manager’s
Tenure
Step 1: Tool Manager Selection
• Tool Manager Selection should look for
the following skills in the candidate:
Organization, Training abilities, Tool
proficiency, Managerial Qualities such as
planning, organizing, directing,
controlling, etc.
• In addition to the tool manager, an
assistant tool manager should also be
named for each tool.
• This individual will not have any direct
managerial responsibilities, but will
serve as a backup for the tool manager.
Step-2: Assign the Tool Manager Duties
• The Tool Managerial Duties are as
follows:-
– Assisting colleagues in the use of tools
• The tool manager should be available to
assist other staff members in the use of the
tool.
• This is normally done using a concept called
‘hot-line’.
– Tool training
• The initial tool training normally comes from
the vendor.
• However, additional tool training is the
responsibility of the tool manager.
– Tool Vendor Contact
• The tool manager would be the official contact
for the tool vendor.
• Questions from staff regarding the use of the
tool that can only be answered by the vendor
should be funneled through the tool manager
to the vendor.
– Annual Tool Plan
• The tool manager should develop an annual
tool plan complete with planned usage,
schedule, and resources needed to effectively
utilize the tool.
– Installing Tool Upgrade
• As vendors issue new versions of the tool,
the tool manager is responsible for ensuring
that those upgrades are properly
incorporated and that the involved parties
are made aware and trained, if necessary.

– Preparing Annual tool Report

• At the end of year, or planning period, the
tool manager should prepare for IT
management an overview of the use of the
tool during the year.
• This will require the tool manager to
maintain statistics on tool usage, problems,
costs, upgrades and so forth.
– Determining timing of tool replacements
• The tool manager, being responsible for a
specific software tool, should also be
responsible for determining when the tool is no
longer effective or when better tools can be
acquired to replace it.
Step-3: Limiting the Tool Manager’s Tenure
• It is recommended that an individual
serve two years as a manager for a
specific tool.
• Bringing in a new tool manager every two
years tends to revitalize the use of that
tool in the organization.
• The tool manager can be transferred to
manage another tool.
• In instances where tools are highly
specialized, very complex, it may be
desirable to keep an individual manager
for longer than a two- year period.