You are on page 1of 5



A Test Manager must:

ensure that all tools introduced in the organization add


ensure a positive ROI for new tools

perform a cost-benefit analysis before acquiring or

building a tool

consider both recurring and non-recurring costs when

calculating ROI including covering at least monetary,

resources, time and risks that may impact the value of

the tool

Non-recurring costs

Defining tool requirements to meet the objectives and


Evaluating and selecting the correct tool and tool


Purchasing, adapting or developing the tool

Performing the initial training for the tool

Integrating the tool with other tools

Procuring the hardware/software needed to

support the tool

Recurring costs

Owning the tool (licensing, maintenance costs of the

tool and of the tool artifacts, ongoing training, etc.)

Porting the tool to different environments

Adapting the tool to future needs

Improving the quality and processes to ensure optimal

use of the selected tools

Opportunity costs

usually inherent in the tool

The time spent on acquiring, administering, training

and using the tool could have been spent on actual

testing tasks

More testing resources may be needed up front until the

tool goes on line

The test team usually used more than one tool and the ROI

will be obtained by evaluating the entire tool set.

For larger projects or for organizations, a long term and

comprehensive test tool strategy is recommended.

Nothing comes without risk and a Test Manager should

also consider the following when assessing ROI:

Immaturity of the organization (not ready)

Artifacts created by the tool may be difficult to

maintain, requiring multiple revisions when the

software under test is changed

Reduction of Test Analysts involvement in testing tasks

may reduce the value of testing

A Test Manager must:

look at the benefits that may accrue from the

use of the tool

consider a prospective tool from several

different viewpoints like




Benefits that may accrue can be a REDUCTION in

Repetitive work

Test cycle time (automated regression)

Test execution costs

Human error in different phases of testing because:

Test data may be more valid using data generation


Test result comparisons may be more accurate using

comparison tools

Test data entry may be more correct by entering with

a scripting tool

Effort required to access information about tests:

Tool generated reports and metrics

Reuse of test assets such as test cases, test scripts

and test data

and even an INCREASE can still be a benefit like:

Test scope increase (for regression testing)

Specialized testing increase for the areas which were

not possible without tools (performance, load, etc.)

When looking from other viewpoints at a tool prospect,

the ones we should most likely consider are:

Business viewpoint (Positive ROI is required)

we need to assure that the existing and non test tool

work together with the new test tool

look at improving the processes and inter

connectivity between tool usage

Project viewpoint (Tool must be effective)

The tool may require an appreciable amount of time

in order to start earning positive ROI

In many cases, ROI may occur in the second release

or during maintenance rather than during the initial

project when it was implemented

Tool user viewpoint (must support in doing tasks)

learning curve must be considered to ensure the

users will be able to learn the tool quickly with

minimal stress

When first introduced, test tools require training and

mentoring for the users

In order to assure that all viewpoints are

considered, it is important to create a

ROADMAP for the test tool introduction.

1. Assess organizational maturity

2. Identify requirements for tools

3. Evaluate tools

4. Evaluate the vendor or service support

5. Identify internal requirements for

coaching and mentoring for the tool

6. Evaluate training needs considering

the current test team’s test skills

7. Estimate cost-benefits (ROI)