Testing Tools

Agenda
● The benefits and risks of using testing tools
● Selection criteria for testing tools
● Tools for specification management
● Defect management tools
● Test repositories
● Additional tools
Potential benefits of using testing tools
● Automation - reduction of repetitive activities
● Increase repeatability (eg performing tests)
● Objective evaluation of measures (eg covering the code with tests)
● Easier access to metrics
Potential risks of using testing tools
● Unrealistic expectations - the tool will give us benefits without any work
input
● Unrealistic implementation dates of the tool (training, data migration,
infrastructure problems, pilot project)
● Underestimation of the time needed to achieve noticeable benefits
● Underestimation of the time needed to maintain the tool and related
artifacts
● Over-reliance on the tool
● Poor quality of producer support
● Cessation of support for the tool (eg due to the end of the project)
● Insufficient compatibility with the operating system or other tools
Choice - general criteria for tool evaluation
● Assessment of the possibilities to improve the testing process by
implementing tools
● Defining the requirements that particular tools should meet
● Proof-of-concept
● Evaluation of the software manufacturer
● Estimation of implementation costs (infrastructure, training etc.)
● Estimate the potential benefits
Choice - a pilot project
● Choosing a project on which we “test” tools
● Getting to know the tool and its possibilities
● Assessment of tool compatibility with our process
● Possible process adjustment or tool change
● Determining the use of the tool
● Launch of a pilot project
● Analysis of comments from team members and possible process /
configuration change
● Tool evaluation
Tools for specification management
● IBM Doors
● Siemens Polarion Requirements
● Enterprise Architect
● Confluence
● Jira (+ Requirements for JIRA (R4J))
● Word / Excel / Powerpoint
Selection criteria for specification management
tools
● What is the form of specification in our organization?
○ Detailed description of each functionality
○ Use case
○ User story
● The degree of formalization of the project or documentation
● Customer requirements for the tool
● How many people work on the specification?
● Integration with already implemented tools
● Additional built-in tools:
○ Re-use of requirements for various functionalities
○ Grouping of requirements
○ Maintenance support
○ Possibility of linking requirements for testing / measuring coverage of
requirements
○ Comparison of specific versions of requirements
○ Version requirements / creating baselines
IBM Doors
Siemens Polarion Requirements
Enterprise Architect
Confluence
Jira
Jira + Requirements for Jira
Defect management tools - selection criteria
● Customer requirements
● Compatibility with IDE, test repository, tools for specification management
● Generating reports
● Expendable with additional plugs
● Ability to define your own statuses, workflow
● Reporting time spent on fixing the error
● Adding custom fields to reports
● Export to CSV, Excel
Defect management tools (Bug trackers)
● Jira
● YouTrack
● IBM ClearQuest
● Mantis
Jira - bug report
Jira - simple workflow
YouTrack
ClearQuest
Mantis
Tests repository
● Quality Center
● Polarion
● Zephyr
● X-Ray
● TestLink
● TestRail
Test repositories - selection criteria
● Customer requirements
● Possibility of linking bug reports and requirements
● Tracking progress
● Prioritization of tests
● Grouping of tests
● Change history
● Versioning
● Ability to add attachments
Quality Center
Polarion QA
Zephyr
X-Ray
TestLink
TestRail
Tools for software development management
cycle
● Integrated tools for managing specification, defects and tests
● They provide additional benefits, eg:
○ Planning and monitoring of project tasks
○ Support for Agile Project Management
○ Generation of change logs
Tools for software development management
cycle
● IBM Jazz
● Siemens Polarion ALM
● Atlassian:
○ Jira,
○ Confluence
○ Zephyr / X-Ray
○ Draw.io
Additional tools
● Requirements modeling tools:
○ Enterprise Architect
○ Draw.io
● Code repositories
○ SVN
○ GIT
■ Bitbucket
■ GitHub
● Continuous Integration:
○ Jenkins
○ TeamCity
○ Bamboo
● Test data generators
● Password generators
● Screenshots, screen recording