Software Testing and Quality Assurance

SEng4142

Chapter 2: Software Testing Process

3/16/22

Bit
Department of Software Engineering
 The Concept of a Process

• A process is a series of activities performed to fulfill a

purpose and produce a tangible output based on a given
input.

• Processes can be described and hence monitored and

improved. A process description must always include:
 A definition of the input
 A list of activities—the procedure
 A description of the output

03/16/2022 2
 The Concept of a Process

For a more comprehensive and more useful process

description the following information could also be included:
Entry criteria—What must be in place before we can start?
 Purpose—A description of what must be achieved ?
 Role—Who is going to perform the activities?
 Methods, techniques, tools—How exactly are we going to
perform the activities?
 Measurements—What metrics are we going to collect for the
process?
Templates—What should the output look like?
 Verification points—Are we on the right track?
 Exit criteria—What do we need to fulfill before we can say that
we have finished?

03/16/2022 3
The Concept of a Process
The inputs on which this process is based are:
• Test strategy
• Project plan
• Master test plan
• Information about how the testing is progressing
The activities are:
• Test planning and control
• Test development
• Test analysis and design
• Test implementation and execution
• Evaluating exit criteria and reporting
• Test closure activities

03/16/2022 4
The Concept of a Process

The output consists of:

• Level test plan
• Test specification in the form of test conditions, test design,
test cases, and test procedures and/or test scripts
• Test environment design and specification and actual test
environment including test data
• Test logs
• Progress reports
• Test summary report
• Test experience report

03/16/2022 5
 Processes Dependency

• The input to a process must be the output from one or

more proceeding process(es)
• The output from a process must be the input to one or
more other processes—even the final product, which is the
input to the maintenance process.

03/16/2022 6
 Generic Test Process
Testing is a process rather than a single activity. Testing must
be planned and it requires discipline to act upon it. The quality
and effectiveness of software testing are primarily determined
by the quality of the test processes used.
The activities of testing can be divided into the following basic
steps:
1. Planning and Control
2. Analysis and Design
3. Implementation and Execution
4. Evaluating exit criteria and Reporting
5. Test Closure activities
03/16/2022 7
 Generic Test Process
• The generic test process is applicable for each of the dynamic
test levels to be included in the course of the development
and maintenance of a product. So the process should be used
in testing such as:
• Component testing
• Integration testing
• System testing
• Acceptance testing

03/16/2022 8
 Generic Test Process
• The generic test process is iterative—not a simple
straightforward process.

Figure 1. generic test process

03/16/2022 9
 Generic Test Process
• The test activities need not be performed in strict sequential
order
• The generic test process is iterative—not a simple
straightforward process.
• The resulting iterations may be:
1. The defect is in the test object.
2. The defect is in the test procedure
3. More test cases must be specified to increase
coverage, and these must then be executed.
4. The exit criteria are relaxed or strengthened in the
test plan.

03/16/2022 10
 Generic Test Process
1) Planning and Control
Test Planning : Test planning involves producing a document that describes
an overall approach and test objectives. It involves reviewing the test basis,
identifying the test conditions based on analysis of test items, writing test
cases and Designing the test environment. Completion or exit criteria must
be specified so that we know when testing (at any stage) is complete.
•Purpose
 To determine the scope and risks and identify the objectives of
testing.
 To determine the required test resources like people, test
environments etc.
 To schedule test analysis and design tasks, test implementation,
03/16/2022
execution and evaluation. 11
 Generic Test Process

Control
This is the activity of comparing actual progress against the
plan, and reporting the status, including deviations from the
plan. It involves taking actions necessary to meet the mission
and objectives of the project.

03/16/2022 12
Generic Test Process
2) Analysis and Design
The purpose of the test analysis and design activities is to produce test
designs with test conditions and tests cases and the necessary test
environment based on the test basis and the test goals and approach
outlined in the test plan.
Analysis and Design

Test analysis and Test Design has the following major tasks:
 To review the test basis. The test basis is the information on which test
cases are based, such as requirements, design specifications, product risk
analysis, architecture and interfaces
 To identify test conditions
 To design the tests
 To design the test environment set-up and identify the required
infrastructure and tools
Analysis and Design

• The input from the level test plan that we need for this process is:
 Test objectives
 Scheduling and staffing for the activities
 Definition of test object(s)
 Approach—especially test case design techniques to use and
 structure and contents of the test specification
 Completion criteria, not least required coverage
 Deliverables

03/16/2022 15
 Analysis and Design

• The result of the test analysis and design should be documented in the test
specification. This document or series of documents encompasses
 The test designs—also called test groups
 The test cases—many test cases per test design
 Test procedures—often many-to-many relationship with test cases

03/16/2022 16
 Activities in Test Analysis and Design

 Defining Test Designs

The number of test groups we can define depends on the test level and the
nature, size, and architecture of the test object:
 In component testing we usually have one test group per component
 For integration testing there are usually a few groups per interface
 For system and acceptance testing we typically have many test groups

03/16/2022 17
 Activities in Test Analysis and Design
 Identification of Test Conditions
The features to be tested mentioned in the test design can be expressed as
test conditions or test requirements. A test condition is a verifiable item or
element.

The nature of a test condition depends on the nature of the test basis
documentation. It may for example be a function, a transaction, a feature, a
requirement, or a structural element like an interface parameter or a statement in the code.

The test conditions are based on or identical to our coverage items. They
are the items we are covering when we test the test object.

03/16/2022 18
Activities in Test Analysis and Design

03/16/2022 19
 Creation of Test Cases
Based on the test conditions, high-level test cases and low-level test cases are
produce.

A high-level test case is a test case without specific values for input data
and expected results, but with logical operators or other means of defining
what to test in general terms.

03/16/2022 20
• High level test case example

03/16/2022 21
• A low-level test case is a test case with specific values defined for both input
and expected result.

03/16/2022 22
Generic Test Process
3) Implementation and Execution
Test execution involves actually running the specified test on a computer system either
manually or by using an automated test tool. It is a Fundamental Test Process in which actual
work is done.
Test implementation has the following major task:
To develop and prioritize test cases by using techniques and create test data for those tests.
To create test suites from the test cases for efficient test execution. Test suite is a collection
of test cases that are used to test a software program
To re-execute the tests that previously failed in order to confirm a fix.
To log the outcome of the test execution. A test log is the status of the test case (pass/fail).
To Compare actual results with expected result.
The purpose of the test implementation is to organize the test cases in
procedures and/or scripts and to perform the physical test in the correct
environment.

The inputs on which this process is based are:

• Level test plan
• Test conditions and test design
• Other relevant documents
• The test object

03/16/2022 24
The activities are:
• Organizing test procedures
• Design and verify the test environment
• Execute the tests
• Record the testing
• Check the test results
The output consists of:
• Test specification
• Test environment
• Test logs
• Incident reports
• Tested test object

03/16/2022 25
Generic Test Process
4) Evaluating Exit criteria and Reporting
Evaluating exit criteria is a process defining when to stop testing. It depends on
coverage of code, functionality or risk. Basically it also depends on business risk, cost
and time and vary from project to project. Exit criteria come into picture, when:
 Maximum test cases are executed with certain pass percentage
 Bug rate falls below certain level
 When we achieve the deadlines
Evaluating exit criteria has the following major tasks:
 To assess if more test are needed or if the exit criteria specified should be
changed
 To write a test summary report for stakeholders
The inputs on which this process is based are:
• Test plan
• Measurements from the test development and execution processes
The activities are:
• Comparing actual measurements with estimates and planned values
• Reporting test results
The output consists of:
• Presentation of test progress
• Test report

03/16/2022 27
Generic Test Process
5)Test Closure activities:
Test closure activities are done when software is ready to be delivered. The
testing can be closed for the other reasons also like:
 When a project is cancelled
 When some target is achieved
 When a maintenance release or update is done

Test closure activities have the following major tasks:

 To check which planned deliverables are actually delivered and to ensure that all incident reports have
been resolved
 To finalize and archive testware such as scripts, test environments, etc. for later reuse
 To handover the testware to the maintenance organization. They will give support to the software
 To evaluate how the testing went and learn lessons for future releases and projects
The inputs on which this process is based are:
• Level test plan
• Test ware, including test environment
The overall procedure consists of the activities:
• Final check of deliveries and incident reports
• Secure storage/handover of test ware
• Retrospection
The output generated in this process is:
• Test experience report
• Configuration management documentation

03/16/2022 29