Software Testing

• The process of executing a program or

application with the intent of finding
software bugs (errors or other defects), and
verifying that the software product is fit for
use.
 OBJECTIVES

The student should be made to:

• Expose the criteria for test cases.
• Learn the design of test cases.
• Be familiar with test management and test
automation techniques.
• Be exposed to test metrics and
measurements.
 UNIT-1
INTRODUCTION
Testing as an Engineering Activity – Testing as a
Process – Testing axioms – Basic definitions –
Software Testing Principles – The Tester‟s Role
in a Software Development Organization –
Origins of Defects – Cost of defects – Defect
Classes – The Defect Repository and Test Design
– Defect Examples – Developer/Tester Support
of Developing a Defect Repository – Defect
Prevention strategies.
 Unit-II
TEST CASE DESIGN
Test case Design Strategies – Using Black Box
Approach to Test Case Design – Random Testing –
Requirements based testing – Boundary Value
Analysis – Equivalence Class Partitioning – State-
based testing – Cause-effect graphing – Compatibility
testing – user documentation testing – domain testing
– Using White Box Approach to Test design – Test
Adequacy Criteria – static testing vs. structural testing
– code functional testing – Coverage and Control Flow
Graphs – Covering Code Logic – Paths – code
complexity testing – Evaluating Test Adequacy
Criteria.
 Unit-III
LEVELS OF TESTING
The need for Levels of Testing – Unit Test – Unit Test
Planning – Designing the Unit Tests – The Test Harness
– Running the Unit tests and Recording results –
Integration tests – Designing Integration Tests –
Integration Test Planning – Scenario testing – Defect
bash elimination System Testing – Acceptance testing –
Performance testing – Regression Testing –
Internationalization testing – Ad-hoc testing – Alpha,
Beta Tests – Testing OO systems – Usability and
Accessibility testing – Configuration testing –
Compatibility testing – Testing the documentation –
Website testing.
 UNIT IV
TEST AMANAGEMENT
People and organizational issues in testing –
Organization structures for testing teams –
testing services – Test Planning – Test Plan
Components – Test Plan Attachments –
Locating Test Items – test management – test
process – Reporting Test Results – The role of
three groups in Test Planning and Policy
Development – Introducing the test specialist
– Skills needed by a test specialist – Building a
Testing Group.
 UNIT-V
TEST AUTOMATION
Software test automation – skill needed for
automation – scope of automation – design
and architecture for automation –
requirements for a test tool – challenges in
automation – Test metrics and measurements
– project, progress and productivity metrics.
TEXT BOOKS:
1. Srinivasan Desikan and Gopalaswamy Ramesh,
“Software Testing – Principles and Practices”, Pearson
Education, 2006.
2. Ron Patton, “ Software Testing”, Second Edition, Sams
Publishing, Pearson Education, 2007.

REFERENCES:
1. Ilene Burnstein, “ Practical Software Testing”, Springer
International Edition, 2003.
2. Edward Kit,” Software Testing in the Real World –
Improving the Process”, Pearson Education, 1995.
3. Boris Beizer,” Software Testing Techniques” – 2nd
Edition, Van Nostrand Reinhold, New York, 1990.
4. Aditya P. Mathur, “Foundations of Software Testing _
Fundamental Algorithms and Techniques”, Dorling
Kindersley (India) Pvt. Ltd., Pearson Education, 2008.
 Testing as an
Engineering Activity
• Poor quality software that can cause loss of
life or property is no longer acceptable to
society.
• Failure can result in catastrophic losses.
• Highly qualified staff make sure that software
products are built on time, within budget, and
are of the highest quality
• IEEE & ACM - code of ethics for the software
engineering discipline
 Using an engineering approach to software
development means the following:
• The development of the process is well understood.
• Projects are planned.
• Life cycle models are defined and adhered to.
• Standards are in place for product and process
• Measurements are employed to evaluate product and
process quality.
• Components are reused.
• Validation and verification processes play a key role in
quality determination;
• Engineers have proper education, training, and
certification.
• A test specialist is one whose education is based
on the principles, practices and processes that
constitute the software engineering discipline
and whose specific focus is on one area of that
discipline, software testing.
Knowledge on the following
• Test related principles,
• Processes,
• Measurements,
• Standards,
• Plans,
• Tools and methods
• How to apply them to the testing tasks.
 Role of process in software quality

• Process- the set of methods, practices,

standards, documents, activities, policies,
and procedures that software engineers use
to develop and maintain a software system
and its associated artifacts, such as project
and test plans, design documents, code, and
manuals.
Testing as a process
 Testing as a process
• Validation is the process of evaluating a software
system or component during, or at the end of,
the development cycle in order to determine
whether it satisfies specified requirements.
• Verification is the process of evaluating a
software system or component to determine
whether the products of a given development
phase satisfy the conditions imposed at the start
of that phase
 Testing Axioms
• An axiom – A sentence or proposition that is
not proved and considered as obvious for
theory building and acceptation
 Testing Axioms
1. THE STAKEHOLDER AXIOM: testing needs
stakeholders
2. THE VALUE AXIOM: the value of intelligence is
independent of who produce it
3. THE SCOPE MANAGEMENT AXIOM: if we don’t
manage scope, we may never meet stakeholder
expectations
4. THE GOOD-ENOUGH AXIOM: the scope of
testing and acceptance are always compromises
5. TEST BASIS AXIOM: testers need sources of
knowledge to select things to test
6. THE ORACLE AXIOM: testers need sources of
knowledge to evaluate actual outcomes or
behaviours
7.THE COVERAGE AXIOM: testing needs a test
coverage model or models
8. THE PRIORITISATION AXIOM: prioritizing the tests
needs a mechanism for ordering tests by value
9. THE FALLIBILITY AXIOM: our sources of
knowledge are fallible and incomplete(Ready for
retesting)
10. THE REPEAT-TEST AXIOM: some repeated tests
are unavoidable
11. THE EXECUTION SEQUENCING AXIOM: run our
most valuable tests first – we may not have time
to run them later
12. THE ENVIRONMENT AXIOM: test execution
requires a known, controlled environment
13. THE EVENT AXIOM: testing never goes as
planned; evidence arrives in discrete quanta
14. THE EXECUTION SEQUENCING AXIOM: run
our most valuable tests first – we may not
have time to run them later
15.The design axiom:Test design is based on
models
16.The delivery axiom:Intelligence produced by
test determines the value of testing
 Basic definitions
Errors: An error is a mistake, misconception, or
misunderstanding on the part of a software developer.
Faults (Defects) :A fault (defect) is introduced into the
software as the result of an error. It is an anomaly in
the software that may cause it to behave incorrectly,
and not according to its specification.
Failures: A failure is the inability of a software system or
component to perform its require functions within
specified performance requirements.
A test case :A test-related item which contains
the following information:
1. A set of test inputs.
2. Execution conditions.
3. Expected outputs.
Test :A test is a group of related test cases and
test Procedures.
Test Oracle:A test oracle is a document, or piece
of software that allows testers to determine
whether a test has been passed or failed.
Test Bed: A test bed is an environment that
contains all the hardware and software
needed to test a software component or a
software system.
Quality -the degree to which a system, system
component, or process meets specified
requirements.
Metric:A metric is a quantitative measure of the
degree to which a system, system component, or
process possesses a given attribute.
The software quality assurance (SQA) group is a
team of people with the necessary training and
skills to ensure that all necessary actions are
taken during the development process so that the
resulting software conforms to established
technical requirements.
REVIEW: A review is a group meeting whose
purpose is to evaluate a software artifact or a set
of software artifacts.
 Principles
Principle 1. Testing is the process of exercising a
software component using a selected set of
test cases, with the intent of (i) revealing
defects, and (ii) evaluating quality.
Principle 2. When the test objective is to detect
defects, then a good test case has a high
probability of revealing undetected defect(s).
Principle 3. Test results should be inspected
meticulously.
Principle 4. A test case must contain the
expected output or result.
Principle 5. Test cases should be developed for
both valid and invalid input conditions.
Principle 6. The probability of the existence of
additional defects in a software component is
proportional to the number of defects already
detected in that component.
Principle 7. Testing should be carried out by a
group that is independent of the development
group.
Principle 8. Tests must be repeatable and
reusable.
Principle 9. Testing should be planned.
Principle 10. Testing activities should be
integrated into the software life cycle.
Principle 11. Testing is a creative and challenging
task.
 The Tester’s Role in a Software
Development Organization

1.The tester’s job is to

• Reveal defects,
• Find weak points,
• Inconsistent behaviour,
• Circumstances where the software does not work
as expected.
2. A tester needs very good programming
experience in order to understand how code is
constructed, and to know where and what types
of, defects could occur.
3. work with the developers to produce high-quality
software that meets the customers’
requirements.
4. A testers also need to work with requirements
engineers to make sure that requirements are
testable, and to plan for system and acceptance
test.
5. Testers also need to work with designers to plan
for integration and unit test.
6. Testers also need to cooperate with software
quality assurance staff and software engineering
process group members.
7. Testers need the support of management.
• Testers are specialists, their main function is to
plan, execute, record, and analyse tests.
• They do not debug software.
• When defects are detected during testing,
software should be returned to the
developers.
 ORIGINS OF DEFECTS
• Defects have negative effects on software use.
Software engineers work very hard to produce
high-quality software with a low number of
defects.
• Education: The software engineer did not
have the proper educational background to
prepare the software artifact.
• Communication: The software engineer was
not informed about something by a colleague.
• Oversight: The software engineer omitted to
do something.
• Transcription: The software engineer knows
what to do, but makes a mistake in doing it.
• Process: The process used by the software
engineer misdirected tester’s actions
 A tester develops hypotheses about possible
defects. Test cases are then designed based on
the hypotheses. The hypotheses are used to,
• Design test cases.
• Design test procedures.
• Assemble test sets.
• Select the testing levels suitable for the tests.
• Evaluate the results of the tests.
1. Fault Model
A fault (defect) model can be described as a
link between the error made, and the
fault/defect in the software.
2. Defect Repository
To increase the effectiveness of their testing
and debugging processes, software
organizations need to initiate the creation of a
defect database, or defect repository. The
defect repository supports storage and
retrieval of defect data from all projects in a
centrally accessible location.
 DEFECT CLASSES
The four classes of defects are as follows,
• Requirements and specifications defects,
• Design defects,
• Code defects,
• Testing defects
1. Requirements and Specifications Defects

• The beginning of the software life cycle is

important for ensuring high quality in the
software being developed. Defects injected in
early phases can be very difficult to remove in
later phases. Since many requirements
documents are written using a natural language
representation, they may become
• Ambiguous,
• Contradictory,
• Unclear,
• Redundant,
• Imprecise.
1.1 Functional Description Defects

• The overall description of what the product

does, and how it should behave
(inputs/outputs), is incorrect, ambiguous,
and/or incomplete.
1.2 Feature Defects

• Features is described as distinguishing

characteristics of a software component or
system. Feature defects are due to feature
descriptions that are missing, incorrect,
incomplete, or unnecessary.
1.3 Feature Interaction Defects

• These are due to an incorrect description of how

the features should interact with each other.
1.4 Interface Description Defects

• These are defects that occur in the description of

how the target software is to interface with
external software, hardware, and users.
2. Design Defects

Design defects occur when the following are

incorrectly designed,
• System components,
• Interactions between system components,
• Interactions between the components and
outside software/hardware, or users
2.1 Algorithmic and Processing Defects

• These occur when the processing steps in the algorithm as

described by the pseudo code are incorrect.
2.2 Control, Logic, and Sequence Defects

• Control defects occur when logic flow in the pseudo code is

not correct.
2.3 Data Defects

• These are associated with incorrect design of data

structures.
2.4 Module Interface Description Defects

• These defects occur because of incorrect or inconsistent

usage of parameter types, incorrect number of parameters
or incorrect ordering of parameters.
2.5 Functional Description Defects

• The defects in this category include incorrect,

missing, or unclear design elements.
2.6 External Interface Description Defects

• These are derived from incorrect design

descriptions for interfaces with COTS(commercial
off-the-shelf) components, external software
systems, databases, and hardware devices.
3. Coding Defects

• Coding defects are derived from errors in

implementing the code. Coding defects classes
are similar to design defect classes. Some
coding defects come from a failure to
understand programming language
constructs, and miscommunication with the
designers.
3.1 Algorithmic and Processing Defects

• Code related algorithm and processing defects

include
• Unchecked overflow and underflow conditions,
• Comparing inappropriate data types,
• Converting one data type to another,
• Incorrect ordering of arithmetic operators,
• Misuse or omission of parentheses,
• Precision loss,
• Incorrect use of signs.
3.2 Control, Logic and Sequence Defects

• This type of defects include incorrect

expression of case statements, incorrect
iteration of loops, and missing paths.
3.3 Typographical Defects

• These are mainly syntax errors, for example,

incorrect spelling of a variable name that are
usually detected by a compiler or self-reviews,
or peer reviews.
3.4 Initialization Defects

• This type of defects occur when initialization

statements are omitted or are incorrect. This may
occur because of misunderstandings or lack of
communication between programmers, or
programmer`s and designer`s, carelessness, or
misunderstanding of the programming environment.
3.5 Data-Flow Defects

• Data-Flow defects occur when the code does not

follow the necessary data-flow conditions.
3.6 Data Defects

• These are indicated by incorrect implementation of

data structures.
3.7 Module Interface Defects

• Module Interface defects occurs because of using

incorrect or inconsistent parameter types, an
incorrect number of parameters, or improper
ordering of the parameters.
3.8 Code Documentation Defects

• When the code documentation does not describe

what the program actually does, or is incomplete
or ambiguous, it is called a code documentation
defect.
3.9 External Hardware, Software Interfaces Defects

These defects occur because of problems related to

• System calls,
• Links to databases,
• Input/output sequences,
• Memory usage,
• Resource usage,
• Interrupts and exception handling,
• Data exchanges with hardware,
• Protocols,
• Formats,
• Interfaces with build files,
• Timing sequences.
4. Testing Defects

• Test plans, test cases, test harnesses, and test

procedures can also contain defects. These
defects are called testing defects. Defects in
test plans are best detected using review
techniques.
4.1 Test Harness Defects
A test harness or automated testframework is a
collection of software and test data configured
to test a program unit by running it under varying
conditions and monitoring its behavior and
outputs. It has two main parts: the test execution
engine and thetest script repository.
• In order to test software, at the unit and
integration levels, auxiliary code must be
developed. This is called the test harness or
scaffolding code. The test harness code should be
carefully designed, implemented, and tested
since it is a work product and this code can be
reused when new releases of the software are
developed.
4.2 Test Case Design and Test Procedure Defects

• These consists of incorrect, incomplete,

missing, inappropriate test cases, and test
procedures.
 Defect Example: The Coin Problem

Specification for the program calculate_coin_value

This program calculates the total rupees value for a
set of coins. The user inputs the amount of 25p,
50p and 1rs coins. There are size different
denominations of coins. The program outputs the
total rupees and paise value of the coins to the
user
Input : number_of_coins is an integer
Output : number_of_rupees is an integer
number_of_paise is an integer
• 1. Design Description for the Coin Problem
• Design Description for Program calculate_coin_values
Program calculate_coin_values
number_of_coins is integer
total_coin_value is integer
number_of_rupees is integer
number_of_paise is integer
coin_values is array of six integers representing
each coin value in paise
initialized to: 25,25,100
begin
initialize total_coin_value to zero
initialize loop_counter to one
while loop_counter is less than six
begin
output “enter number of coins”
read (number_of_coins )
total_coin_value = total_coin_value +
number_of_coins * coin_value[loop_counter]
increment loop_counter
end
number_rupees = total_coin_value/100
number_of_paise = total_coin_value – 100 * number_of_rupees
output (number_of_rupees, number_of_paise)
end
• 2. Design Defects in the Coin Problem
• Control, logic, and sequencing defects. The defect in this
subclass arises from an incorrect “while” loop condition
(should be less than or equal to six)

• Algorithmic, and processing defects. These arise from the

lack of error checks for incorrect and/or invalid inputs, lack
of a path where users can correct erroneous inputs, lack of
a path for recovery from input errors.

• Data defects. This defect relates to an incorrect value for

one of the elements of the integer array, coin_values,
which should be 25, 50, 100.
• External interface description defects. These are
defects arising from the absence of input messages or
prompts that introduce the program to the user and
request inputs.
3. Coding Defects in the Coin Problem
• Control, logic, and sequence defects. These include
the loop variable increment step which is out of the
scope of the loop. Note that incorrect loop condition
(i<6) is carried over from design and should be counted
as a design defect.
• Algorithmic and processing defects. The division
operator may cause problems if negative values are
divided, although this problem could be eliminated
with an input check.
• Data Flow defects. The variable total_coin_value is not
initialized. It is used before it is defined.
• Data Defects. The error in initializing the array
coin_values is carried over from design and
should be counted as a design defect.
• External Hardware, Software Interface
Defects. The call to the external function “scanf”
is incorrect. The address of the variable must be
provided.
• Code Documentation Defects. The
documentation that accompanies this code is
incomplete and ambiguous. It reflects the
deficiencies in the external interface description
and other defects that occurred during
specification and design.
 Developer/Tester Support for Developing a
Defect Repository
• A requirement for repository development
should be a part of testing and/or debugging
policy statements.
• Forms and templates should be designed to
collect the data.
• Each defect and frequency of occurrence must
be recorded after testing.
• Defect monitoring should be done for each
on-going project.
• The defect data is useful for test planning. It is
a TMM level 2 maturity goal. It helps a tester
to select applicable testing techniques, design
the test cases, and allocate the amount of
resources needed to detect and remove
defects. This allows tester to estimate testing
schedules and costs.
• The defect data can support debugging
activities also.
A defect repository can help in implementing
several TMM maturity goals including
• Controlling and monitoring of test,
• Software quality evaluation and control,
• Test measurement,
• Test process improvement.
 DEFECT PREVENTION STRATEGIES
• The purpose of Defect Prevention is to identify
the Root cause of defects and prevent them
from recurring.
• This involves analyzing defects that were
encountered in the past and taking specific
actions to prevent the occurrence of those
types of defects in the future.
• It also enhances the Productivity.
• It Reduces rework effort.
Methods of Defect Preventions
• Reviews & Inspections: Self-Review, Peer Review
& Inspections.
• Walkthroughs: prototyping of the actual design
that gives the you the basic idea of the product
functionality along with its look & feel.
• Defect Logging and Documentation: provide key
parameters that supports Defect Analysis and
Measurements.
• Root Cause Analysis.
• Fishbone Analysis.
 SOFTWARE DEFECT FRAMEWORK
The software defect framework highlighting the 5 Ds of defect
origin is proposed in this work.
Each one of the D’s concentrates on defects in one particular
stage of software development lifecycle like Requirements,
Design, Coding, Testing and defects due to timeline
problem.
After doing a thorough analysis of various defect types under
each stage of software development, the most prominent
defects are identified.
The defects are then prioritized based on their importance
and the top six defects from each category are taken for study.
Certain type of these defects are marked with Explicit (E)
signifies influencing the process and certain other
defect types are marked with Implicit (I) to signify
those defects that are reported by customer.
Utmost care should be given to such defect types in order
to satisfy customers. For each one of the identified
defect type, the reason for such defect is found out and
the DP actions are suggested. These DP actions, when
introduced at all stages of a software lifecycle, can
reduce the time and resource necessary to develop
high quality systems.
 Deficiency in Requirements ( D1)

• Training needed
• Feasibility study
•
• requirements signed off
•
• Quality team validate checkl

• BA and QT make DT
• understand the req
• BA and QT freeze scope
•
DESIGN FLAWS
PM &Design experts formalize
best design approach

New requirement undergo

formal configuration mgmt

PM seek help of design experts

QT need justification

Need support of document

writers

Traceability matrix used

Defective coding process
Delinquency in testing
Duration slippage
Defect severity
 Test Maturity Model
Test Maturity Model is one of such model
which has a set of structured levels. TMM is
now replaced by Test Maturity Model
Integration(TMMI) is a 5 level model which
provides a framework to measure the
maturity of the testing processes.
• Maturity goals
Each maturity level, except 1, contains certain
maturity goals.
For an organization to reach a certain level, the
corresponding maturity goals must be met by the
organization.
• Maturity subgoals
Maturity goals are supported by maturity
subgoals
• ATRs
Maturity subgoals are achieved by means of
ATRs.
ATRs address issues concerning implementation
of activities and tasks.
Three groups:
• Managers
• Developers and test engineers
• Customers