Software Testing Techniques

TEACHER: AIMAN NAEEM

Basics of Software Testing

Week 2
Software Quality Attributes

 Static quality attributes: structured, maintainable, testable code as well as the

availability of correct and complete documentation.

 Dynamic quality attributes: software reliability, correctness, completeness,

consistency, usability, and performance

 Completeness refers to the availability of all features listed in the requirements, or in

the user manual. An incomplete software is one that does not fully implement all
features required.
Software Quality Attributes

 Consistency refers to adherence to a common set of conventions and assumptions. For

example, all buttons in the user interface might follow a common color coding
convention. An example of inconsistency would be when a database application displays
the date of birth of a person in the database.
 Usability refers to the ease with which an application can be used. This is an area in itself
and there exist techniques for usability testing. Psychology plays an important role in the
design of techniques for usability testing.
 Performance refers to the time the application takes to perform a requested task. It is
considered as a non-functional requirement. It is specified in terms such as “This task
must be performed at the rate of X units of activity in one second on a machine running at
speed Y, having Z gigabytes of memory."
Software Reliability

 First Definition:
Software reliability is the probability of failure-free operation of software over a
given time interval and under given conditions.
The probability referred to in this definition depends on the distribution of the
inputs to the program. Such input distribution is often referred to as the
operational profile. According to this definition, software reliability could vary
from one operational profile to another. An implication is that one user might
say “this program is lousy” while another might sing praises for the same
program.
Software Reliability

 Second Definition:
Software reliability is the probability of failure-free operation of software in its
intended environment.
This definition is independent of “who uses what features of the software and
how often”. Instead, it depends exclusively on the correctness of its features. As
there is no notion of operational profile, the entire input domain is considered as
uniformly distributed. The term environment refers to the software and
hardware elements needed to execute the application. These elements include
the operating system (OS), hardware requirements, and any other applications
needed for communication
Requirements, Behavior and Correctness

 Requirements specify the functions that a product is expected to perform.

 Once the product is ready, it is the requirements that determine the expected behavior.

 Of course, during the development of the product, the requirements might have changed from
what was stated originally.

 Regardless of any change, the expected behavior of the product is determined by the tester’s
understanding of the requirements during testing.
Requirements

Requirements leading to two different programs:

 Requirement 1:
It is required to write a program that inputs two integers and outputs the maximum of
these.
 Requirement 2:
It is required to write a program that inputs a sequence of integers and outputs the sorted
version of this sequence.
Requirements: Incompleteness

Suppose that program max is developed to satisfy Requirement 1. The expected

output of max when the input integers are 13 and 19 can be easily determined to be
19.

Suppose now that the tester wants to know if the two integers are to be input to the
program on one line followed by a carriage return, or on two separate lines with a
carriage return typed in after each number. The requirement as stated above fails to
provide an answer to this question.
Requirements: Ambiguity

Requirement 2 is ambiguous. It is not clear whether the input

sequence is to sorted in ascending or in descending order. The
behavior of sort program, written to satisfy this requirement, will
depend on the decision taken by the programmer while writing
sort.
Input Domain and Program Correctness

 INPUT DOMAIN The set of all possible inputs to a program P is known as the input domain, or input
space, of P.
 CORRECTNESS A program is considered correct if it behaves as expected on each element of its input
domain.
Program is considered correct if it behaves as desired on all possible test inputs. Usually, the set of all
possible inputs is too large for the program to be executed on each input.
A tester often needs to determine what constitutes all possible inputs. The first step in determining all
possible inputs is to examine the requirements. If the requirements are complete and unambiguous, it should
be possible to determine the set of all possible inputs.
Examples

 Example 1: Using Requirement 1 from Example 1.3, we find the input domain of NBY to be the set of
all pairs of integers where each element in the pair integers is in the range from −32,768 to 32,767.
 Example 2: Using Requirement 2 from Example 1.3, it is not possible to find the input domain for the
TPSU program. Let us, therefore, assume that the requirement was modified to be the following:
Modified Requirement 2: It is required to write a program that inputs a sequence of integers and outputs the
integers in this sequence sorted in either ascending or descending order. The order of the output sequence is
determined by an input request character which should be “A” when an ascending sequence is desired, and
“D” otherwise. While providing input to the program, the request character is entered first followed by the
sequence of integers to be sorted; the sequence is terminated with a period
Example Contd.

Based on the above modified requirement, the input domain for TPSU is a set of pairs. The first element of the
pair is a character. The second element of the pair is a sequence of zero or more integers ending with a period.
For example, following are three elements in the input domain of sort:

The first element contains a sequence of four integers to be sorted in ascending order, the second one has a
sequence to be sorted in descending order, and the third one has an empty sequence to be sorted in ascending
order.
 VALID AND INVALID INPUTS

 The modified requirement for sort mentions that the request characters can be “A'' and “D'', but fails to
answer the question “What if the user types a different character ?”
 When using sort it is certainly possible for the user to type a character other than “A'' and “D''. Any
character other than “A'’ and “D'' is considered as invalid input to sort. The requirement for sort does not
specify what action it should take when an invalid input is encountered.
Identifying the set of invalid inputs and testing the program against these inputs are important parts
of the testing activity. Even when the requirements fail to specify the program behavior on invalid
inputs, the programmer does treat these in one way or another. Testing a program against invalid
inputs might reveal errors in the program.
CORRECTNESS VERSUS RELIABILITY
Correctness
 Correctness refers to whether a program is
error-free and performs its intended function
accurately.
 It is usually established via mathematical
proofs based on the formal specification of
requirements and program text.
 However, testing is also important to find
errors in the program and increase confidence
in its correct functioning
Reliability
 Reliability refers to the probability of a
program's successful execution on randomly
selected inputs from its input domain.
 It is a continuous metric over a scale from 0 to
1, whereas correctness is a binary metric.
 Removing errors from a program usually
improves its reliability, but there may be cases
where the probability of program failure does
not change upon error removal.
OPERATIONAL PROFILE

 An operational profile is a numerical description of how a program is used.

 Consider a sort program which, on any given execution, allows any one of two types of input
sequences. Sample operational profiles for sort.

 The two operational profiles above suggest significantly different uses of TPSU. In one case it is
used mostly for sorting sequences of numbers and in the other case it is used mostly for sorting
alphanumeric strings
Testing and Debugging

 Testing is the process of determining if a program has any errors.

 When testing reveals an error, the process used to determine the
cause of this error and to remove it, is known as debugging.
Preparing Test Plan

 A test plan is a document that outlines the approach, objectives, and scope of testing for a software
system or component.
 The test plan should include information about the testing strategy, test objectives, testing activities,
test schedule, test environment, and test deliverables.
 The test plan should also specify the testing techniques, tools, and resources that will be used, as
well as the roles and responsibilities of the testing team members.
 The test plan should be reviewed and approved by the relevant stakeholders, including the project
manager, development team, and quality assurance team.
 The test plan should be updated as necessary throughout the software development life cycle to
reflect changes in the system or component being tested and changes in testing requirements.
 I hope this updated summary accurately reflects the content of the book. Please let me know if you
have any further questions or if there's anything else I can assist you with.