ITECH7409: Software Testing

Lecture 3. Test Planning. Types of Testing:

Static & Dynamic.
This lecture
• Test planning
• Test engineers & test managers
• Reviews
• Static testing
• Dynamic testing
• White box testing
• Flow charts to decide test cases
Software testing: Who does it?

Developer: Understands the Independent Tester: Must learn

system but will test "gently“ and about the system but will attempt
is driven by "delivery“ to break it and is driven by quality.
Programmers often stick to the
data set that makes the program
work: "Don’t mess up my code!"
Software testing: Who does it?
Before a user gets their hands on software it should be thoroughly
tested by a team of professionals.
• Test engineer: an IT professional in charge of one or more technical
test activities, such as
• Designing test inputs.
• Producing test values.
• Running test scripts.
• Analysing results.
• Reporting results to developers and managers.
Software testing: Who does it?
• Test manager: a professional in charge of one or more test engineers,
whose role is to
• Understand the software requirements and program model and
decide how to test it.
• Set test policies and processes.
• Interact with other managers on the project development.
• Assist and support test engineers in their role.
Software testing: Who does it?
Test engineers and managers need
• To have considerable technical expertise as software continues
to evolve.
• To be specialists in testing and other systems.
• To maintain up-to-date industry skills/education.
• Adopt practices and techniques to improve their efficiency.
• Access to more and better software tools, especially to
automate some testing.
Software testing: Who does it?
Test engineers and managers need to ensure there is a suitable test
infrastructure in place:
• Technical resources.
• Human resources.
• Configuration.
• Data.
• Test design specifications.

• Test engineers and managers need to document the entire process.

Software testing: Who does it?
Test engineers and managers need to
• Track defects via either tool or manually.
• Determine cause of defect.
• Rate severity of defect.
• Rate priority of defect.
• Assign to defect management or change management.
Software testing: Who does it?
Test Planning
Test plans are documents setting out standards for the testing process.
They are used by:
• Software engineers to design and carry out testing.
• Technical staff to get an overview of system and understand its
workings.
• Management to monitor progress.
Static Testing
Verification is checking that the right system is being developed and built.
Verification is also known as static testing.

Static testing is desk-checking of the source code in order to pick up errors in

coding before dynamic testing of code is undertaken. It is done through a series of
reviews, including:
• Management reviews.
• Technical reviews.
• Code reviews.
• Test-case reviews.

The reviews may be formal, that is follow a Test Plan or other documentation, or
informal, conducted as a ‘walk through’.
Static Testing
The goals of reviews are to:
• Detect problems and defects in requirement specifications.
• Detect defects in design.
• Find deviations in coding standards.
• Confirm if prepared test cases are sufficient to validate software.
• Help document progress.
Reviews
• These involve people examining the source representation with the
aim of discovering anomalies and defects.
• Inspections do not require execution of a system so may be used
before implementation.
• They may be applied to any representation of the system
(requirements, design, configuration data, test data, etc.).
• They have been shown to be an effective technique for discovering
program errors.
Reviews
Static Testing: Code Reviews
An Error Checklist for Inspections:
• The checklist is largely language independent.
• Sometimes (unfortunately) concentrate more on issues of style
than on errors (for example, “Are comments accurate and
meaningful?” and “Are if- else, code blocks, and do-while groups
aligned?”), and the error checks are too nebulous to be useful
(such as “Does the code meet the design requirements?”).
Advantages:
• The programmer usually receives feedback concerning
programming style, choice of algorithms, and programming
techniques.
• The other participants gain in a similar way by being exposed to
another programmer’s errors and programming style.
Static Testing Example: Code Review
Static Testing Example: Code Review
Data Flow Errors
Will the program, module, or
subroutine eventually terminate?
(Halting Problem)
Comparison errors
Does the way in which the compiler
evaluates Boolean expressions
affect the program?

while(flag){ if ((x = = 0) || (y/x)>z)

…
}
This statement may be acceptable
for compilers that end the test as
soon as one side of an OR is true but
What happens if flag never becomes may cause a division-by-zero error
false? with other compilers.
Advantages of Reviews
• During testing, errors can mask (hide) other errors. Because
inspection is a static process, you don’t have to be concerned with
interactions between errors.
• Incomplete versions of a system can be inspected without additional
costs. If a program is incomplete, then you need to develop
specialized test harnesses to test the parts that are available.
• As well as searching for program defects, an inspection can also
consider broader quality attributes of a program, such as compliance
with standards, portability and maintainability.
Reviews & Testing
• Reviews and testing are complementary and not opposing verification
techniques.
• Both should be used during the Verification & Validation process.
• Inspections can check conformance with a specification but not
conformance with the customer’s real requirements.
• Inspections cannot check non-functional characteristics such as
performance, reliability, usability, etc.
Dynamic Testing
Dynamic testing is a type of software testing that involves executing the
software and evaluating its dynamic behaviour during runtime.
Objectives of dynamic testing include:
• Finding errors and bugs.
• Verifying functionality of the software.
• Assessing performance.
• Checking if information is properly handled by the software.
• Assessing scalability.
• Etc.
Levels of Dynamic Testing
Dynamic testing is used at different testing levels:
• Unit testing.
• Integration testing.
• System testing.
• Acceptance testing.
• Performance testing.
• Security testing.
Dynamic Testing Techniques
There are two techniques of dynamic testing based on whether the tester
has access to the source code or not:
• Black-box Testing.
• White-box Testing.

Black-box testing can further be divided in:

• Functional and
• Non-functional testing.
Dynamic Testing Techniques
White-box testing uses the following approaches:
• Statement Testing (Algebraic Testing): Test single statements.
• Loop Testing:
• Cause execution of the loop to be skipped completely. (Exception: Repeat
loops)
• Loop to be executed exactly once
• Loop to be executed more than once
• Path testing:
• Make sure all paths in the program are executed
• Branch Testing (Conditional Testing): Make sure that each possible
outcome from a condition is tested at least once
Advantages of Dynamic Testing
Some advantages of dynamic testing:
• Reveals the defects not covered by static testing.
• Tests the actual working software end to end.
• Helps in identifying weak links in the runtime environment.
• Data/control flow analysis checks the code functionality.
• Helps in increasing the reliability, security, and usability of the
application.
Disadvantages of Dynamic Testing
Some disadvantages of dynamic testing:
• It is time and resource-consuming as the execution of the software
needs lots of resources.
• It is done in the later stages of the development life cycle; hence,
fixing the defects identified by this testing is costly.
• It is difficult to trace the vulnerability in the code.
Example: Primality Check
In this example we test two different Python implementations of
is_prime(n) function.
The function takes a positive integer as an argument and returns true if
the number is a prime number, otherwise, it returns false.
Here’s the first implementation:

def is_prime(n):
for i in range(2,n):
if (n%i == 0):
return False
return True
Primality Check: First Implementation
• Quick static analysis shows that the function is written correctly – it
simply follows the definition of a prime number.
• Indeed, the function consecutively attempts dividing its argument, n,
by the numbers 2, …, (n-1). If succeeded at least once it returns true,
otherwise, returns false.
• So, it satisfies the functional requirement.
• What about the performance? Is it acceptable? Can we do better?
• In order to assess the performance, will use the function is_prime to
generate a list of all prime numbers smaller than a given number.
Primality Check: First Implementation
def is_prime(n):
for i in range(2,n):
if (n%i == 0):
return False
return True

def primes_to_list(n):
list = []
for i in range(2, n+1):
if(is_prime(i)):
list.append(i)
return list
Primality Check: First Implementation
def main():
n = int(input('Enter an upper bound for primes: '))
import time
start = time.time()
l = primes_to_list(n)
end = time.time()

print('\nTime spent = {} seconds\n'.format(end-start))

print('\nThe number of prime numbers: {}\n'.format(len(l)))
Primality Check: First Implementation
The program spent about 20 seconds to find all prime numbers < 100000
Primality Check: Second Implementation
• Let’s consider another implementation of is_prime(n).
def is_prime_fast(n):
k = int(n**0.5)
for i in range(2,k+1):
if (n%i == 0):
return False
return True

• It looks a bit more cumbersome and is not as obvious as the first

implementation.
• So, the white-box analysis requires certain level of expertise, and it is not
clear if this implementation is better than the first one.
• Let’s do the black-box analysis, i.e., run the previous program with the
second implementation of is_prime.
Primality Check: Second Implementation
Functional result is the same – 9592 prime numbers found. However, the time spent
is just 0.1 seconds, i.e., it is about 12000 times faster than in the first
implementation!