Module A

Fundamentals of Testing
by
K.N.Sasidhar

Fundamentals of Testing

Why is Testing necessary?

What is Testing?
General Testing Principles
Fundamental Test Process
The Psychology of Testing
Re-testing and Regression testing
Expected Results
Prioritisation of Tests

Why is testing necessary?

Software systems context

Define errors/mistake,faults,failures and reliability
Errors and how they occur
Cost of errors
Exhaustive testing is impossible
Testing and risk
Testing and quality
Testing and contractual, legal,regulatory
requirements
How much testing is enough

Software Systems context

Software systems are an increasing part of life,
from business applications to healthcare systems
to consumer products etc
Software that does not work correctly can lead to
many problems including loss of money, time or
business reputation, and could even cause injury
or death.

Define Errors, Fault,Failures

According to BS 7925-1
Error: A human action that produces an incorrect result.
Fault: Manifestation of an error in software or in
implementation.
Failure: Deviation of the software from its expected
delivery or service.
A human being can make an error(mistake),which
produces a defect (fault,bug) in the code, in software
system.If a defect in code is executed , the system will
fail to do what it should do, causing a failure.
Bug Generic term for defect,fault,failure

Errors and How they occur

- Can be introduced at any stage of SDLC
- We are all prone to making simple human errors
- Miscommunication, Poor requirements
- Working under pressures such as tight deadlines, budget
restrictions,conflicting priorities
- Complexity of technologies / infrastructure
- Poorly documented code
- Due to environmental conditions

Cost of Errors
May vary from nothing at all to large amount of money loss
and even loss of life.
A single failure can cost nothing or a lot. Software in safetycritical systems can cause death or injury if it fails.
Ex:
Online trading systems: $100,000 per minute revenue loss

Exhaustive Testing is impossible

Exhaustive testing:
Executing a program with all possible combinations
of inputs or values for program variables
Only exhaustive testing can show a program is free
from defects.
- It is practically impossible due to time and resources
- It is not cost effective
It is much better to :
- use testing techniques to give confidence
- Base the testing on the risks to be contained

Testing and Risk

- Testing is a risk reduction process
The amount of testing performed depends on the
risks involved ( Failure / Loss)
Risks - As a basis for allocating the test time that is
available and for selecting what to test and where to
place emphasis.
A priority must be assigned to each test so that most
important tests are run first.
The objective is to always minimise risk

Testing and Quality

- Testing increases software quality
Testing identifies faults and whose removal increases
software quality by increasing the softwares potential
reliability .
- Testing is the measurement of software quality
We measure how closely we have achieved quality by
testing the relevant factors such as
Correctness,Reliability,Usability, Maintainability etc.

Quality Control :
is the activity performed to ensue that a Product is fit for
Purpose .
Quality Assurance:
is the activity performed to test the process is being
followed
Quality Management:
Responsible choosing the correct process to ensure that the
product is fit for purpose.

Testing and Contractual,Legal,Regulatory requirements

Other factors that may determine the testing performed may
be:
- Legal
- Contractual
- Normally defined industry specific standards
Ex: FSA, US FAA, OFTEL

How much testing is enough ?

Testing is a matter of judging risks against the cost
of extra testing efforts.
Planning testing effort before you begin and
setting completion criteria ensure that testing is
properly prioritised.

What is Testing ?
Definition
Common objectives of testing
Purpose of testing at different phases of SDLC

Testing Definition
The Process of exercising software to verify that
it satisfies specified requirements and to detect
errors
BS 7925-1
Testing is :
- a Risk reduction process
- aimed at discovering faults
- an iterative process
- the measurement of quality

General Testing Principles

Principle 1- Testing shows presence of defects

Principle 2- Exhaustive testing is impossible
Principle 3- Early testing
Principle 4- Defect clustering
Principle 5- Pesticide Paradox
Principle 6- Testing is context dependent
Principle 7- Absence-of-errors fallacy

Fundamental Test Process

Test Planning
Test Analysis and Design
Test implementation and execution
Test Recording
Checking for test completion(Exit criteria)

The Psychology of Testing

Why do we test?
Tester-Developer relationships
Test Independence

Why do we test ?
Primarily to find faults in the software
This can be perceived as being a destructive
process,not constructive
How does this fit with the mind set of the
developer and tester ?

Developer characteristics ?

Specialized
Creative
Valued by the organisation
Trained
Sensitive to criticism

Testers characteristics:

Methodical
Happy when they are finding faults
Under-valued by the organisation
Not always intended to be a tester
Good communicators
Need to be multi-talented
Tact & diplomacy are essential

Is this true always ?

Do Developers & Testers contribute to their

problems ?

Developer Tester Relationship

-Communication is the Key !!
- must be constructive not destructive
- Developers must inform testers of any
changes made to the system
- Testers must report problems to developers
clearly and concisely.
- It is a two-way process !!

Test Independence
Independent testing is more effective.
The author should not test their own work
-Assumptions made are carried into testing
-People see what they want to see
-There can be emotional attachment with
the product
- Were human

 Levels of Independence
Where test cases are
- designed by the person who writes the
software under test
- designed by another person
- designed by people from another department
- designed by people from another
organisation
- not chosen by a person

Re-Testing and Regression Testing

Fault-fixing and re-testing
Regression testing and automation
Selecting regression test cases

Fault-fixing and Re-testing

When ever a fault is detected and fixed,then the software
should be re-tested to ensure that the original fault has
been successfully removed.
The re-run of tests to show that what was not working has
been corrected and is now working satisfactorily.
-

Make your tests repeatable

Normally performed when a fault has been fixed.
Can occur at any level of testing
Should consider testing for similar or related faults

What is Regression testing ?

The re-run of tests with the intention of checking that
what was working is still working
i.e modification have not caused unintended adverse side
defects in the unchanged software
- It is performed whenever the software or its environment
is changed
- When a major modification to the software is done, then
the entire regression pack is likely to be run.
- For a minor changes. Test manager must be selective in
identifying regression tests
- Regression tests are good candidates for test automation

Expected Results
What are expected results ?
-They are expected outcome of a test
-The outcome may include output
-It is the identification of the desired behavior of a test
-They are determined before a test is executed
Outcome is everything you expect to happen and
output is physical output ( ex:Database changes and
table changes)
Outcome of a test may include a physical output.

Why do we need expected results ?

- To ensure that the outcome of a test is as expected
- To make a judgment as to whether to Pass or Fail a test
How do we derive expected results?
- From documentation:
- From existing system
- From specialized user knowledge
- Also known as the Oracle assumption
- Should never be derived from the delivered system
What if we dont have them?
-Testers can make the wrong judgment as to Pass/Fail
-Go back to the Oracle

Prioritisation of tests
Why do we need to Prioritise?
How do we Prioritise? What is the Criteria?

Why do we need to Prioritise ?

- Time constraints
- Resource Constraints
- We need to get the best done in the time
available
How do we Prioritise?
Identification of the Most Important Tests
first(MITs)

Prioritisation basis:
- Business criticality
- Severity of potential failure
- Visibility of failure
- Likelihood of error
- User requirements and priorities
- Technical criticality and complexity
- contractual requirements / Compliance
requirement