SOFTWARE TESTING (SWT301)

FUNDAMENTALS OF TESTING
(CHAPTER 1)
 Content
• What is testing?
• Why is testing necessary?
• Seven testing principles
• Test Process
• The psychology of testing
• Code of ethics

2 / 29
 What is testing?
A method to check whether the actual software product matches
expected requirements and to ensure that software product is defect-
free.
• It involves execution of software/system components using manual
or automated tools to evaluate one or more properties of interest.
• The purpose of software testing is to identify errors, gaps or
missing requirements in contrast to actual requirements.

3 / 29
 What is testing?
Testing is more than running tests
The process consisting of all life cycle
activities, both static and dynamic,
concerned with planning, preparation and
evaluation of software products and related
work products
• To determine that they satisfy specified
requirements
• To demonstrate that they are fit for
purpose
• To detect defects
4 / 29
 What is testing?
Testing is more than verification
• Verification: check for
correctness
Have we built the system right
(correctly)?
• Validation: check if the
software will meet
expectation
Have we built the right system?

5 / 29
 What is testing?
Typical objectives of testing
• Find failures & defects
• Prevent defect
• Evaluate work products
• Provide sufficient information to
stakeholders
• Verify whether all specified requirements
have been fulfilled
• Validate whether the test objective is
complete & work as expected
• Build confidence in the level of quality of
the test object
6 / 29
 Why is testing necessary?
Errors, Defects, and Failures
• Error: a human action that produces an
incorrect result
• Fault: a manifestation of an error in
software
– also known as a defect or bug
– if executed, a fault may cause a failure
• Failure: deviation of the software from its
expected delivery or service (found
Failure is an event; fault is a state of the
defect) software, caused by an error

7 / 29
 Why is testing necessary?
Errors, Defects, and Failures

A person makes
an error ...

… that creates a
fault in the
software ...

… that can cause

a failure
in operation

8 / 29
 Why is testing necessary?
Errors, Defects, and Failures
Four typical scenario of failures

9 / 29
 Why is testing necessary?
Software Testing is Important
• if there are any bugs or errors in the
software, it can be identified early and can
be solved before delivery of the software
product.
• Properly tested software product ensures
reliability, security and high performance
which further results in time saving, cost
effectiveness and customer satisfaction.
Contribute to the overall success of
software
10 / 29
development & maintenance
 Why is testing necessary?
Testing & Quality Management
• Quality management consists of
– Quality assurance ~ process
– Quality control ~ product
• Testing measures software quality
• Testing can find faults; when they are
removed, software quality (and possibly
reliability) is improved
• What does testing test?
– system function, correctness of operation
– non-functional qualities: reliability, usability,
maintainability, reusability, testability, etc.
11 / 29
 Seven testing principles
• Testing shows the presence of defects
• Exhaustive testing is impossible
• Early testing
• Defect clustering
• Pesticide paradox
• Testing is context dependent
• Absence-of-errors fallacy

12 / 29
 Fundamental Test Process
Consists of five parts that
encompass all aspects of
testing
• Planning and control.
• Analysis and design.
• Implementation and
execution.
• Evaluating exit criteria
and reporting.
• Test closure activities.

13 / 29
 Fundamental Test Process
Test Planning & Specification
Output of following three distinct tasks:
1. identify: determine ‘what’ is to be tested
(identify test conditions) and
prioritise
2. design: determine ‘how’ the ‘what’ is to be
tested
(i.e. design test cases)
3. build: implement the tests (test
procedure, data, scripts, etc.)

14 / 29
 Fundamental Test Process
Test Analysis & Design 1/2
Task 1: identify conditions
(determine ‘what’ is to be tested and prioritise)
• list the conditions that we would like to test:
– use the test design techniques specified in the
test plan
– there may be many conditions for each system
function or attribute
– e.g.
• “life assurance for a winter sportsman”
• “number items ordered > 99”
• “date = 29-Feb-2004”
• prioritise the test conditions
– must ensure most important conditions are
covered
15 / 29
 Fundamental Test Process
Test Analysis & Design 2/2
Task 2: design test cases
(determine ‘how’ the ‘what’ is to be tested)
• design test input and test data
– each test exercises one or more test
conditions
• determine expected results
– predict the outcome of each test case, what
is output, what is changed and what is not
changed
• design sets of tests
– different test sets for different objectives such
16 / 29
 Fundamental Test Process
Test Implementation
• prepare test scripts
– less system knowledge tester has the more
detailed the scripts will have to be
– scripts for tools have to specify every detail
• prepare test data
– data that must exist in files and databases at
the start of the tests
• prepare expected results
– should be defined before the test is executed

17 / 29
 Fundamental Test Process
Test Execution
• Execute prescribed test cases
– most important ones first
– would not execute all test cases if
• testing only fault fixes
• too many faults found by early test cases
• time pressure
– can be performed manually or automated

18 / 29
 Fundamental Test Process
Test Recording 1/2
• The test record contains:
– identities and versions of
• software under test
• test specifications
• Follow the plan
– mark off progress on test script
– document actual outcomes from the test
– capture any other ideas you have for new test
cases
– note that these records are used to establish
that all test activities have been carried out as
specified
19 / 29
 Fundamental Test Process
Test Recording 2/2
• Compare actual outcome with expected
outcome. Log discrepancies accordingly:
– software fault
– test fault (e.g. expected results wrong)
– environment or version fault
– test run incorrectly
• Log coverage levels achieved (for
measures specified as test completion
criteria)
• After the fault has been fixed, repeat the
required test activities (execute, design,
20 / 29
 Fundamental Test Process
Test completion criteria
• Check test completion
– Test completion criteria were specified in the test
plan
– If not met, need to repeat test activities, e.g. test
specification to design more tests
• Completion or exit criteria apply to all levels
of testing - to determine when to stop
– coverage, using a measurement technique, e.g.
• branch coverage for unit testing
• user requirements
• most frequently used transactions
– faults found (e.g. versus expected)
– cost or time
21 / 29
 Psychology of Testing
Who could test software?
• Those who wrote the code.
• Members of the same development team.
• Members of a different group
(independent test team).
• Members of a different company (a testing
consultancy/outsourced).

22 / 29
 Psychology of Testing
Test Independence
• Test your own work?
– find 30% - 50% of your own faults
– same assumptions and thought processes
– see what you meant or want to see, not what
is there
– emotional attachment
• don’t want to find faults
• actively want NOT to find faults

23 / 29
 Psychology of Testing
Levels of Independence
• None: tests designed by the person who
wrote the software
• Tests designed by a different person
• Tests designed by someone from a
different department or team (e.g. test team)
• Tests designed by someone from a
different organisation (e.g. agency)
• Tests generated by a tool (low quality tests?)

24 / 29
 Psychology of Testing
Tester’s Rights
– accurate information about progress and
changes
– insight from developers about areas of the
software
– delivered code tested to an agreed standard
– be regarded as a professional (no abuse!)
– find faults!
– challenge specifications and test plans
– have reported faults taken seriously (non-
reproducible)
– make predictions about future fault levels
– improve your own testing process
25 / 29
 Psychology of Testing
Tester’s Responsibilities
• follow the test plans, scripts etc. as
documented
• report faults objectively and factually (no
abuse!)
• check tests are correct before reporting s/w
faults
• remember it is the software, not the
programmer, that you are testing
• assess risk objectively
• prioritise what you report
• communicate the truth
26 / 29
 Psychology of Testing
Who wants to be a tester?
• A destructive process
• Bring bad news (“your baby is ugly”)
• Under worst time pressure (at the end)
• Need to take a different view, a different
mindset (“What if it isn’t?”, “What could
go wrong?”)
• How should fault information be
communicated (to authors and
managers?)

27 / 29
 Code of Ethics
• Public – certified software testers shall consider the wider public
interest in their actions.
• Client and employer – certified software testers shall act in the
best interests of their client and employer (being consistent with the
wider public interest).
• Product – certified software testers shall ensure that the
deliverables they provide (for any products and systems they work
on) meet the highest professional standards possible.
• Judgement – certified software testers shall maintain integrity and
independence in their professional judgement.
• Management – certified software test managers and leaders shall
subscribe to and promote an ethical approach to the management
of software testing.
• Profession – certified software testers shall advance the integrity
and reputation of the profession consistent with the public interest.
• Colleagues – certified software testers shall be fair to and
supportive of their colleagues, and promote cooperation with
software developers.
• Self – certified software testers shall participate in lifelong learning
regarding the practice of their profession, and shall promote an
ethical approach to the practice of the profession.
28 / 29