Software Testing

(Ref: D. Gallin Ch. 9 dan R. Patton Ch. 3,4,15,16)

Sony H. Wijaya

Departemen Ilmu Komputer FMIPA IPB

10 November 2023

Fakultas Matematika dan Ilmu Pengetahuan Alam IPB 2023

Software Testing

2
 What is Software testing ?
Examination of a software unit, several integrated software units or
an entire software package by running it.
● execution based on test cases
● expectation – reveal faults as failures
● Failure incorrect execution of the system
● usually consequence of a fault
● Fault/defect/bug result of a human error
 Objective of testing
Directly:
§ To find defects before they cause a production system to fail.
§ To bring the tested software, after correction of the identified defects and
retesting, to an acceptable level of quality.
§ To perform the required tests efficiently and effectively, within the limits
budgetary and scheduling limitation.

Indirectly:
§ To compile a record of software errors for use in error prevention (by
corrective and preventive actions)
 When does a fault occur ?
1. Software doesn't do something that the specification says it should do
2. Software does something that the specification says it shouldn't do
3. Software does something that the specification doesn't mention
4. Software doesn't do something that the specification doesn't mention but
should
5. Software is difficult to understand, hard to use, slow, or – in the software
tester's eyes – will be viewed by the end user as just plain not right
 Testing Axioms
● It's impossible to test a program completely
– large input space
– large output space

– large state space

– large number of possible execution paths

– subjectivity of specifications
 Testing Axioms
Large Input/State space
int exFunction(int x, int y) {
...
}

Exhaustive testing à trying all possible combination of x and y

 Testing Axioms
Large number of possible execution paths
... Number of paths =
for (int i = 0;i<n;++i)
{
if (a.get(i) == b.get(i))
x[i] = x[i] + 100;
else
x[i] = x[i]/2;
With 10-3sec per test case
}
... need since big-bang for n = 36
 Testing Axioms
● Large number of possible execution paths
 Testing Axioms
● Upper limit to total number of tests (Binder)

– n number of decisions
– Li number of times a decision can loop
– X number of decisions that cause loops
– Vi number of all the possible values each input variable
could have
– Y number of input variables.
 Testing Axioms
● Software testing is a risk-based exercice
– need to balance cost and risk of missing bugs
 Testing Axioms
● Testing can't prove the absence of bugs

The more bugs you find, the more bugs there are
 Testing Axioms
● The pesticide paradox
– a system tends to build resistance to a particular testing
technique
 Testing Axioms
● Not all bugs found are fixed
● Difficult to say when a bug is a bug
– only when observed (latent otherwise)

● Product specifications are never final

● Software testers aren't the most popular members of a project
team
● Software testing is a disciplined technical profession that
requires training
 What Testing can accomplish
● Reveal faults that would be too costly or impossible to find
using other techniques

● Show the system complies with its stated requirements for a

given test suite

● Testing is made easier and more effective by good software

engineering practices
 Types of testing
● Unit testing ● System testing - includes
o Black box testing à based on o Function testing
specification o Performance testing
o White box testing à based on
internal logic ● Acceptance testing
o Alpha testing
o Gray box testing à based on
design model o Beta testing

● Integration testing ● Regression testing

 Software Testing Strategies
● Big bang testing

● Incremental testing (unit testing, integration testing, system

testing)
o Bottom-up Horizontally sequenced (breadth-first)
o Top-down Vertically sequenced (depth-first)
 Bottom-up Testing
Stage 4 M11
Integration B Integration C
Stage 3 M9 M10
Integration
A
Stage 2 M8

Stage 1 M1 M2 M3 M4 M5 M6 M7
 Top-down Testing
Integration D
Integration C
Integration B
Integration A M11
Stage 1

Stage 2 M9 M10

Stage 3 M8

Stage 4 M6 M7

Stage 5 M1 M2

Stage 6 M3 M4 M5
 Testing is a Process…
Test Planning
Planning include completion criteria (coverage goal)
Design - approaches for test case selection to
Test Design
achieve coverage goal
Implementation - find for test cases
Test
Implementation
input/output data
state before/after
Test test procedure
Execution
Execution – run tests
Results
Result verification – pass or fail?
Verification Test Library Management – maintain
relationships, keeping track, etc.
Test Library
Management
 Testing Automation
● Use of automated tools is essential
– provide speed, efficiency, accuracy, precision, etc
– allow repeatibility (regression testing)

● Types of tools:
– viewers and monitors (e.g. code coverage tool, debugger)
– drivers and stubs
– stress and load tools
– analysis tools (e.g. file comparison, screen capture and comparison)
– random testing tools (monkeys)
– defect tracking
22
 Testing Automation
Advantages
• Accuracy and completeness of performance.
• Accuracy of results.
• Comprehensiveness of information.
• Few manpower resources required for performing of tests.
• Shorter duration of testing.
• Performance of complete regression tests.
• Performance of test classes beyond the scope of manual testing.

Disadvantages
• High investments required in package purchasing and training.
• High package development investment costs.
• High manpower requirements for test preparation.
• Considerable testing areas left uncovered.
Testing Automation Tools

https://katalon.com/resources-center/blog/automation-testing-tools 25
 terima kasih

Fakultas Matematika dan Ilmu Pengetahuan Alam IPB 2023