Professional Documents
Culture Documents
Test Management: Software Testing ISEB Foundation Certificate Course
Test Management: Software Testing ISEB Foundation Certificate Course
1 Principles
2 Lifecycle
4 Dynamic test
5 Management
techniques
3 Static testing
6 Tools
Test Management
Chapter 5
Test Management
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Importance of Independence
No. faults
Release to
End Users
Time
Testing by developers
Pros:
- know the code best
- will find problems that the testers will miss
- they can find and fix faults cheaply
Cons
- difficult to destroy own work
- tendency to 'see' expected results, not actual results
- subjective assessment
Pros:
- some independence
- technical depth
- on friendly terms with buddy - less threatening
Cons
- pressure of own development work
- technical view, not business view
- lack of testing skill
Pros:
- independent view of the software
- dedicated to testing, no development responsibility
- part of the team, working to same goal: quality
Cons
- lack of respect
- lonely, thankless task
- corruptible (peer pressure)
- a single view / opinion
Pros:
- dedicated team just to do testing
- specialist testing expertise
- testing is more objective & more consistent
Cons
- over the wall syndrome
- may be antagonistic / confrontational
- over-reliance on testers, insufficient testing by
developers
Pros:
- highly specialist testing expertise, providing support and help
to improve testing done by all
- better planning, estimation & control from a broad view of
testing in the organisation
Cons
- someone still has to do the testing
- level of expertise enough?
- needs good people skills - communication
- influence, not authority
Pros:
- highly specialist testing expertise (if out-sourced to
a good organisation)
- independent of internal politics
Cons
- lack of company and product knowledge
- expertise gained goes outside the company
- expensive?
Usual choices
Component testing:
- done by programmers (or buddy)
Integration testing in the small:
- poorly defined activity
System testing:
- often done by independent test team
Acceptance testing:
- done by users (with technical help)
- demonstration for confidence
independence is important
- not a replacement for familiarity
different levels of independence
- pro's and con's at all levels
test techniques offer another dimension to
independence (independence of thought)
test strategy should use a good mix
- "declaration of independence
balance of skills needed
Technique specialists
Automators
Database experts
Business skills & understanding
Usability expert
Test environment expert
Test managers
Test Management
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
A definition of Configuration
Management
Configuration Management
Configuration identification
Configuration
Identification
Configuration
Control
Status
Accounting
CI Planning
Configuration
Structures
Selection
criteria
Naming
Conventions
Version/issue
Numbering
Baseline/release
Planning
Configuration
Auditing
Configuration control
Configuration
Identification
Controlled
Area/library
Configuration
Control
Problem/fault
Reporting
Status
Accounting
Configuration
Auditing
Change
Control
Configuration
Control Board
CI
Submission
Investigation
Impact
Analysis
Status/version
Control
Clearance
Authorised
Amendment
Withdrawal/
Distribution
control
Review/
Test
Configuration
Control
Traceability,
impact
analysis
Status
Accounting
Configuration
Auditing
Status Accounting
Database
Procedural
Conformance
Input to SA
Database
CI
Verification
Queries and
Reports
Data
Analysis
Agree with
customer what
has been built,
tested & delivered
test plans
test designs
test cases:
- test input
- test data
- test scripts
- expected results
actual results
test tools
CM
CM is
is critical
critical
for
for controlled
controlled
testing
testing
What would not be under
configuration management?
Live data!
Test Management
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test
Iden
Des
Bld
Ex
Ver
Retest
Practice:
Test
Debug
Retest
Estimating iterations
past history
number of faults expected
- can predict from previous test effectiveness and
previous faults found (in test, review, Inspection)
- % faults found in each iteration (nested faults)
- % fixed [in]correctly
time to report faults
time waiting for fixes
how much in each iteration?
Mike Royce: suspension criteria: when testers spend > 25% time on faults
what would
you do?
tests run
tests passed
what does this
mean?
now
release
date
Diverging S-curve
Possible causes
poor test entry criteria
ran easy tests first
do more debugging
run
passed
action
taken
old release
date
new release
date
run
passed
action
taken
old release
date
new release
date
Case history
Incident Reports (IRs)
200
180
160
140
120
Opened IRs
100
Closed IRs
80
60
40
20
0
04-Jun
24-Jul
12-Sep
01-Nov
21-Dec
09-Feb
Control
"tested"
Test
Phase 1
is it ready for my
testing?
Phase 2
Phase 1
Entry criteria
Exit criteria
Acceptance
criteria
Completion
criteria
poor
clean compiled
programmer claims it is working OK
lots of tests have been run
tests have been reviewed / Inspected
no faults found in current tests
all faults found fixed and retested
specified coverage achieved
better
all tests run after last fault fix, no new faults
rework effort
which faults to be fixed [first]
quality of fixes (entry criteria
to retest)
Test Management
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Incident management
Incidents
What happened
in that week?
report
10
5
dev
cant
reproduce
fault fixed
Test ID
Test environment
Software under test ID
Actual & expected results
Severity, scope, priority
Name of tester
Any other relevant information (e.g. how to
reproduce it)
Severity
- impact of a failure caused by this fault
Priority
company name,
- urgency to fix a fault
board member:
Examples
priority, not severe
- minor cosmetic typo
- crash if this feature is used
Experimental,
not needed yet:
severe, not priority
Incident Lifecycle
Tester Tasks
1 steps to reproduce a
fault
2 test fault or system fault
3 external factors that
influence the symptoms
Developer Tasks
Metrics Exercise
Test Management
Contents
Organisation
Configuration Management
Test estimation, monitoring and control
Incident management
Standards for testing
Test Management